RubyGems and Linux (especially Debian / Ubuntu) are like oil and water.
Don't install Ruby libraries through apt-get, and also don't install them through RubyGems globally (i.e. never use "sudo" for installing gems).
Instead install gems for the local user only, with the help of RVM. Then to upgrade, if everything breaks, you can just delete your RVM directory and start from scratch again.
Of course, this brings with it a whole other can of worms, because then it's your responsibility for upgrading your libraries with the latest security-related fixes (which otherwise would have been updated with a simple "aptitude safe-upgrade", which can be a cronjob or something). But then you've got the same responsibility on Heroku.
Don't install Ruby libraries through apt-get, and also don't install them through RubyGems globally (i.e. never use "sudo" for installing gems).
Instead install gems for the local user only, with the help of RVM. Then to upgrade, if everything breaks, you can just delete your RVM directory and start from scratch again.
Of course, this brings with it a whole other can of worms, because then it's your responsibility for upgrading your libraries with the latest security-related fixes (which otherwise would have been updated with a simple "aptitude safe-upgrade", which can be a cronjob or something). But then you've got the same responsibility on Heroku.
See http://beginrescueend.com/