I am surprised nobody so far has mentioned AWS as a hosting provider. You can start with their free micro instance plan for one year (http://aws.amazon.com/free) and resize the server as needed. Having the ability to take a snapshot of the full server (you need to use EBS) and clone it to test changes before going into production or just for backup purposes gives me a peace of mind that I never had with dedicated servers (even if they had RAID). Plus you can get a myriad of services around EC2, like managed MySQL (RDS) that you cannot get from traditional VPS hosting providers. (Disclaimer, I created http://bitnami.org/ and we work a lot with Amazon)