Trust me, Heroku is not costly compared to setting up a server: $ git push master heroku is a lot cheaper then: Setup a machine. Then setup mysql, nginx, REE 1.8.6, or did you go with MRI 1.9.1? You're not gonna launch on Rails3 right? Because then you can't do 1.9.1, you have to go to 1.9.2. And you know how to setup nginx to pipe requests through to a Rails app. Are you going with Passenger or Unicorn? Oh make sure…
If your startup has any challenging technical elements, I think having 100% full control over your app architecture is quite important. You can live without it, but it's a potentially costly trade-off.