Live data from Hacker News

Why I left Heroku, and notes on my new AWS setup

holovaty.com

131–140 of 231 posts

Re: Why I left Heroku, and notes on my new AWS setup

#131
post #59

> The way we set up Soundslice is relatively simple. We made a custom AMI with our code/dependencies, then set up an Elastic Load Balancer with auto-scaling rules that instantiate app servers from that AMI based on load. Doesn't sound that simple to me (as a complete sysadmin noob). Somebody should write a book about this.

I pitched a book to No Starch and a few others, and they all said there wasn't enough demand...

Leanpub.

Re: Why I left Heroku, and notes on my new AWS setup

#132

> The way we set up Soundslice is relatively simple. We made a custom AMI with our code/dependencies, then set up an Elastic Load Balancer with auto-scaling rules that instantiate app servers from that AMI based on load. Doesn't sound that simple to me (as a complete sysadmin noob). Somebody should write a book about this.

Would you be interested in a blog post about it? I'm a sysadmin/devops with 12 years in, and build stuff like this on a daily basis. I wouldn't mind sharing how the sausage is made.

yes please.. ohh god yes.

Re: Why I left Heroku, and notes on my new AWS setup

#133
post #57

This really resonates with me. I have been having identical issues with Dotcloud, a Heroku competitor. There's nothing worse than wondering if deploying your stupid one-character typo fix will hang or leave the app in an incomplete state. I myself am working on setting up salt stack for deployments. I like that all of it is in python, and after a couple of days I've begun to make progress. I am migrating things slowl…

Does having an identical staging environment help, or are these deploy errors transient/non-deterministic?

Re: Why I left Heroku, and notes on my new AWS setup

#134
post #57

This really resonates with me. I have been having identical issues with Dotcloud, a Heroku competitor. There's nothing worse than wondering if deploying your stupid one-character typo fix will hang or leave the app in an incomplete state. I myself am working on setting up salt stack for deployments. I like that all of it is in python, and after a couple of days I've begun to make progress. I am migrating things slowl…

Does having an identical staging environment help, or are these deploy errors transient/non-deterministic?

Re: Why I left Heroku, and notes on my new AWS setup

#135
post #50

When exactly do you bake a new AMI? On every Soundslice deploy?

No, definitely not on every deploy! For a deploy, I just do a "git pull" on all of the production boxes and restart the web server. Baking a new AMI only happens when there are new underlying dependencies, like a new package from apt-get or a new Python module from pip. In other words, it's rare. Hope that helps!

Is there any reason for not letting pip update things automagically from your `prod_requirements.txt` using a simple fabric `fab update_pip` when necessary?

Re: Why I left Heroku, and notes on my new AWS setup

#136

We're working on solving these sorts of devops problems in Ubuntu with Juju: https://juju.ubuntu.com/ As an alternative approach would be that a Juju charm (script) would handle the initial deployment of a stock Ubuntu AMI and the customization in one step (or with puppet/chef) and then allow you to add new instances based on scale (though currently not automatic). When you have changes to your service you update the…

FWIW, I tested juju a few months ago and found it to be buggy and unreliable. Sometimes the instances would connect together correctly, and sometimes they would fail inexplicably. Didn't seem ready for any kind of production use to replace config mgmt tools.

If you are looking for another general purpose orchestrator, I'd suggest taking a look at Ansible - http://ansible.cc/

Disclaimer: I am the primary author.

Re: Why I left Heroku, and notes on my new AWS setup

#137
post #116

Earlier quoted context omitted.

I have no idea, just things don't work on EC2. > Do your VPSs already have things installed or a GUI? Nope. I prefer straight-up Arch linux. ssh in and go from there. To add some concrete-ness to the mix, I was installing ejabberd. When it came time to ping the server... no response. I did the exact same steps on my Digital Ocean VPS and everything went fine. I had done whatever commands EC2 expects to open the right…

For EC2 security groups you have to open access to both the correct ports and protocols. Ports are a concept at layer 4 of the OSI model, while ping, or more correctly 'ICMP Echo Request', is lower down the TCP/IP stack at layer 3. So when configuring the security group, look for the option to choose Protocols, then enable ICMP :)

Thanks. This is exactly the kind of stuff that I could learn from said blog post, and why I think "EC2 == VPS" is false.

The documentation I was following made no mention of this at all.

Re: Why I left Heroku, and notes on my new AWS setup

#138
post #119
post #45

Earlier quoted context omitted.

That's actually, honestly, a bad way to do it. It's fine to pre-load an instance with source and application/package dependencies. At this point though, you really should be backing those up with it being deployed by Puppet/Chef. Essentially you pre-run a puppet manifest and have it do all the first-run processing and then store THAT as the AMI/instance. This way you have an instance that only needs a few seconds to…

Puppet/Chef is overkill for a site running a config this simple.

What do you recommend for simple configurations where you still want the repeatability/documentation/versioning tools like Chef/Puppet provide? Shell scripts?

Re: Why I left Heroku, and notes on my new AWS setup

#139
I love Heroku: git deployment, the "dyno" abstraction, Procfiles, buildpacks, putting configuration in environment variables (all the 12 Factor App stuff: http://www.12factor.net/).

Is there an open source implementation of Heroku (close to 100% compatible, not just similar ideas) that runs on your own cloud?

Re: Why I left Heroku, and notes on my new AWS setup

#140
post #119
post #45

Earlier quoted context omitted.

That's actually, honestly, a bad way to do it. It's fine to pre-load an instance with source and application/package dependencies. At this point though, you really should be backing those up with it being deployed by Puppet/Chef. Essentially you pre-run a puppet manifest and have it do all the first-run processing and then store THAT as the AMI/instance. This way you have an instance that only needs a few seconds to…

Puppet/Chef is overkill for a site running a config this simple.

I don't really agree with this. If you're defining a whole crazy master/agent relationship with Puppet then it's overkill for them, no doubt.

There's a reason many Vagrant boxes I see use Puppet to instantiate though: It's friggin simple to get started with. More advanced configurations might even end up requiring you to build your own modules and extended manifests, but I expect this configuration detailed in the article would probably amount to a couple hundred lines for the entire manifest.

I think this also underscores one of the things I like the most about Puppet (instead of Chef, which is a fine CM system as well); It's easy to get started with but powerful enough to get things done down the road. Honestly, there's never a better time to start working with Puppet than from the very beginning of a project!

Post reply on HN