Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I left Heroku for OpsWorks and shaved 40% off response times

stefanwrobel.com

Post reply on HN
Print view View original

11–20 of 53 posts

Re: I left Heroku for OpsWorks and shaved 40% off response times

#11

Post by incision »

One limitation with OpsWorks is that as far as I know, there's no option to use spot instances whereas it's not a problem with EC2 launch configurations for AutoScaling Groups. Also, ASGs and OpsWorks both have options for time and load-based scaling with different interfaces.

It's easy enough to understand how this happened as OpsWorks came about via an acquisition, but consistency would certainly be nice.

Presenting overlapping yet incompatible features under the blanket of AWS is confusing and potentially pretty frustrating.

Re: I left Heroku for OpsWorks and shaved 40% off response times

#13

Post by imperialWicket »

sync wrote: post #6
andrewvc wrote: post #2

Opsworks is nice, and it's a good packaged solution. A large number of smaller apps would benefit from a switch. It's also more fairly priced. That being said, opsworks is still based on chef, which is still cumbersome. For most people Ansible is far easier to deal with if they don't fit into the opsworks box. The overhead of starting from scratch with ansible can be made up for with the faster overall development ti…

Seems like Opsworks does a lot more than just the provisioning, no? "AWS OpsWorks can scale your application using automatic load-based or time-based scaling and maintain the health of your application by detecting failed instances and replacing them. You have full control of deployments and automation of each component." How would you suggest using Ansible to cover that side of things?

This just introduces additional tools/products to the stack, but you can use Ansible with the Netflix OSS stack, such that Ansible provisions your AMIs and Asgard manages auto-scaling and deployment (with very fine-grained control).

Re: I left Heroku for OpsWorks and shaved 40% off response times

#14

Post by nthj »

iagooar wrote: post #9

While I can understand the advantages the author of the article mentions, I must say that one should think about making the move to OpsWorks, because you can't compare their prices without considering other costs / tradeoffs. A friend of mine and I have been working on a startup for the last 8 months. It has been tough as we have been working full time for a company and hacking like crazy once we got home, or on week…

I'd be really interested to know more about how the switch shaved 40% off of the response times. What was the major bottleneck? RAM? CPU? Network IO to PostgreSQL? Network IO to memcache?

Heroku is offering 2X and soon, 4X dynos; did he experiment with these?

The author also sounds rather excited about nginx serving static assets. The Rack::Cache problem on Heroku is easily solved with a CDN layer (and Heroku even recommends you do so.) (Cult Cosmetics /is/ set up with CloudFront, so the author obviously knows this.)

There are a lot of Rails architecture smells in this article, such as worrying about the 60-second boot timeout and the 30-second request timeout. [1] I have to wonder if the time invested in learning Chef and managing the architecture may have been spent making the app more performant.

Developer time is really, really expensive. Not only can you bill $100-$150 hour instead of learning how to use Chef, you could be shipping features for your own products, which can bring you passive income forever. A lot of companies are successfully running on Heroku, and it seems strange to throw everything out for the explained reasons.

[1] Specifically, adding 'require: false' to your Gemfile and only loading heavy libraries when you need them greatly helps with boot time. Ideally, you won't run any image processing libraries (like RMagick) except in background jobs, and your main app should not even need these libraries available. Same goes for PDF generation, scraping technology, or any other heavy processing.

The 30-second timeouts either need to be handled via background jobs (the author mentioned they use sidekiq) or uploading directly to S3, side-stepping the web process entirely.

Re: I left Heroku for OpsWorks and shaved 40% off response times

#16

Post by swrobel »

nthj wrote: post #14
iagooar wrote: post #9

While I can understand the advantages the author of the article mentions, I must say that one should think about making the move to OpsWorks, because you can't compare their prices without considering other costs / tradeoffs. A friend of mine and I have been working on a startup for the last 8 months. It has been tough as we have been working full time for a company and hacking like crazy once we got home, or on week…

I'd be really interested to know more about how the switch shaved 40% off of the response times. What was the major bottleneck? RAM? CPU? Network IO to PostgreSQL? Network IO to memcache? Heroku is offering 2X and soon, 4X dynos; did he experiment with these? The author also sounds rather excited about nginx serving static assets. The Rack::Cache problem on Heroku is easily solved with a CDN layer (and Heroku even re…

I was using 2X dynos on Heroku. I did have memory issues and had to continually reduce the number of unicorns that I was running. I think CPU and disk were the major bottlenecks, but again, Heroku makes it hard to track down exactly what the cause is.

Also, we're an ecommerce site so we rely on a lot of external services (specifically for payments) where we have to run the call in the web process ... hence the issue with 30s timeouts.

Re: I left Heroku for OpsWorks and shaved 40% off response times

#17

Post by ericd »

Based on his mention of SSD-backed instances (c3.large) being only $100/mo, I looked at EC2 pricing again to see if it had become more competitive. Turns out it only has 2x16 gigs of storage and 1 physical core.

Looking at the c3.8xlarge instances, they're specced pretty much exactly like the servers I just built for $9k/each, except they have half the ram (60 vs. 128) and half the storage (2x320 vs 4x320). And they cost the same to rent for 6 months as it does to buy one, including coloing.

So, if you need more than a fraction of a physical server, EC2 still loses pretty hard on the pricing. It seems as though it has become competitive with Softlayer, though, at least.

EDIT: Ah, but if you pay $11.6k upfront to reserve it for three years, it reduces the monthly bill from $1728 to ~$370, which is a much better deal than Softlayer offers. That's actually a pretty reasonable premium compared to racking them yourself, though you lose flexibility and the hardware is worse than what you'd get yourself. Well played, Amazon.

Re: I left Heroku for OpsWorks and shaved 40% off response times

#19

Post by ghayes »

darklajid wrote: post #7
andrewvc wrote: post #2

Opsworks is nice, and it's a good packaged solution. A large number of smaller apps would benefit from a switch. It's also more fairly priced. That being said, opsworks is still based on chef, which is still cumbersome. For most people Ansible is far easier to deal with if they don't fit into the opsworks box. The overhead of starting from scratch with ansible can be made up for with the faster overall development ti…

Ansible: I was in the "Don't go there" camp a while ago - they documentation was horrible, the whole website a mess and lots of things weren't discoverable at all (people using ansible told me that they used google site search to find stuff in the manual). I returned a couple of days ago to look at the current state of things and I have to admit that I was impressed. Technically I really liked ansible for quite some…

I was really excited by ansible, but I quickly found out that using `register: result` and `when: result|success` throughout so many playbooks meant I was back to the mess that is understanding the flow of a chef script that you didn't personally write.

Maybe that's not a problem since I rarely found an open source ansible playbook for anything I needed and had to always write my own.

Re: I left Heroku for OpsWorks and shaved 40% off response times

#20

Post by mercurial »

darklajid wrote: post #7
andrewvc wrote: post #2

Opsworks is nice, and it's a good packaged solution. A large number of smaller apps would benefit from a switch. It's also more fairly priced. That being said, opsworks is still based on chef, which is still cumbersome. For most people Ansible is far easier to deal with if they don't fit into the opsworks box. The overhead of starting from scratch with ansible can be made up for with the faster overall development ti…

Ansible: I was in the "Don't go there" camp a while ago - they documentation was horrible, the whole website a mess and lots of things weren't discoverable at all (people using ansible told me that they used google site search to find stuff in the manual). I returned a couple of days ago to look at the current state of things and I have to admit that I was impressed. Technically I really liked ansible for quite some…

The doc is indeed pretty good, and available offline (ansible-doc). I quite like that it uses a very declarative syntax, and it's relatively easy to get idempotent operations. And the fact that you don't need to install anything on the machine you administer is definitely a strong point.

One downside is that I can see how tracking down role dependencies would be tricky if you had many different roles. I also haven't tried at scale.

Post reply on HN