Live data from Hacker News

Heroku - Bamboo Routing Performance

blog.heroku.com

11–20 of 152 posts

Re: Heroku - Bamboo Routing Performance

#11
Working closely with our customers to develop long-term solutions

Of the five action items they listed, it seems that only the last of them is about actually solving the problem. I hope they are committed to it - better visibility of the problem can help, but I'd rather not have the problem in the first place.

Re: Heroku - Bamboo Routing Performance

#12
post #2

Interesting -- he seems to be saying that they'll explain all about the problem, but not do anything about it.

That's not fair. What do you think "tools to understand and improve the performance of your apps" and "develop long-term solutions" from his bullets mean?

But, I'm surprised they didn't wait until the "in-depth technical review" was available to apologize. And the idea that they were informed of a problem "yesterday" doesn't quite match the impression RapGenius gave, that they'd been discussing this with Heroku support for a while.

Re: Heroku - Bamboo Routing Performance

#13
post #6

Wait, so those guys were on Bamboo, and complaining? Fuck, that is so not cool. We've been on cedar ever since it launched, and been running puma threads or unicorn workers. The idea of one dyno per request is bullshit, and I wasn't sure if they were on cedar or not. A dyno is an allocated resource (512mb, not counting db, k/v store etc) How ballsy of them to complain when they are doing it wrong.

Does Cedar solve the problem? Random routing can still end up overloading a dyno. What's the solution?

Re: Heroku - Bamboo Routing Performance

#14
It's so refreshing to see this kind of communication. I don't use Heroku, and don't know much about this specific issue, but they're responses to downtime and complaints have been so direct and BS-free that I'll definitely consider them when I need a PaaS.

Re: Heroku - Bamboo Routing Performance

#15
I feel like there is an answer for this, but why are two companies in the "YC family" at odds so publicly? If RapGenius is "starting beef" like is done in the music industry, I find it odd that it would happen with someone on their own "label".

Perhaps this is ignorance on my behalf of how companies who have already been sold (Heroku) fit into the picture, but some explanation would be appreciated.

Re: Heroku - Bamboo Routing Performance

#16
post #2

Interesting -- he seems to be saying that they'll explain all about the problem, but not do anything about it.

The first couple bullet points are about increasing transparency into how it works and what it is doing in an app. The last point is about fixing it.

| Working closely with our customers to develop long-term solutions

I cannot imagine this will be simple to fix. I don't know what Heroku's deployment is like^, but I figure it is very massive and complex and solving performance issues at that scale are not done overnight.

^That is the deployment and configuration of the Heroku platform, not how we as devs deploy to Heroku.

Re: Heroku - Bamboo Routing Performance

#17
post #6

Wait, so those guys were on Bamboo, and complaining? Fuck, that is so not cool. We've been on cedar ever since it launched, and been running puma threads or unicorn workers. The idea of one dyno per request is bullshit, and I wasn't sure if they were on cedar or not. A dyno is an allocated resource (512mb, not counting db, k/v store etc) How ballsy of them to complain when they are doing it wrong.

[deleted]

Re: Heroku - Bamboo Routing Performance

#18
It's a good response in that they are taking responsibility, but it is pretty obvious that they are reluctant to say anything about a fix. In my mind, "it's hard" isn't a valid excuse in this case, especially when there are relatively straightforward solutions that will solve this at a practical level. For example, you could imagine a naive form of intelligent routing that would work simply by keeping a counter per dyno:

- request comes in and gets routed to the dyno with the lowest count. Inc the count.

- response goes out. Dec the counter.

Since they control the flow both in and out, this requires at most a sorted collection of counters and would solve the problem at a "practical" level. Is it possible to still end up with one request that backs up another one or two? Sure. Is it likely? No. While this isn't as ideal as true intelligent routing, I think it's likely the best solution in a scenario where they have incomplete information about what a random process on a dyno can reliably handle (which is the case on the cedar stack).

Alternatively, they could just add some configuration that allows you to set the request density and then you could bring intelligent routing back. The couple of milliseconds that lookup/comparison would take is far better than the scenario they're in now.

EDIT: I realized my comment could be read as though I'm suggesting this naive solution is "easy". At scale it certainly isn't, but I do believe it's possible and as this is their business, that's not a valid reason to do what they are.

Re: Heroku - Bamboo Routing Performance

#20
post #15

I feel like there is an answer for this, but why are two companies in the "YC family" at odds so publicly? If RapGenius is "starting beef" like is done in the music industry, I find it odd that it would happen with someone on their own "label". Perhaps this is ignorance on my behalf of how companies who have already been sold (Heroku) fit into the picture, but some explanation would be appreciated.

Heroku has been owned by Salesforce.com since dec. 2010
Post reply on HN