Live data from Hacker News

Fly.io Status – Consul cluster outage

status.flyio.net

11–20 of 123 posts

Re: Fly.io Status – Consul cluster outage

#11
post #2

They seem to have a lot of issues with Consul, is it the design of Consul or the way they use it that is the problem?

Both, though the latter is likely due to marketing/promises from HashiCorp. Consul (and the entire hashicorp stack, really) is overengineered, under-optimized, and generally terrible to use at any scale beyond "small".

Re: Fly.io Status – Consul cluster outage

#12

Been a fan of fly and have had most, if not all, of my side and semi-side projects on there for some time now. But...the ratio of good/fun/snarky blog posts to reliable service has gotten a bit too large for me, starting to look for other providers at this point just in case they can't turn this trend around. Honestly been a good object lesson for me in the importance of backing up marketing/hype/"mind-share" stuff w…

(unpkg seems to be up now)

Re: Fly.io Status – Consul cluster outage

#13
post #8
post #2

They seem to have a lot of issues with Consul, is it the design of Consul or the way they use it that is the problem?

The former ish -- they relied on consul marketing that the hammer fit the square hole. Hashicorp has been pretty bad about marketing themselves as the right tool for any job, but they really only fit the narrowest of tasks before you find yourself needing an alternative or being compelled to buy a support contact. It's atlassian from Arkansas, just faster

Can you cite this? Because it seems like the opposite is true: https://news.ycombinator.com/item?id=35048318

Re: Fly.io Status – Consul cluster outage

#14
I love this update:

“ We are working to build a new Consul cluster with 10x the RAM. We aren't yet sure, but believe a routine DNS change might have created a thundering herd problem causing Consul servers to immediately increase RAM usage by 500%. This is not ideal.”

_This is not ideal._

Re: Fly.io Status – Consul cluster outage

#16
post #2

They seem to have a lot of issues with Consul, is it the design of Consul or the way they use it that is the problem?

Roblox had a massive 3-day outage [1] in October 2021 due to a Consul feature that didn’t work as expected.

My gut with Consul is don’t use it for high-load distributed services.

[1] https://blog.roblox.com/2022/01/roblox-return-to-service-10-...

Re: Fly.io Status – Consul cluster outage

#17

Been a fan of fly and have had most, if not all, of my side and semi-side projects on there for some time now. But...the ratio of good/fun/snarky blog posts to reliable service has gotten a bit too large for me, starting to look for other providers at this point just in case they can't turn this trend around. Honestly been a good object lesson for me in the importance of backing up marketing/hype/"mind-share" stuff w…

Relevant response from the Fly community forums: https://community.fly.io/t/frequent-outages-is-really-demons...

Re: Fly.io Status – Consul cluster outage

#18
Fly is building everything in hard mode - since they are not layering on top of an existing cloud like pretty much everyone else (heroku, render, railway, ...).

It's either very smart (if they pull it off) because they will have a ginormous cost advantage or they fail.

I'm personally of the opinion that the ux on top of aws/gcp/... is worse than a doo-doo in a shoe. However, they are as stable as can be (all complex systems go down once in a while). There are very few mature projects that do not rely on aws/gcp/... managed services anyway. Might as well put in the little bit of effort to set yourself up for the future instead of painful migrations. This obviously doesn't hold for hobby projects.

In any case, I have a lot of respect for the engineering that fly does. Kudos.

Re: Fly.io Status – Consul cluster outage

#19

Been a fan of fly and have had most, if not all, of my side and semi-side projects on there for some time now. But...the ratio of good/fun/snarky blog posts to reliable service has gotten a bit too large for me, starting to look for other providers at this point just in case they can't turn this trend around. Honestly been a good object lesson for me in the importance of backing up marketing/hype/"mind-share" stuff w…

Relevant response from the Fly community forums: https://community.fly.io/t/frequent-outages-is-really-demons...

Yeah, I saw; I've kept up w/ everything pretty closely. Still decently frustrating as a paying customer, but I hope they can figure it out. If they can and can show some real reliability, I'll be an even bigger fan.
Post reply on HN