Live data from Hacker News

Fly.io Postgres cluster down for 3 days, no word from them about it

webcache.googleusercontent.com

301–310 of 493 posts

Re: Fly.io Postgres cluster down for 3 days, no word from them about it

#301

Earlier quoted context omitted.

I use both and am very satisfied, especially by Hetzner.

Only complaint with Hetzner is they don't have some kind of OAuth setup for machines or scoped API tokens, just read/write. I'd like to use the former for doing Vault authentication from instances, and the latter for writing a dynamic Vault secret provider.

Can’t you use a third party IAM solution for this? Like Okta or keycloak?

Re: Fly.io Postgres cluster down for 3 days, no word from them about it

#302

Earlier quoted context omitted.

Was that an attempt to discredit criticism of Fly's operational processes by pointing out that another company also has issues in how they handle outage notifications?

[flagged]

Was the sarcasm an attempt to discredit criticism of Fly's operational processes by pointing out that another company also has issues in how they handle outage notifications?

Re: Fly.io Postgres cluster down for 3 days, no word from them about it

#303

Earlier quoted context omitted.

I adore DO. They’re seriously underrated. I love how they’ll just give you a server and say here, have at it. No abstractions, no fancy crap, just get out of my way and let me do my thing.

I love their high value content about dev ops, I have learned most of what I know in this field tinkering with a VPS with their great tutorials on how to set up stuff.

They filled the Slicehost vacuum nicely in this area. That's where I got my start in running my own servers about 15 years ago and the tutorials were the driving factor.

Re: Fly.io Postgres cluster down for 3 days, no word from them about it

#304
post #80

Y'all, this is going to be deeply unsatisfying, but it's what I can report personally: I have no earthly clue why this thread on our community site is unlisted. We're looking at the admin UI for it right now, and there's like, a little lock next to do the story, but the "unlist story" option is still there for us to click. The best I can say is: I'm reasonably sure there wasn't some top-down edict to hide this thread…

Thanks for publicly responding to the criticism, that can't be taken for granted. I hope you'll manage to actually address them.

Re: Fly.io Postgres cluster down for 3 days, no word from them about it

#305

Earlier quoted context omitted.

Should losing a single host machine be a big deal nowadays? Instance failure is a fact of life. Even if customers are only running one instance, I would expect the whole thing to rebalance in an automated way especially with fly.io being so container centric. It also sounds like this is some managed Postgres service rather than users running only one instance of their container, so it’s even more reasonable to expect…

If you lose a single instance on RDS and you don't have replication set up, you'll also have downtime. (Maybe not with Aurora?) And +1 to the sibling comment; Fly makes it very clear that single instance postgres isn't HA, and talks about what you need to do architecturally to maintain uptime.

> Maybe not with Aurora

If a read replica fails, I'd expect no downtime (possibly a few errors as connections get cut off abruptly). Although there's always the risk that the remaining instances aren't able to handle the additional load.

If the master fails, you'll get a ~2min downtime

Re: Fly.io Postgres cluster down for 3 days, no word from them about it

#306

Earlier quoted context omitted.

Downtime but limited downtime since the data is stored with redundantly across multiple machines in the same AZ. So unless the AZ goes down (which is a different failure than what happened here) you can restart the DB on a different instance pretty quickly and I'm guessing AWS will do it automatically for you. edit: Remove triple as not certain about level of redundancy

I don't believe their RDS / EBS has 3x redundancy. With SSD, that would be super costly for them. But if that's correct, that would be incredible.

May not be 3x but it is replicated so even a total instance failure would not make you lose data:

>Amazon EBS volumes are designed to be highly available, reliable, and durable. At no additional charge to you, Amazon EBS volume data is replicated across multiple servers in an Availability Zone to prevent the loss of data from the failure of any single component. For more details, see the Amazon EBS Service Level Agreement.

https://aws.amazon.com/ebs/features/#Amazon_EBS_availability...

Re: Fly.io Postgres cluster down for 3 days, no word from them about it

#307
post #99

Earlier quoted context omitted.

(a) Not even close to the first negative HN thread about us. (b) We definitely didn't make the thread private in response to HN. (c) It should be public again.

what's up with the status page?

Status pages are usually for marketing purposes.

Why would anyone want to become a new customer if all they see is jumble of green, yellow and red?

Green status pages attract business.

Re: Fly.io Postgres cluster down for 3 days, no word from them about it

#308

Wondering if for small/bootstrapped projects there's any alternative people suggest? Fly has a nice UX and accessible prices, but it's unstable at best. I use the big clouds at work, but for personal they are $$$. Also I want to keep devops tending asymptotically to zero.

Although, i have never used them, you can explore railway.app. it is the closest to fly.io and never heard any bad things. I personally at the moment use digitalocean without any issues, but there's always the maintenance overhead of managing a server yourself.

I've been using a Postgres DB on Railway's free plan (that is going away) and it was great. It did everything I wanted (excluding external access and PostGIS) for cents. The support community is nice.

I didn't use it for much more, but my experience has been great. They deserve way more air time than they currently get.

Re: Fly.io Postgres cluster down for 3 days, no word from them about it

#309

Earlier quoted context omitted.

Ok as long as we’re getting conspiratorial, something similar I observed has bugged me. About a year ago fly awarded a few people in the forums, I think it was 3, the “aeronaut” badge. Basically just pointless bling for a “routinely very helpful” person or somesuch. Still, I can imagine it was cool to get it. No, it wasn’t me. One person I saw with it absolutely deserved it: this person is, to this day, always hoppin…

Get out of here with this nonsense. We tell people when we’re a bad option all the time. Do you really think we have a desire (or time) to punish somebody for doing the same? Also, here’s the long forgotten badge, still with 3 people… https://community.fly.io/badges/107/aeronaut

Why are you acting so hostile? If you don't like that the community is dunking on you, then maybe posting on Hacker News isn't for you.

Re: Fly.io Postgres cluster down for 3 days, no word from them about it

#310
post #66

Wondering if for small/bootstrapped projects there's any alternative people suggest? Fly has a nice UX and accessible prices, but it's unstable at best. I use the big clouds at work, but for personal they are $$$. Also I want to keep devops tending asymptotically to zero.

I use Dokku on top of Hetzner for my hobby projects - hosting is super cheap, for a little extra I can add a mounted volume for storage, and if the project outgrows a single server I can always just break out of Dokku and use some Docker containers behind a load balancer. If you are outside of Europe, Digital Ocean or Linode may work better for you.

I like Hetzner, they certainly radiate the feeling of quality (the management UI is great, for instance). The servers themselves are competitively priced (and they have ARM boxes!) - but for more storage than the little that they include I find the price pretty outrageous, compared to the base price, anyway. You'd end up about doubling the price for a "reasonable" amount of storage you can confidently run your base system on.
Post reply on HN