Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fly.io Postgres cluster down for 3 days, no word from them about it

webcache.googleusercontent.com

Post reply on HN
Print view View original

21–30 of 493 posts

Re: Fly.io Postgres cluster down for 3 days, no word from them about it

#21

Post by bongobingo1 »

Fire-Dragon-DoL wrote: post #20
gowthamgts12 wrote: post #12

Earlier quoted context omitted.

Although, i have never used them, you can explore railway.app. it is the closest to fly.io and never heard any bad things. I personally at the moment use digitalocean without any issues, but there's always the maintenance overhead of managing a server yourself.

I wish digitalocean offered decent pricing for spaces (s3). Unfortunately it starts at 5$, which is an enormous price for storing 70 small images, but s3 would greatly simplify my server management moving state entirely outside the server (managed database + managed object storage)

> price for storing 70 small images

Do you have to use an object store in that case? Or does it have to be separate from whatever application instance?

Re: Fly.io Postgres cluster down for 3 days, no word from them about it

#22

Post by skullone »

jssjr wrote: post #15
dcchambers wrote: post #4

I like fly.io a lot and I want them to succeed. They're doing challenging work...things break. Have to admit it's disappointing to hear about the lack of communication from them, especially when it's something the CEO specifically called out that they wanted to fix in his big reliability post to the community back in March. https://community.fly.io/t/reliability-its-not-great/11253#s...

I appreciate the honest feedback. We could have done better communicating about the problem. We've been marking single host failures in the dashboard for affected users and using our status page to reflect things like platform and regional issues, but there's clearly a spot in the middle where the status we're communicating and actual user experience don't line up. We've been adding a ton more hardware lately to stay…

What hardware are you buying? Across tens of thousands of physical nodes in my environment, only a few would have "fatal" enough problems that required manual intervention per year. Yes we had hundreds of drives die a year, some ECC ram would exceed error thresholds, but downtime on any given node was rare (aside from patching, but we'd just live migrate KVM instances around as needed.

Re: Fly.io Postgres cluster down for 3 days, no word from them about it

#25

Post by bongobingo1 »

spiderice wrote: post #14

There is now a response to the support thread from Fly[1]: > Hi Folks, > Just wanted to provide some more details on what happened here, both with the thread and the host issue. > The radio silence in this thread wasn’t intentional, and I’m sorry if it seemed that way. While we check the forum regularly, sometimes topics get missed. Unfortunately this thread one slipped by us until today, when someone saw it and flag…

Seems like the OP should have made a HN thread in the first place instead of posting to community.stri^H^H^H^Hfly.io

Re: Fly.io Postgres cluster down for 3 days, no word from them about it

#26

Post by iampims »

Fire-Dragon-DoL wrote: post #20
gowthamgts12 wrote: post #12

Earlier quoted context omitted.

Although, i have never used them, you can explore railway.app. it is the closest to fly.io and never heard any bad things. I personally at the moment use digitalocean without any issues, but there's always the maintenance overhead of managing a server yourself.

I wish digitalocean offered decent pricing for spaces (s3). Unfortunately it starts at 5$, which is an enormous price for storing 70 small images, but s3 would greatly simplify my server management moving state entirely outside the server (managed database + managed object storage)

You could use Cloudflare R2, it's pretty cheap overall.

Re: Fly.io Postgres cluster down for 3 days, no word from them about it

#27

Post by unmole »

xyzzy_plugh wrote: post #7

I think fly.io is pretty incredible but I can't help but feeling they're doomed to follow in heroku's footsteps (unclear if good or bad). They've built some pretty wild stuff and I can't help but wonder if they're overcooking the ocean instead of just solving problems for their users. Durable and available storage are all they really need to draw me away from big cloud providers but this combined with their answer to…

> use S3 or run minio

Is using Cloudflare R2 not an option?

Re: Fly.io Postgres cluster down for 3 days, no word from them about it

#28

Post by pech0rin »

I really want to love Fly.io. It's super easy to get setup and use, but to be honest I don't think anyone should be building mission critical applications on their service. I ended up migrating everything over to AWS (which I reallllly didn't want to do) because:

* Frequent machines not working, random outages, builds not working

* Support wasn't responsive, didn't read my questions (kept asking same questions over and over again) -- I paid for a higher tier specifically for support.

* General lack of features (can't add sidecars, hard to integrate with external monitoring solutions)

* Lack of documentation -- For happy path its good but any edge cases the documentation is really lacking.

Anyway, for hobby projects its fine and nice. I still host a lot of personal projects there. But I have to move my companies infrastructure off of it because it ended up costing us too much time/frustration, etc. I really had high hopes going into it as I had read it was a spiritual successor of sorts to Heroku which was an amazing service in its day, but I don't think its there yet.

Re: Fly.io Postgres cluster down for 3 days, no word from them about it

#29

Post by jdkfoo »

mrcwinn wrote: post #13
dcchambers wrote: post #4

I like fly.io a lot and I want them to succeed. They're doing challenging work...things break. Have to admit it's disappointing to hear about the lack of communication from them, especially when it's something the CEO specifically called out that they wanted to fix in his big reliability post to the community back in March. https://community.fly.io/t/reliability-its-not-great/11253#s...

Yes, this. It's tough when you've already played your "we messed up but we're making it right" card, and then you continue to not have it right.

Hosting service that cannot get basics right after a decade plus of solving these problems as an industry.

Are we even trying or just repeating ourselves because we don’t know what else to do?

How can the entire industry keep making the same basic errors?

“Let’s keep it simp… ohh nope we invented a Turing complete language and customer service is terri… wait do we have customer service?”

I get the world turning against SaaS lately.

Computers are so fast now, enthusiasts would be better served DIY; put a beige box in a local colo, use one of the big 3 for big business.

This is just starting to look disreputable and disrespectful to humanity itself putting such resources into one time bomb after another.

Re: Fly.io Postgres cluster down for 3 days, no word from them about it

#30

Post by windexh8er »

marban wrote: post #8

They're less humble in communicating other things https://fly.io/blog/we-raised-a-bunch-of-money/

I tried Fly once, but, at the end of the day it seemed way too expensive for what it was and the completeness of the vision. And then I started to see the complaints in random corners of the Internet.

I don't read their blog regularly but I always thought they had great content. But not after reading this.

The irony: "What people actually wanted to talk about, though? Databases."

...but apparently not when they are the problem behind said databases?

Post reply on HN