Well that's not gonna fly.
Fly.io Postgres cluster down for 3 days, no word from them about it
61–70 of 493 posts
Re: Fly.io Postgres cluster down for 3 days, no word from them about it
#62Wondering if for small/bootstrapped projects there's any alternative people suggest? Fly has a nice UX and accessible prices, but it's unstable at best. I use the big clouds at work, but for personal they are $$$. Also I want to keep devops tending asymptotically to zero.
I’m quite happy with https://render.com after leaving Heroku
i've also had success with render.com so far! been running an app & DB for $14/mo for a almost 6 months and it's been solid.
Re: Fly.io Postgres cluster down for 3 days, no word from them about it
#63I think fly.io is pretty incredible but I can't help but feeling they're doomed to follow in heroku's footsteps (unclear if good or bad). They've built some pretty wild stuff and I can't help but wonder if they're overcooking the ocean instead of just solving problems for their users. Durable and available storage are all they really need to draw me away from big cloud providers but this combined with their answer to…
At least I could rely on Heroku in production. I've wanted to give Fly.io a try but this gives me pause. I really do miss the Heroku DX whenever I'm putzing around with the increasing complexity of AWS.
For hobby projects - where I dare not touch AWS for fear of going bankrupt from a misconfigured service - I found the sweet spot to be Dokku on top of a Hetzner or Digital Ocean instance. It provides a Heroku like interface on top of cheap hosting, and is fine where you don't expect to scale very much.
Re: Fly.io Postgres cluster down for 3 days, no word from them about it
#64You unfortunately get what you pay for. AWS is more expensive than God, but I'll be damned if you can't have a throat to choke in less than 10 minutes whenever something like this happens.
I had one situation where a Hetzner dedi didn't come back up on a reboot. Their dedis are cheap, this one is like $40ish/mo?
Opened a ticket and support had it back up again within about 10 minutes, turned out to be a failed CPU fan which caused an overheat condition and made it so the system wouldn't complete the boot. They swapped the fan and it came up. It's the only failure I've had in years of dealing with them and was just impressed how quickly a physical failure event like that got handled.
Re: Fly.io Postgres cluster down for 3 days, no word from them about it
#65There is now a response to the support thread from Fly[1]: > Hi Folks, > Just wanted to provide some more details on what happened here, both with the thread and the host issue. > The radio silence in this thread wasn’t intentional, and I’m sorry if it seemed that way. While we check the forum regularly, sometimes topics get missed. Unfortunately this thread one slipped by us until today, when someone saw it and flag…
The irony or perhaps the tragedy of building a low friction service is that you have to have experts on the lower level high friction stuff. I would hope that after a couple of hours downtime, they'd bring up a fresh machine with Ansible or whatever. Hardware or AWS/GCP Vm.
> I would hope that after a couple of hours downtime, they'd bring up a fresh machine with Ansible or whatever.
It is not just about a fresh machine which hopefully sits in each datacenter. I can imagine they needed the clone of the system due to the design of the fly.io service and that's where the "fun" begins.
Re: Fly.io Postgres cluster down for 3 days, no word from them about it
#66Wondering if for small/bootstrapped projects there's any alternative people suggest? Fly has a nice UX and accessible prices, but it's unstable at best. I use the big clouds at work, but for personal they are $$$. Also I want to keep devops tending asymptotically to zero.
I use Dokku on top of Hetzner for my hobby projects - hosting is super cheap, for a little extra I can add a mounted volume for storage, and if the project outgrows a single server I can always just break out of Dokku and use some Docker containers behind a load balancer.
If you are outside of Europe, Digital Ocean or Linode may work better for you.
Re: Fly.io Postgres cluster down for 3 days, no word from them about it
#67There is now a response to the support thread from Fly[1]: > Hi Folks, > Just wanted to provide some more details on what happened here, both with the thread and the host issue. > The radio silence in this thread wasn’t intentional, and I’m sorry if it seemed that way. While we check the forum regularly, sometimes topics get missed. Unfortunately this thread one slipped by us until today, when someone saw it and flag…
> While we check the forum regularly, sometimes topics get missed. Unfortunately this thread one slipped by us until today, when someone saw it and flagged it internally. If it really got missed, then I don't understand how the thread was made private to only logged-in users?
[deleted]
Re: Fly.io Postgres cluster down for 3 days, no word from them about it
#68There is now a response to the support thread from Fly[1]: > Hi Folks, > Just wanted to provide some more details on what happened here, both with the thread and the host issue. > The radio silence in this thread wasn’t intentional, and I’m sorry if it seemed that way. While we check the forum regularly, sometimes topics get missed. Unfortunately this thread one slipped by us until today, when someone saw it and flag…
> While we check the forum regularly, sometimes topics get missed. Unfortunately this thread one slipped by us until today, when someone saw it and flagged it internally. If it really got missed, then I don't understand how the thread was made private to only logged-in users?
Whoa, what? That's a much bigger red flag than the downtime itself.
Re: Fly.io Postgres cluster down for 3 days, no word from them about it
#69The worst thing about Fly is, when something goes wrong, it's not just one thing, there's bunch of things broken at the same time and their status page will show everything green.
Their typical response is either silence or so casual ("oh this is what happens we deploy on friday"). The product looks amazing but it's just a nice package around the most unreliable hosting service I've ever used.
You can't just keep breaking people's work every once a week, make them spend their weekend nights trying to bring back their stuff, and give these "we could have done better" answers. This is an excuse for exceptions, not patterns.
Re: Fly.io Postgres cluster down for 3 days, no word from them about it
#70Earlier quoted context omitted.
FWIW, our aws enterprise support reps are available 24/7 and usually respond within a few minutes. But again, you get what you pay for.
I was working for a pretty big early AWS customer--one that had realized that for the low low price of all your money you could make DynamoDB scale to some truly massive numbers--and one time when we were having trouble around noon Eastern, a colleague called up our TAM. As he told it, the TAM sounded half-asleep, so my colleague asked if everything was alright. "I'm in Hawaii on my honeymoon and my backup missed you…
Weird, I just begrudgingly went from Postgres to Dynamo because it was so much cheaper. We're not huge scale though, so I'm wondering where the costs start to diverge the other way.