Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Migrating from AWS to Fly.io

terrateam.io

Post reply on HN
Print view View original

81–90 of 189 posts

Re: Migrating from AWS to Fly.io

#81

Post by garganzol »

joao wrote: post #75
tommoor wrote: post #59

Fly is great. I know the Fly team has an aversion to it but I really just wish they would hire some database folks and take on managed Postgres. I'm already running 8+ apps on there and it would be the peace of mind needed to move the rest across… one day.

Would love this also. The best option I could find was using Digital Ocean Managed Databases. The cheapest costs $15, but you can host multiple databases with good insights and backups. You can choose where you host it, and place it close to the region where youre Fly.io apps are, with low latency. Only caveat is there isn't an easy way to automatically update the IPs whitelist of the database, to support the Fly bui…

The best value option I've seen so far is Azure SQL Server databases. Not exactly Postgres, but it's only $5/month and works wonders.

Re: Migrating from AWS to Fly.io

#82

Post by Existenceblinks »

itake wrote: post #5
l5870uoo9y wrote: post #2

> The main motivation to migrate was cost. Is Fly.io really than much cheaper for hosting CDN/Postgres/Docker Container?

I saved about 25% from Heroku, but fly is much less reliable. I've had TLS certs expire without warning and redis has timeouts

I tested websocket on fly about 2 years ago, it seemed to have kinda weird characteristic. It's hard to explain because I can't dig it. It's like throttle + batch scheduling somehow (I don't know what it actually does though)

I tested Fly, DO app platform, Render. Ended up with DO (but it's meh, can't mount disk volume, can't deploy to a specific sha code git)

I'm going to [stress]test their wss:// again this year, I'm thinking to finally move a commercial one to fly (already have hobby apps there)

Re: Migrating from AWS to Fly.io

#83

Post by rendaw »

yodon wrote: post #66
mrdoops wrote: post #7

Earlier quoted context omitted.

The biggest $ cost I'd have on my radar WRT AWS vs Fly is complexity and need-to-know of employee time to do any given thing. Fly is way easier to navigate and use than AWS and its labyrinth of Cloud Scale™ horrors. The trade-off being you can do more given things with AWS.

>Fly is way easier to navigate and use than AWS and its labyrinth of Cloud Scale™ horrors Give fly.io a couple years and it will end up forced to take on all the complexity and edge cases AWS and Azure have to deal with today. A shiny new codebase is always nicer than an old one, because the old one has had to accommodate all those pesky customer needs.

Did that ever happen to Heroku? Ignoring 10% of use cases for their target customers and making the 90% doesn't sound crazy to me.

Re: Migrating from AWS to Fly.io

#84

Post by minxomat »

garganzol wrote: post #79
atonse wrote: post #70

Earlier quoted context omitted.

The lack of “managed” Postgres is the only reason we haven’t moved 100% to fly. The idea of having to do my own upgrades by updating the docker container scares me. I have literally never run a stateful service off docker and don’t see the point, thanks to AWS. RDS has been extremely stable and boring. Maybe it’s out together by similar duct tape under the hood but it’s been incredibly solid for nearly a decade acros…

What stops you from using a managed database from a provider who specializes in it? If Fly builds it, it will create another Goliath who tries to eat everything.

Not really. I see fly competing with Workers for smaller services, and with KV and D1, managed edge DBs are so easy it’s hard to live without them. Very different use case from a traditional managed DB - because it’s either all scaling and running in fly or it isn’t.

Re: Migrating from AWS to Fly.io

#85

Post by Spivak »

garganzol wrote: post #79
atonse wrote: post #70

Earlier quoted context omitted.

The lack of “managed” Postgres is the only reason we haven’t moved 100% to fly. The idea of having to do my own upgrades by updating the docker container scares me. I have literally never run a stateful service off docker and don’t see the point, thanks to AWS. RDS has been extremely stable and boring. Maybe it’s out together by similar duct tape under the hood but it’s been incredibly solid for nearly a decade acros…

What stops you from using a managed database from a provider who specializes in it? If Fly builds it, it will create another Goliath who tries to eat everything.

Network posture? I can’t imagine trying to put the public internet in between my app servers and db.

Re: Migrating from AWS to Fly.io

#86

Post by mtlynch »

garganzol wrote: post #62
mtlynch wrote: post #48

I've migrated several services from GCP to Fly.io, and I've been very happy with Fly. All of my apps are low-volume hobbyist web apps that run on a single machine, so I don't do any Terraform, k8s, or Postgres, so my use case is a little simpler than OP. My biggest complaint has been with outages,[0, 1, 2] but that's gotten better. The original architecture meant that larger servers could evict other users' running s…

To avoid outages, just scale up the number of app instances to like 2 or 3 and choose a diverse geography for them. Outages is not something specific to Fly. For example, if you have an app on Azure and it has only one instance then... well, you asked for it to go down from time to time. Maybe not a big deal for hobby projects, but for business critical stuff one should always consider deploying several instances of…

Yeah, but going from 1 to N instances is a big jump in complexity for not much more uptime.

I'd rather have a host that can do around three nines and just accept that as my upper bound.

Re: Migrating from AWS to Fly.io

#87

Post by mostly_lurks »

tptacek wrote: post #20
jokethrowaway wrote: post #12

I found debugging setup errors on fly to be incredibly frustrating You can't even ssh until the container started and they have some requirements on hostnames (0.0.0.0) At least a few of their examples are broken. The documentation could be better but it's pretty good Is it better than Aws? Yes of course, you need to spend way more to get a dx as bad as Aws. Setting up a VPS is a smoother experience than fly, though.…

Help me understand what you'd be SSH'ing into before your VM boots up? (I'm responsible for most of our SSH goop, so if there's an improvement to be made, I'm happy to make it). A reminder: we don't run "containers". We take containers, unpack them, and transform them into VMs. There's no host OS for you to access.

Caveats: 1) this may not be what grandparent was trying to do and 2) I have limited experience debugging "regular" Docker, let alone firecracker or whatever Fly.io does.

With that said, I succesfully deployed a simple Rust/Actix/Sqlite app with a Dockerfile to Fly.io. I then thought I'd try out litestream. For reasons I'm still not sure about, having `ENTRYPOINT ["litestream replicate … -exec myapp"]` resulted in immediate kernel panics.[1]

As I was debugging, my instinct was to want to `fly ssh console` into a running container to see if running the same command from the shell produced any clues for further debugging. Though I understand this is the wrong mental model, the thought was something like "well, I know everything else works, I just need Fly to ignore this failing command so I can have a minute to poke around." To do this, I ended up just removing litestream from the ENTRYPOINT so the deploy would succeed, then I could SSH in and play around at the shell to see what was going on.

Again, I have no idea whether this is the same sort of problem the other person was having, but for my case, what would be helpful is probably not changing how `fly ssh console` behaves, but perhaps some documentation of suggested debugging techniques in case your app is failing to start.

[1] Putting the exact same command into a "start.sh" and making that the ENTRYPOINT worked fine so that's what I ended up doing.

Re: Migrating from AWS to Fly.io

#88

Post by tptacek »

mostly_lurks wrote: post #87
tptacek wrote: post #20

Earlier quoted context omitted.

Help me understand what you'd be SSH'ing into before your VM boots up? (I'm responsible for most of our SSH goop, so if there's an improvement to be made, I'm happy to make it). A reminder: we don't run "containers". We take containers, unpack them, and transform them into VMs. There's no host OS for you to access.

Caveats: 1) this may not be what grandparent was trying to do and 2) I have limited experience debugging "regular" Docker, let alone firecracker or whatever Fly.io does. With that said, I succesfully deployed a simple Rust/Actix/Sqlite app with a Dockerfile to Fly.io. I then thought I'd try out litestream. For reasons I'm still not sure about, having `ENTRYPOINT ["litestream replicate … -exec myapp"]` resulted in imm…

One of my biggest UX nits with Fly (I have no excuses, I have all the access I need to go fix this myself) is that we "kernel panic" when your entrypoint command fails. Of course, our kernel is not really panicking --- we've just run out of things for our `init` to manage, so it exits, and when pid 1 exits, so does the kernel. But you get the terrifying stack dump.

We can clean this up, so that you get a clearer, simpler error ("your entrypoint exited, here's the exit code, there's nothing else for this VM to do so it's exiting, have a nice day"), and it's been on the docket for months. We'll get it done!

We could conceivably add a flag for our `init` to hang around waiting for you to SSH in after your entrypoint exits. But that's clunky and complicated. Usually, you want your kernel to exit when your entrypoint fails, so that your service restarts! What you should do instead is push a container that has enough process supervision to hang around itself. Here's a doc:

https://fly.io/docs/app-guides/multiple-processes/

Re: Migrating from AWS to Fly.io

#89

Post by garganzol »

mtlynch wrote: post #86
garganzol wrote: post #62

Earlier quoted context omitted.

To avoid outages, just scale up the number of app instances to like 2 or 3 and choose a diverse geography for them. Outages is not something specific to Fly. For example, if you have an app on Azure and it has only one instance then... well, you asked for it to go down from time to time. Maybe not a big deal for hobby projects, but for business critical stuff one should always consider deploying several instances of…

Yeah, but going from 1 to N instances is a big jump in complexity for not much more uptime. I'd rather have a host that can do around three nines and just accept that as my upper bound.

The key moment is not to have any persistent state in the app. App may use a database, that's ok, but the app itself should not store any persistent state on a hard drive.

Once that simple requirement is met, the scaling becomes easy.

Re: Migrating from AWS to Fly.io

#90

Post by mrkurt »

rendaw wrote: post #77
mrkurt wrote: post #53

Earlier quoted context omitted.

The Terraform provider was a one off project from the community. We haven't really focused on it in earnest yet, unfortunately. We ended up sponsoring it, but until we get Fly apps off Nomad I don't think we'll have the best terraform story.

Sorry, not to use this as a support forum, but what exactly does it need? Just some provider-side bug fixes? Or are there Fly-side changes that need to be made before it will work? If it's something small I wouldn't mind poking at it.

Just provider bug fixes / developments. The APIs it use are pretty stable. The particular errors in those issues look like either badly formed API calls, or something weird with auth.

It also probably needs an update to use api.machines.dev. It is currently trying to connect over wireguard to hit our private API, because we didn't have a public API endpoint when it was built. This is probably brittle. :)

Post reply on HN