I like Fly.io but I wish it offered something with Nix in the build process like Railway has recently introduced.
Fly.io: The reclaimer of Heroku's magic
231–240 of 320 posts
Re: Fly.io: The reclaimer of Heroku's magic
#232Earlier quoted context omitted.
Fly did have a built-in redis cache (albeit multi-tenant / shared) and for the life of me I can't figure out why they'd deprecate it (though, it still works): https://community.fly.io/t/debugging-a-failing-image-without... and https://community.fly.io/t/please-bring-redis-back/1563 If anything, I'd prefer they moved their PaaS more towards serverless and managed offerings than towards IaaS.
On their forums they say they envision some other company handling Postgres too. Personally I would prefer they handle the databases, redis etc because that way the services you need can be provisioned right alongside your app servers otherwise it undermines the value of having edge servers near your users. Heroku could get away with 3rd parties being responsible for the database software because using AWS it was eas…
Same.
My money is on Supabase building for Fly as a target / default IaaS, and that's as much close to managed services we are going to get, given Fly's insistence that they're not really good at (or want to) building (and maintaining) managed services.
Re: Fly.io: The reclaimer of Heroku's magic
#233The only thing I don't like is their usage-based pricing. On Heroku I could pay $7 a month and know I'd never be charged more than that. I'm sure when you're scaling a service it's fine - maybe even better - to do it on a sliding scale. But for a fire-and-forget blog site, I don't want to have to worry about stuff like that.
This is a problem. And a bit of an own goal on our part. I hate services that don't put a price on things like bandwidth (because there's always a price!). So we priced bandwidth and made it transparent. You can put an app on Fly.io and server petabytes of data every month, if you want. We'll never complain that you're serving the wrong content type. But the reality is – having an unlimited bandwidth promise is perfe…
When I worked in adtech (Brightroll) we hosted a segment mapping server on heroku. All it did was look at a url query for the target url, append a value from a cookie to it, and 302 to that url. The value in adtech is that I can tell you which segment a user is in by taking the data in my cookie and appending it to your url.
We launched it on Heroku using Nodejs and sent billions of requests a day to it. The thing ran on like a few dozen dynos, but it was responsible for like 20-30% of all of Heroku's bandwidth. Fantastic value for us. Immediate headache for Heroku.
Re: Fly.io: The reclaimer of Heroku's magic
#234Earlier quoted context omitted.
You don’t need to route every write to primary though, but only those writes that have dependencies on other writes. Things like telemetry can be written in edge instances. Depends on your application of course, but in many cases that should be only a tiny fraction of all requests needing redirects to primary. And why would you get 20 instances, all around the world right out of the gate? 6-7 probably do the job quit…
> You don’t need to route every write to primary though, but only those writes that have dependencies on other writes. Thanks, can you give an example of how that works? Did you write your own fork of Postgres or are you using a third party solution like BDR? Also do you have a few use cases where you'd want writes being dependent on another write? > 6-7 probably do the job quite well You could, let's call it 5. For…
> Thanks, can you give an example of how that works?
I just noticed I formulated it wrong, my apologies. What I meant is that the replicating regions don’t need to wait for the primary writes to go through before they respond to clients. They will still be read-only Postgres replicas, and info could be shuttled to primary in a fire-and-forget manner, if that’s an option.
Whenever an instance notices that it‘s not primary, but it is currently dealing with a critical write, it can refuse to handle the request, and return a 409 with the fly-replay header that specifies the primary region. Their infra will replay the original request in the specified region.
> Did you write your own fork of Postgres or are you using a third party solution like BDR?
When using fly.io, the best option would probably be to use their postgres cluster service which supports read-only replicas (can take a few seconds for updates to reach replicas): https://fly.io/docs/getting-started/multi-region-databases/
> For a 2gb set up would that be about $50 for the web app, $50 for the background workers, $160ish for postgres and then $50 for Redis? We're still at $300+?
Maybe. A few thoughts:
- Why would you need 5 web workers, would one running on primary not be ideal? If you need so much compute for background work, then that’s not fly‘s fault, I guess.
- Not sure the Postgres read replicas would need to be as powerful as primary
- Crazy idea: Use SQLite (replicated with Litestream) instead of Redis and save 50 bucks
Re: Fly.io: The reclaimer of Heroku's magic
#235Earlier quoted context omitted.
This is a problem. And a bit of an own goal on our part. I hate services that don't put a price on things like bandwidth (because there's always a price!). So we priced bandwidth and made it transparent. You can put an app on Fly.io and server petabytes of data every month, if you want. We'll never complain that you're serving the wrong content type. But the reality is – having an unlimited bandwidth promise is perfe…
> We'll never complain that you're serving the wrong content type. Cloudflare shouldn't restrict media (video, images, and audio) from its unlimited bandwidth promise for Workers and R2 (though, ToS doesn't yet reflect that). https://news.ycombinator.com/item?id=28682885 > But the reality is – having an unlimited bandwidth promise is perfect for for a fire and forget blog site I think, an auto flyctl pause -a when my…
Yup. Not too hard. This outputs the current bandwidth in gigabytes when you pass your app name.
#!/usr/bin/env bash
set -euo pipefail
QUERY=$(cat Re: Fly.io: The reclaimer of Heroku's magic
#236Earlier quoted context omitted.
I was moving a Ghost blog from Render. Ghost is notorious for having a difficult time with hosting assets in S3, so it uses disks. I needed to move the Ghost assets directory for all the posts. This was a ten minute thing in Render. SOL in Fly.
Can you not ssh into a fly instance and then pull?
You can. You gotta get Wireguard running on your local machine [0] and be able to see the ".internal" network, but once you do, you can ssh into any edge VM and if access its filesystem. I've SCP'd SQLite db files and dumps. It's also a pretty anti-pattern way of debugging an app if you're an outlaw :)
Re: Fly.io: The reclaimer of Heroku's magic
#237I like Fly.io but I wish it offered something with Nix in the build process like Railway has recently introduced. https://github.com/railwayapp/nixpacks
It looks like these just generate OCI containers; that's all we need at Fly.io to take your container, so you might be able to just use these. (We've been working for a couple months on Nix, especially for Rails apps, but I don't have status to report).
Re: Fly.io: The reclaimer of Heroku's magic
#238The only thing I don't like is their usage-based pricing. On Heroku I could pay $7 a month and know I'd never be charged more than that. I'm sure when you're scaling a service it's fine - maybe even better - to do it on a sliding scale. But for a fire-and-forget blog site, I don't want to have to worry about stuff like that.
This is the reason I use Heroku and would never on a personal level use some of the bigger solutions, or apparently Fly.io. As an individual and despite being generally careful, I just cannot have a tiny risk of having a $100k+ accidental bill. I'd rather my project goes down if there's a DDOS attack, or if I made a typo and created an infinite loop. If I take some of my hobby projects to the "next level", it'd be de…
Is it possible to rack up a 100k+ bill with personal projects on platforms like fly.io? Like some absolutely massive DDOS or something?
Re: Fly.io: The reclaimer of Heroku's magic
#239> I wouldn't straight out recommend fly.io. See, part of Heroku's value proposition was their excellent support. I had asked fly.io to delete my account since there wasn't a provision to delete it from their interface and they never bothered to reply and put me in some kind of shadow ban from re-registering with my email.
I mean if this is how you're going to treat your customers, then good luck! I'm yet to try out railway.app and..looks interesting to say the least.
Re: Fly.io: The reclaimer of Heroku's magic
#240Earlier quoted context omitted.
There are three ways to manage bandwidth prices: 1. Put servers where bandwidth is cheap (not Sydney, for example) 2. Constrain throughput per server 3. Buy from cheap transit providers like Cogent Hetzner does all three. Bandwidth in the US/EU is very cheap. They meter total throughput on their services. And they use cheap providers. None of these are bad choices, just different than ours. Our product has multiple l…
The chennai pricing in incredibly steep. It's weird that bandwidth prices for end-users in India is among the cheapest (if not the cheapest) in the world but enterprise customers are paying so much.
Jio and friends are (were?) subsidizing connectivity and trying to drive growth to make it up later/elsewhere like payments and services.
Service providers come to the opposite problem. Local infra sucks, the market is full of incumbents, and india is generally protective of those markets.
Very similar dynamics with china as well.