Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reliability: It’s not great

community.fly.io

Post reply on HN
Print view View original

191–200 of 476 posts

Re: Reliability: It’s not great

#191

Post by satvikpendem »

notpushkin wrote: post #184
satvikpendem wrote: post #104

Earlier quoted context omitted.

I'm going to plug Coolify, an open source Heroku alternative (with Docker support too) that I'm using on a cheap $5 Hetzner server which is a lot cheaper than the equivalent Fly or Render etc service, and it really doesn't have much upkeep from me even if you add in the time setting up the server initially, which is like an hour, and afterwards, it Just Works™. https://coolify.io

Dokku is also nice and battle-tested: https://dokku.com/ And may I also plug Lunni, a self-hosted Docker Swarm-based PaaS I'm working on right now: https://lunni.dev/ Both work pretty well on $5 servers.

Lunni looks really interesting! Looks like a Coolify competitor, I'll definitely check it out. Do you have a Discord to join? Coolify has one and I found it great to discuss the project and talk directly to the creator.

I used to use Dokku but I personally liked the GUI from Coolify so I've been using that. Nice to see that you have a GUI as well, makes configuring apps a lot easier.

Re: Reliability: It’s not great

#192

Post by te_chris »

sethammons wrote: post #108
te_chris wrote: post #48

Earlier quoted context omitted.

Not gonna happen. Both will get acquired because that’s how things work now

I'm guessing that downvotes come from those who see the macro environment changing. With increased rates, borrowing to purchase companies may make less sense.

Macro makes it harder to raise funding too though - VC no longer as attractive given the risks and higher interest rates available

Re: Reliability: It’s not great

#193

Post by mota7 »

azurelake wrote: post #169
zamnos wrote: post #112

Earlier quoted context omitted.

Where does the misalignment between what the customer thinks they want, and what they actually want fit in to your philosophy? Google Spanner is a great example of this because who doesn't want instantaneous global writes? It's just that, y'know, there's a ton of businesses, especially smaller ones, that don't actually need that. The smarter customers realize this themselves, and can judge the premium they'd pay for…

> who doesn't want instantaneous global writes I want to gently note since I see a lot of misunderstanding around Spanner and global writes: Global writes need at least one round trip to each data center, and so they're still subject to the speed of light.

Like most things, it's more complex than that, and as a result it can be either faster or slower than 'median(RTT to each DC in quorum)'.

It's a delicate balance based on the locations that rows are being read and written. In the case where a row being repeatedly written from only one location and not being read from different location, the writes can be significantly faster than would be naively expected.

Re: Reliability: It’s not great

#194

Post by jrochkind1 »

mrkurt wrote: post #159
samwillis wrote: post #18

Fundamentally I think some of the problems come down to the difference between what Fly set out to build and what the market currently want. Fly (to my understanding) at its core is about edge compute. That is where they started and what the team are most excited about developing. It's a brilliant idea, they have the skills and expertise. They are going to be successful at it. However, at the same time the market is…

This is, indeed, the exciting part. As Heroku fans, we never really felt like it needed a replacement. And if it did, it seemed like Render was the natural Heroku v.next. One thing we've noticed, though, is that people do actually want Heroku but close to users. It's not exactly edge compute. In some cases, it's "Heroku in Tokyo". In others it's "Heroku, but running in all the english speaking regions". I think the t…

> As Heroku fans, we never really felt like it needed a replacement.

If Salesforce kept investing in heroku, it might not. But there is a huge loss of confidence in heroku's future going on among heroku's customers right now, which is part of what you're seeing, as I'm sure you know. (Also I think to some extent you are being political/kind towards heroku... if heroku's owners were still investing in heroku for real, adding 'edge' functionality like fly.io is focusing on is what one would probably expect...)

And frankly... your tool seems more mature and... not to be rude to competitors but seems to have more of that certain `je ne sais quoi` of Developer Experience Happiness that heroku _used_ to have and other potential heroku competitors don't really quite seem to have yet. Does what you expect/need in a polished and consistent way.

I think work you put into the underlying infrastructure definitely shows there, and was the right choice. Tidy infrastructure helps with tidy consistent developer experience.

So I understand why people are looking to you as a heroku replacement. I am too! (And I don't really need the edge compute stuff; although I could potentially see using it in the future, and it shows you folks are on top of things).

And while I kept reading fly staff saying on HN comments that you didn't want to be a heroku replacement, so were unconcerned with the few places people were mentioning where you still felt short of it -- when I saw your investment in Rails documentation and tools (and contribs back to Rails), I thought, aha, i think they've realized this is a market looking for them, which they are only a couple steps from and it would make sense to meet.

When you mention in OP a "heroku exodus" to you... I'm curious if that was all people who left when heroku ended free tier stuff, and they've all come to you for your free tier stuff... becuase that does seem dangerous, such a giant spike in users who are not paying and don't bring revenue with them! I don't personally use very much heroku free tier stuff. I hope if that's a challenge, it's one you can get over. I don't think you are under any obligation to offer free stuff that can be used for real production workloads indefinitely -- although, as I'm sure you know, free stuff is huge for allowing people to try _before_ they buy, and whatever limits you put on it to try to prevent indefinite production use get in the way of someone's "try before you buy" too... and at this point, _reducing_ your free offerings is a lot harder PR-wise than having started out with less in the first place. :(

Re: Reliability: It’s not great

#195

Post by tptacek »

pyentropy wrote: post #179
tptacek wrote: post #166

Earlier quoted context omitted.

We are asking to HashiCorp products to do things they were not designed to do, in configurations that they don't expect to be deployed in. Take a step back, and the idea of a single global namespace bound up with Raft consistency for a fleet deployed in dozens of regions, providing near-real-time state propagation, is just not at all reasonable. Our state propagation needs are much closer to those of a routing protoc…

I respect that. Can you elaborate a bit on the routing protocol thing? I assume you used WAN gossip? I love the simplicity of fly.io & wish you all the best improving Fly's reliability!

If you've ever implemented IS-IS or OSPF before, like 80% of the work is "LSP flooding", which is just the process that gets updates about available links from one end of the network to another as fast as possible without drowning the links themselves in update messages. Flooding algorithms don't build consensus, unlike Raft quorums, which intrinsically have a centralized set of authorities that keep a single source of truth for all the valid updates.

An OSPF router uses those updates to do build a forwarding table with a single-point shortest path first routine, but there's nothing to say that you couldn't instead use the same notion of publishing weighted advertisements of connectivity to, for instance, build a table to map incoming HTTP requests to backends that can field them.

The point is, if you're going to do distributed consensus, you've got a dilemma: either you're going to have the Ents moot in a single forest, close together, and round trip updates from across the globe in and out of that forest (painfully slow to get things in and out of the cluster), or you're going to try to have them moot long distance (painfully slow to have the cluster converge). The other thing you can do, though, is just sidestep this: we really don't have the Raft problem at all, in that different hosts on our network do not disagree with each other about whether they're running particular apps; if worker-sfu-ord-1934 says it's running an instance of app-4839, I pretty much don't give a shit if worker-sfu-maa-382a says otherwise; I can just take ORD's word for it.

That's the intuition behind why you'd want to do something like SWIM update propagation rather than Raft for a global state propagation scheme.

But if you're just doing service discovery for a well-bounded set of applications (like you would be if you were running engineering for a single large company and their internal apps), Raft gives you some handy tools you might reasonably take advantage of --- a key-value store, for instance. You're mostly in a single data center anyways, so you don't have the long-distance-Entmoot problem. And HashiCorp's tools will federate out across multiple data centers; the constraints you inherit by doing that federation mostly don't matter for a single company's engineering, but they're extremely painful if you're servicing an unbounded set of customer applications and providing each of them a single global picture of their deployments.

Or we're just holding it wrong. Also a possibility.

Re: Reliability: It’s not great

#196

Post by [deleted] »

pyentropy wrote: post #179
tptacek wrote: post #166

Earlier quoted context omitted.

We are asking to HashiCorp products to do things they were not designed to do, in configurations that they don't expect to be deployed in. Take a step back, and the idea of a single global namespace bound up with Raft consistency for a fleet deployed in dozens of regions, providing near-real-time state propagation, is just not at all reasonable. Our state propagation needs are much closer to those of a routing protoc…

I respect that. Can you elaborate a bit on the routing protocol thing? I assume you used WAN gossip? I love the simplicity of fly.io & wish you all the best improving Fly's reliability!

[deleted]

Re: Reliability: It’s not great

#197

Post by chime »

aeyes wrote: post #103
claytonjy wrote: post #12

Very interesting to see Kurt assert theyre going to "solve managed Postgres", and I'm super curious to know what that means. Does it mean something like RDS, or more like CrunchyData? I could see them building something RDS-like on their own, but if they're trying to go further than that I wonder if they'll buy or partner with other companies rather than doing it themselves. Neon strikes me as a Postgres-as-a-service…

If I was in their shoes I'd probably aim for a "serverless" Postgres experience where you get a connection string and you know nothing else. I think RDS, Crunchy, Aiven and others aren't quite there yet.

They kind of offer that with their Redis (via Upstash). But for our use-case, we needed it to be managed PG and Redis. Going out of the LAN introduces too much latency.

Re: Reliability: It’s not great

#198

Post by prdonahue »

everybodyknows wrote: post #60
ec109685 wrote: post #52

Earlier quoted context omitted.

The CloudFlare folks wrote a good blog post on how they are seeing their customers use Edge compute — latency is far down on the list: https://blog.cloudflare.com/cloudflare-workers-serverless-we...

Hmm, that post is almost three years old -- still accurate?

Yes, especially as compliance and regulatory frameworks continue to evolve and become more difficult to adhere to as mentioned elsewhere in the comments.

We're inherently faster than other "serverless" platforms due to the scale and homogeneous design of our network, and that network has presence in nearly 50% more cities than it did just 3 years ago. We were plenty fast enough then and we're even faster now.

Other things that customers (still) really care about: developer experience, ease of use, and cost. Nobody likes paying the AWS tax to move data around—they just want to use the best solution from the best cloud provider. Workers and the associated storage primitives allow them to pick and choose from the best that AWS, Azure, Cloudflare, GCP, et al. have to offer.

(Disclaimer: I'm a long time Cloudflare employee focused on App Sec, and I speak to customers regularly who look to Workers largely for compliance reasons, but I don't work on the Developer Platform business. Am sure my Dev Platform peers will chime in with more nuanced answers!)

Re: Reliability: It’s not great

#199

Post by lll-o-lll »

chucky_z wrote: post #178
lll-o-lll wrote: post #78

At first I was all like “Ha ha, losers can’t scale” And then I was “Huh, these technical challenges are actually pretty difficult” And then I was all “crap, these are a bunch of technologies I was about to add to our stack” Thanks heaps fly.io people; having the humility to honestly talk about the challenges and failures massively helps people such as myself as we navigate new unfamiliar technologies. If more compani…

The tech in their stack is still pretty good. Unless you’re supporting tens of thousands of customers and trying to make the promises that fly makes today. Look at the fly engineer replies in this thread. Also they basically only use OSS versions, they could go give Hashicorp some money to solve their Vault problems. They could probably partner with SecondQuadrant for PG as two examples. That might not make sense for…

Sure, I was going for a little humour there. A little riff on the whole “we always judge others until we walk in their shoes”.

The take away I was hoping for is “providing insights into how we struggle helps others”

Re: Reliability: It’s not great

#200

Post by chronark »

chime wrote: post #197
aeyes wrote: post #103

Earlier quoted context omitted.

If I was in their shoes I'd probably aim for a "serverless" Postgres experience where you get a connection string and you know nothing else. I think RDS, Crunchy, Aiven and others aren't quite there yet.

They kind of offer that with their Redis (via Upstash). But for our use-case, we needed it to be managed PG and Redis. Going out of the LAN introduces too much latency.

Upstash Redis for Fly runs on Fly infrastructure and we observe latencies in the low single digit milliseconds.
Post reply on HN