Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Leaving serverless led to performance improvement and a simplified architecture

unkey.com

Post reply on HN
Print view View original

71–80 of 272 posts

Re: Leaving serverless led to performance improvement and a simplified architecture

#71

Post by Esophagus4 »

bamboozled wrote: post #65
gloomyday wrote: post #60

I think developers are drowning in tools to make things "easy", when in truth many problems are already easy with the most basic stuff in our tool belt (a compiler, some bash scripts, and some libraries). You can always build up from there. This tooling fetish hurts both companies and developers.

Excerpt AWS lambda is stupidly cheap!

For certain workloads :)

And that is actually the advantage of serverless, in my mind. For some low-traffic workloads, you can host for next to nothing. Per invocation, it is expensive, but if you only have a few invocations of a workload that isn't very latency sensitive, you can run an entirely serverless architecture for pennies per month.

Where people get burned is moving high traffic volumes to serverless... then they look at their bill and go, "Oh my god, what have I done!?" Or they try to throw all sorts of duct tape at serverless to make it highly performant, which is a fool's errand.

Re: Leaving serverless led to performance improvement and a simplified architecture

#72

Post by stego-tech »

Esophagus4 wrote: post #41
kburman wrote: post #23

The takeaway here isn’t that serverless doesn’t work, it’s that the authors didn’t understand what they were building on. Putting a latency-critical API on a stateless edge runtime was a rookie mistake, and the pain they describe was entirely predictable.

I’ve found this to be true, with one caveat. Most cloud pain people experience is from a misunderstanding / abuse of solutions architecture and could have been avoided with a more thoughtful design. It tends to be a people problem, not a tool problem. However , in my experience cloud vendors sell the snot out of their offerings, and the documentation is closer to marketing than truthful technical documentation. Their…

You took the words right out of my mouth. Between aggressive salespeople marketing any given product as a panacea for everything and mandates from above to arbitrarily use X thing to do Y, there’s a lot of just plain bad architecture out there.

Re: Leaving serverless led to performance improvement and a simplified architecture

#73

Post by hrimfaxi »

ramraj07 wrote: post #61
chronark_ wrote: post #53

Earlier quoted context omitted.

I can assure you that was pretty close to the internal conversation lol Not sure what the different takeaways would be though?

What did your internal discussion conclude for the question "Why did we not take a step back earlier and think, why are we doing it this way?" Im genuinely curious because this is not singling out your team or org, this is a very common occurrence among modern engineering teams, and I've often found myself on the losing end of such arguments. So I am all ears to hear at least one such team telling what goes on in the…

I have had CTOs (two in my career) tell me we had to use our AWS credits since they were going to expire worthless. Both experiences were at vc-backed startups.

Re: Leaving serverless led to performance improvement and a simplified architecture

#74

Post by sgarland »

gloomyday wrote: post #60

I think developers are drowning in tools to make things "easy", when in truth many problems are already easy with the most basic stuff in our tool belt (a compiler, some bash scripts, and some libraries). You can always build up from there. This tooling fetish hurts both companies and developers.

It’s that, and the fact that precious few people seem to understand fundamentals anymore, which is itself fed by the desire to outsource everything to 3rd parties. You can build an entire stack where the only thing you’ve actually made is the core application, and even that is likely to be influenced if not built by AI.

The industry is creating learned helplessness.

Re: Leaving serverless led to performance improvement and a simplified architecture

#75

Post by chronark_ »

ramraj07 wrote: post #61
chronark_ wrote: post #53

Earlier quoted context omitted.

I can assure you that was pretty close to the internal conversation lol Not sure what the different takeaways would be though?

What did your internal discussion conclude for the question "Why did we not take a step back earlier and think, why are we doing it this way?" Im genuinely curious because this is not singling out your team or org, this is a very common occurrence among modern engineering teams, and I've often found myself on the losing end of such arguments. So I am all ears to hear at least one such team telling what goes on in the…

Oh we had it coming for quite some time and knew we would need to rebuild it, we just didn’t have the capacity to do it unfortunately.

I was working on it on and off moving one endpoint at a time but it was very slow until we hired someone who was able to focus on it.

It didn’t feel good at all. We knew the product had massive flaws due to the latency but couldn’t address it quickly. Especially cause we he to build more workarounds as time went on. Workarounds we knew would be made redundant by the reimplementation.

I think we had that discussion if “wtf are we doing here” pretty early, but we didn’t act on it in the beginning, instead we tried different approaches to make it work within the serverless constraints cause that’s what we knew well.

Re: Leaving serverless led to performance improvement and a simplified architecture

#76

Post by scottydelta »

K0IN wrote: post #58

After building my first Serverless/Cloudflare worker app, this is why I migrated to Deno. Deno enables you to run the same codebase in deno (self-hosted/local) and in deno deploy (serverless platform from deno). I wanted my app to be self-hostable as well, and Cloudflare worker is a hard ecosystem lock to their platform, which makes it undesirable (imo). Here is a link to my reasoning from back then: https://github.c…

I ported my worker project into Django since cloudflare workers wouldn’t allow selection of region for hosting workers which is generally required due to data compliances. This is something all cloud providers provide from day one yet cloudflare made it an enterprise feature.

Also the vendor lock-in doesn’t help with durable objects and D2 instead of simply doing what supabase and others are doing by providing Postgres or standard SQLite as a service.

Re: Leaving serverless led to performance improvement and a simplified architecture

#77

Post by osigurdson »

kburman wrote: post #46
czhu12 wrote: post #33

Earlier quoted context omitted.

> Putting a latency-critical API on a stateless edge runtime Isn’t this the whole point of serverless edge? It’s understood to be more complex, with more vendor lockin, and more expensive. Trade off is that it’s better supported and faster by being on the edge. Why would anyone bother to learn a proprietary platform for non critical, latency agnostic service?

You're confusing network proximity with application architecture. Edge deployment helps connection latency. Stateless runtime destroys it by forcing every cache access through the network. The whole point of edge is NOT to make latency-critical APIs with heavy state requirements faster. It's to make stateless operations faster. Using it for the former is exactly the mismatch I'm describing. Their 30ms+ cache reads vs…

Realistically, they should be able to do sub ms cache hits which land in the same datacenter. I know cloudflare doesn't have "named" datacenters like other providers but at the end of the day, there are servers somewhere and if your lambda runs twice in the same one there is no reason why a pull-through cache can't experience a standard intra data-center latency hit.

I wonder if there is anything other than good engineering getting in the way of this and even sub us intra-process pull through caches for busy lambda functions. After all, if my lambda is getting called 1000X per second from the same point of presence, why wouldn't they keep the process in memory?

Re: Leaving serverless led to performance improvement and a simplified architecture

#78

Post by chronark_ »

sgarland wrote: post #64
chronark_ wrote: post #54

Earlier quoted context omitted.

Not everyone is born with experience in distributed systems

Sure, but there are some fundamentals about latency that any programmer should know [0] (absolute values outdated, but still useful as relative comparisons), like “network calls are multiple orders of magnitude slower than IPC.” I’m assuming you’re an employee of the company based on your comments, so please don’t take this poorly - I applaud any and all public efforts to bring back sanity to modern architecture, esp…

I cofounded it yeah

And yeah you’re right in hindsight it was a terrible idea to begin with

I thought it could work but didn’t benchmark it enough and didn’t plan enough. It all looked great in early POCs and all of these issues cropped up as we built it

Re: Leaving serverless led to performance improvement and a simplified architecture

#79

Post by chronark_ »

kburman wrote: post #66
chronark_ wrote: post #54

Earlier quoted context omitted.

Not everyone is born with experience in distributed systems

That's fair, but then the framing matters. The article criticizes serverless architecture rather than acknowledging an evaluation failure. "Serverless was fighting us" vs "We didn't understand serverless tradeoffs" - one is a learning experience, the other is misdirected criticism.

Yeah that’s fair

Re: Leaving serverless led to performance improvement and a simplified architecture

#80

Post by [deleted] »

dlisboa wrote: post #70
Danjoe4 wrote: post #48

Earlier quoted context omitted.

This is exactly why I'd rather get a fat VPS from a reputable provider. As long as the bandwidth is sufficient the only limitation is vertical scaling.

I'm partial to this, the only thing I've found that is harder to achieve is the "edge" part of cloud services. Having a server at each continent is enough for most needs but having users route to the closest one is not as clear to me. I know about Anycast but not how to make it operational for dynamic web products (not like CDN static assets). Any tips on this?

[deleted]
Post reply on HN