Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Serverless: Cold Start War

mikhail.io

Post reply on HN
Print view View original

91–100 of 110 posts

Re: Serverless: Cold Start War

#91

Post by brianwawok »

village-idiot wrote: post #78
TrueTeller wrote: post #47

Earlier quoted context omitted.

That's all plausible except the price issue. Say, you keep 10 containers alive, so you make 10x 100ms calls every 5 minutes. That's gonna be $0.20 per month. 20 cents.

Internal math at my org says that lambdas cost about 20% more than an equivalent amount of EC2 power at full utilization. Now hitting full utilization is very hard, but the point is that lambas are only cheaper compared to underutilized EC2 hosts.

Any many people (like me), think Kubernetes is a better way to increase utilization compared to Lambdas. Especially with things like autoscaling, you can keep a kubernetes cluster pretty highly utilized, and no cold startup dance.

Re: Serverless: Cold Start War

#92

Post by nickjj »

stephenr wrote: post #32

Tech, to Business: So, hear me out guys. With the power of "The Cloud", we can break our compute workload down to the function level, and have them run as a service for us, rather than say an entire VM, or even an entire container. And because it's a "Cloud" service, we pay for what we use, so if there's no workload for the functions to service, there's no cost. We just pay for the time the tiny little container is a…

There's also a parallel universe where Tech Entrepreneur#2 sits in darkness for a few months while building a competing product with Rails or another comparable stack and gets everything up and running 10x faster on a single server and it ends up costing less per month.

Re: Serverless: Cold Start War

#93

Post by biztos »

acjohnson55 wrote: post #71
biztos wrote: post #53

Earlier quoted context omitted.

I'm pretty new to it but so far I think the big selling points in a corporate environment are: 1. (Maybe, someday) no real Ops work. 2. 100% utilization (at a cost). 3. Measurability of cost in #2. I agree about the ball-of-locked-in-mud danger. And so far I've seen the Ops part be actually a bigger issue than it was before, because there's so much opaque, badly documented madness involved. With AWS Lambda + API Gate…

The no-ops part is compelling, but we've already had that for years in the form of PaaS for small teams and K8s for large ones. I think the billing part sounds a bit like a trap to me. At least while FaaS retains so many quirks, it's like trading billing for the engineering time of contorting a business process to a given FaaS model (e.g. trying to eek out the lowest average and worst case latency; also working withi…

> ops will never be automated away; it can only be transmuted to a different form

Precisely, and the promise of FaaS is that it can become a black box you don't care anything about, and bundled into the markup you're paying on CPU, memory and network.

So far in real life it looks more like Ops is very much involved whenever you update a function or -- gods help you -- actually have to debug something in production. As long as this is the case it's a broken model, because as soon as you pull your Ops Ninja away from some other task you're right back in the bad old world of unpredictable costs.

My sense (as a relative newbie) is that the providers know this and are slowly trying to make stuff easier and more flexible -- and more predictable -- to deploy. Amazon Elastic Container Service being one example.

The catch is that the more standard it is, the less lock-in there is, and so far my experience with AWS suggests that lock-in is a major part of their strategy.

Re: Serverless: Cold Start War

#94

Post by abalone »

TrueTeller wrote: post #45
abalone wrote: post #22

Two things. 1- It's amazing that Java has the fastest cold start time! Faster than Nodejs.[1] That's exactly the opposite of what I've heard before. 2- I am so tired of hearing about cold start times for dormant apps as if that is the only cold start scenario. It is arguably a worse problem to have cold starts when scaling! What do I mean by cold starts when scaling? You adopt serverless. Things go great. Your app is…

Yes, one could prewarm N instances by sending N parallel requests every 5 minutes. You have to know N though :) For most apps, scale-out cold start won't be too much of a deal breaker: longer instance lifetime + shorter start will do the trick. And wait for the next wave of optimizations from vendors, I'm sure there's more to come. P.S. I'm the author of OP, thanks for reading!

> Yes, one could prewarm N instances by sending N parallel requests every 5 minutes.

Uhh.. you’re just speculating here right? I seriously doubt Lambda spins up a whole new container for every single “parallel” request in a short burst. There’s probably a little bit of queuing and/or they aren’t exactly parallel.

I think it’s something the vendors need to solve, like by directly prewarming prior to throwing traffic at it.

Oh and thank you for the detailed post!

Re: Serverless: Cold Start War

#95

Post by staticassertion »

village-idiot wrote: post #78
TrueTeller wrote: post #47

Earlier quoted context omitted.

That's all plausible except the price issue. Say, you keep 10 containers alive, so you make 10x 100ms calls every 5 minutes. That's gonna be $0.20 per month. 20 cents.

Internal math at my org says that lambdas cost about 20% more than an equivalent amount of EC2 power at full utilization. Now hitting full utilization is very hard, but the point is that lambas are only cheaper compared to underutilized EC2 hosts.

I'll pay 20% more for a system that I don't have to manage - the cost of patching and general EC2 maintenance is easily higher than that to me.

Re: Serverless: Cold Start War

#96

Post by matchagaucho »

stephenr wrote: post #32

Tech, to Business: So, hear me out guys. With the power of "The Cloud", we can break our compute workload down to the function level, and have them run as a service for us, rather than say an entire VM, or even an entire container. And because it's a "Cloud" service, we pay for what we use, so if there's no workload for the functions to service, there's no cost. We just pay for the time the tiny little container is a…

The ping costs on AWS to keep VMs warm are literally just pennies.

Re: Serverless: Cold Start War

#97

Post by staticassertion »

acjohnson55 wrote: post #26

I'm still a bit puzzled by the hype around FaaS. It seems like a useful tool for things where you don't want major queuing under pressure, but you can tolerate human perceptible delays. But it also seems easy to build a big ball of mud deeply tied to the nuances of the chosen FaaS provider. It just seems like most use cases are probably going to be just fine with more conventional horizontal scaling techniques. But I…

I'm using AWS Lambdas for a project. Every lambda listens on an SQS Queue. This isn't latency sensitive in the sense of user interaction, but I certainly care about latency to a degree - primarily throughput though.

The reason I've chosen this is because I do not want to manage an OS. Patch management is something I am not interested in tackling - it's a hard problem and AWS has it down anyways.

There are other reasons as well (I find ephemeral systems very appealing), but this is the most significant.

Re: Serverless: Cold Start War

#98

Post by hacknat »

While the idea of only paying for what you use seems to be an interesting proposition with serverless. I think the true cost savings come in no longer having to manage VMs.

With that in mind the coldstart problem can be avoided entirely with Fargate and Azure container groups. Sure you pay for an app to be on all the time, but you were doing that anyways.

I’m using Azure container groups and their managed DB service right now, and my CI pipeline is thin and I have had no need of an Ops person to manage VMs.

Re: Serverless: Cold Start War

#99

Post by komuW »

jondubois wrote: post #27

Lambda is not useful. It solves a few problems but creates even more new problems. Some problems include: - It makes managing multiple environments (e.g. development, staging, production) almost impossible. - It makes debugging difficult because you can't run the code on your own machine and step through the code. Most projects cannot be tested end-to-end due to environment incompatibilities between different service…

Some of the problems are easily solvable. I use up[https://github.com/apex/up] which is an open-source serverless framework to solve them.

a. it is possible to have multiple environments[https://up.docs.apex.sh/#configuration.stages]

b. you can run code on your machine[https://up.docs.apex.sh/#commands.start]

c. regarding vendo lockin; alternative serveless providers(apart from aws) are planned [https://github.com/apex/up/issues/4]

d. You're contributing to centralization of wealth and power instead of helping small companies and small open source projects. up is open-source but also has a paid plan[https://up.docs.apex.sh/#guides.subscribing_to_up_pro]

e. It takes the fun out of coding. I'm not sure about that.

Re: Serverless: Cold Start War

#100

Post by weego »

village-idiot wrote: post #78
TrueTeller wrote: post #47

Earlier quoted context omitted.

That's all plausible except the price issue. Say, you keep 10 containers alive, so you make 10x 100ms calls every 5 minutes. That's gonna be $0.20 per month. 20 cents.

Internal math at my org says that lambdas cost about 20% more than an equivalent amount of EC2 power at full utilization. Now hitting full utilization is very hard, but the point is that lambas are only cheaper compared to underutilized EC2 hosts.

That's ultimately the misuse 9f the technology though right. Unless it's a freak, business changing, spike in usage due to exposure or whatever that lambda are not critical path suitable and don't seem to have been implemented with that intent.

For example if your sign up process is lambda you did you got your cost benefit analysis wrong on anything other than a prototype or mvp, but if your 'change your profile photo', something that maybe happens a handful of times across a month on average per user at best, and you implemented that as your lambda to reduce load and delay scaling needs on your core infrastructure then that feels like you did it right.

Post reply on HN