Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Serverless: Cold Start War

mikhail.io

Post reply on HN
Print view View original

101–110 of 110 posts

Re: Serverless: Cold Start War

#101

Post by xrd »

manigandham wrote: post #15
xrd wrote: post #5

I'm still really puzzled why AWS lambda has golang and Azure functions has Java and python but GCF doesn't have anything but JavaScript. That's a big hole that must be leaving a lot of developers feeling frustrated.

GCP is just very slow at releasing and behind in both features and services as compared to AWS and Azure. The trade-off is that the GA services are usually more consistent, cheaper, faster and easier to use and integrate. GCF is still in beta so it's the worst case, but they recently announced Serverless Containers which will let you run anything insider a docker container on-demand. That'll get around the language b…

I do feel like "Firebase Cloud Functions" (GCF) is generally more polished than AWS Lambda. Lambda seems a little like it was slapped together and the UI often seems a little incoherent, which probably reflects their first mover advantages and disadvantages. It is not always straighforward to use the CLI tools for Lambda as well (but you can just wrap them in a bash script and forget about it generally). But, there are subtle gotchas with working in JavaScript (Promises!) especially within the context of a lambda/function. If you return the wrong thing inside your function or misunderstand the way promises work, you can get something that looks like it works and passes lint, but fails in production, and the logging pipelines for both AWS and GCF leave a lot to be desired.

Re: Serverless: Cold Start War

#102

Post by manigandham »

xrd wrote: post #29
steren wrote: post #23

Earlier quoted context omitted.

GCF is now Generally Available. And to sign up for serverless containers: https://g.co/serverlesscontainers (I am a PM on GCP)

I'm assuming there is a blog post detailing this somewhere, can you share it here?

Here's the GCP Next 2018 video: https://www.youtube.com/watch?v=Y1sRy0Q2qig

https://cloudplatform.googleblog.com/2018/07/bringing-the-be...

Re: Serverless: Cold Start War

#103

Post by tomcam »

ssijak wrote: post #16
tomcam wrote: post #3

One of my favorite articles of recent times. What jumped out at me was that there is a dramatic difference in start up time depending on how much memory your cloud server has: the more memory, the faster the start up time.

More memory = faster CPU allocated

Which is not necessarily intuitive. For all I knew there was some kind of penalty of a 1025MB memory image vs 512MB. Didn't think so... but I'm increasingly baffled by hardware issues in an increasingly virtualized world.

Re: Serverless: Cold Start War

#104

Post by DenseComet »

stephenr wrote: post #33
wimbledon wrote: post #31

I think there is a business case for a small app where you could input the url of your function and have it called every ‘x’ seconds or minutes to keep the function warm. Would anyone here use it? I can put together a small app in a few days.

A few days to put `curl $url > /dev/null` in a crontab? Are you kidding me?

Its not a few days to do only that. Its a few days to create the next million dollar idea.

Re: Serverless: Cold Start War

#105

Post by zzzcpan »

mrep wrote: post #87
village-idiot wrote: post #77

Earlier quoted context omitted.

My takeaway is that lambdas are pointless if you’re latency sensitive. If you’re not latency sensitive, such as queue to queue lambdas, then go for it.

Yes. It is pretty much useless for web servers as even the scale out from 1 vm container to 2 will require a cold start. In hindsight, this is kind of obvious as you don't expect them to keep a bunch of containers loaded with your dependencies ready to serve requests. They might offer something like that eventually, but at that point your setup is basically the same as ec2 servers loaded with your service in autoscal…

> you don't expect them to keep a bunch of containers loaded with your dependencies ready to serve requests

I think you would expect them to at least use swap for density and always keep one extra instance running and ready to serve requests. It's not like people generate functions on the fly, so it shouldn't even cost anything extra. Swap will help with unnecessary things kept in memory.

Re: Serverless: Cold Start War

#106

Post by jacques_chester »

TrueTeller wrote: post #47
stephenr wrote: post #32

Tech, to Business: So, hear me out guys. With the power of "The Cloud", we can break our compute workload down to the function level, and have them run as a service for us, rather than say an entire VM, or even an entire container. And because it's a "Cloud" service, we pay for what we use, so if there's no workload for the functions to service, there's no cost. We just pay for the time the tiny little container is a…

That's all plausible except the price issue. Say, you keep 10 containers alive, so you make 10x 100ms calls every 5 minutes. That's gonna be $0.20 per month. 20 cents.

This assumes AWS will never reach a point where they decide to stop subsidising this behaviour by keeping the instances alive.

Re: Serverless: Cold Start War

#107

Post by mrep »

zzzcpan wrote: post #105
mrep wrote: post #87

Earlier quoted context omitted.

Yes. It is pretty much useless for web servers as even the scale out from 1 vm container to 2 will require a cold start. In hindsight, this is kind of obvious as you don't expect them to keep a bunch of containers loaded with your dependencies ready to serve requests. They might offer something like that eventually, but at that point your setup is basically the same as ec2 servers loaded with your service in autoscal…

> you don't expect them to keep a bunch of containers loaded with your dependencies ready to serve requests I think you would expect them to at least use swap for density and always keep one extra instance running and ready to serve requests. It's not like people generate functions on the fly, so it shouldn't even cost anything extra. Swap will help with unnecessary things kept in memory.

1: I've created a lot of experement functions that I hardly ever use and have ones that are no longer used so that would be expensive.

2. The last company I worked for isolated functions to vms on a customer level. That way, if someone hacked their way past the function container, they would then still have to hack their way past the VM container to access another customers data which is what you would have to do anyways on the public VM offering.

Re: Serverless: Cold Start War

#108

Post by mikestaub »

abalone wrote: post #94
TrueTeller wrote: post #45

Earlier quoted context omitted.

Yes, one could prewarm N instances by sending N parallel requests every 5 minutes. You have to know N though :) For most apps, scale-out cold start won't be too much of a deal breaker: longer instance lifetime + shorter start will do the trick. And wait for the next wave of optimizations from vendors, I'm sure there's more to come. P.S. I'm the author of OP, thanks for reading!

> Yes, one could prewarm N instances by sending N parallel requests every 5 minutes. Uhh.. you’re just speculating here right? I seriously doubt Lambda spins up a whole new container for every single “parallel” request in a short burst. There’s probably a little bit of queuing and/or they aren’t exactly parallel. I think it’s something the vendors need to solve, like by directly prewarming prior to throwing traffic a…

I agree the vendors needs to solve this. In the meantime I think you can hack it by using Kinesis shards ( one lambda per shard ) if you know you will need many lambdas in advance. https://github.com/awslabs/aws-lambda-kinesis-prewarming

Re: Serverless: Cold Start War

#109

Post by mwcampbell »

jacques_chester wrote: post #67
iamjustlooking wrote: post #14

I commented on a thread a couple weeks ago about Cloud Functions on GCP here: https://news.ycombinator.com/item?id=17796893 Eager to test it out we ran thousands of tests attempts with different RAM sizes and I can corroborate this persons findings in regards to the reduction of cold start time from functions with larger RAM allocations and seeming unpredictability of cold start on GCP. I hope with time they will imp…

From the parts I can see in Knative-land, it's being given a lot of thought. My view is that the biggest improvement to be made is in smarter handling of raw bits. Kubernetes doesn't quite understand disk locality yet and most docker images are less-than-ideally constructed in any case.

What do you find suboptimal about most Docker images? Just the size, or something else?

Re: Serverless: Cold Start War

#110

Post by jacques_chester »

mwcampbell wrote: post #109
jacques_chester wrote: post #67

Earlier quoted context omitted.

From the parts I can see in Knative-land, it's being given a lot of thought. My view is that the biggest improvement to be made is in smarter handling of raw bits. Kubernetes doesn't quite understand disk locality yet and most docker images are less-than-ideally constructed in any case.

What do you find suboptimal about most Docker images? Just the size, or something else?

I wrote several thousand words on the topic a few months (email me for the link).

The gist is: you can make an image easy for developers, or you can make it performant in production, but you cannot have both.

Ease of development typically leads to kitchen-sink images or squashed images, but production performance requires careful attention to the ordering and content of layers.

Post reply on HN