It's unfortunate than ZGC and Shenandoah weren't tested nor parallel gc
AWS Lambda Cold Start Times
181–190 of 233 posts
Re: AWS Lambda Cold Start Times
#182Earlier quoted context omitted.
Same experience with Firebase. I just joined a team that has been using it. I've never worked with serverless before, and it boggles my mind how anyone thought it would be a good idea. The cold starts are horrendous. In one case, it's consistently taking about 7 seconds to return ~10K of data. I investigated the actual runtime of the function and it completes in about 20ms, so the only real bottleneck is the fucking…
I'm assuming by Firebase you mean Firebase Functions? We have a fairly complex infrastructure running off a combination of Firebase's RTDB, Firestore, and Google Cloud Functions and have never seen anywhere near what you're describing. Are you sure you're experiencing a 7 second "cold start", or is the invocation simply taking 7 seconds to run? Because the latter is far more easily explained.
I've confirmed that the actual execution of the function itself takes ~30ms, and that the time to download is standard (~200ms). That only leaves the cold start; nothing else makes sense.
edit: I even set up a node server on AppEngine itself, copied over the exact code to an endpoint, and it was taking 300-500ms tops (after the initial cold start of the AppEngine server).
Re: AWS Lambda Cold Start Times
#183Earlier quoted context omitted.
>It has automatic scaling, so it can scale up and down with traffic (all of which is configured in ECS) Doesn't scaling take time, though? Doesn't downloading a new docker container definition and starting it take at least as long as initializing a new lambda function? Also with lambda there's no configuring to do for scaling. If anything lambda gives you tools to limit the concurrency.
Thanks for pointing that out. I should have clarified because I agree that "Automatic" is a relative term. Lambda is entirely automatic like you point out. You literally don't need to think about it. You upload your function and it scales to meet demand (within limits). ECS however still requires configuration, but it is extremely simple to do. They actually call it "Service Auto-Scaling". Within there you choose a s…
The difference in scaling is more subtle than that. The thing that makes lambda so nice from scalability point of view is that you don't need to worry about the scalability of your application. You don't need any awkward async stuff or tune application server flags or anything like that. Your only concern with lambda code is to respond to one request as fast as possible. You can write something that burns 100% CPU in a busyloop per request in a lambda if you want and it'll scale all the same. In fargate making sure that the application is able to handle some economical amount of concurrency is your responsibility, and it can in some cases be very much non-trivial problem.
Re: AWS Lambda Cold Start Times
#184Earlier quoted context omitted.
Have you tried provision concurrency[1], this along with ARM based functions can help you with performance and reduce costs. 1 - https://aws.amazon.com/blogs/aws/new-provisioned-concurrency...
Provisioned concurrency is a bit of a non-starter for cold start latency reduction in user-facing applications. Its a tool in the toolbox, but the problem is: Let's say you set the provisioned concurrency at 10, then you have 11 concurrent requests come in. That 11th request will still get a cold start. PC doesn't scale automatically. The ping architecture of warming up functions does scale (better) in this setup. It…
At least in the OP’s case it seems the requests are infrequent. I understand your point about the 11th request, but at that point what % of your customers are impacted. The tail latency is always an issue, PC is a good way to solve that.
Re: AWS Lambda Cold Start Times
#185Earlier quoted context omitted.
Have you tried provision concurrency[1], this along with ARM based functions can help you with performance and reduce costs. 1 - https://aws.amazon.com/blogs/aws/new-provisioned-concurrency...
PC doesn't make that much sense for a use case like mine, where absolute cost minimization is the goal. My request volume (including ping requests) easily fits into the Lambda free tier, whereas with PC I'd be looking at some cost. More generally, I feel PC kinda defeats the purpose of Lambda to begin with. If I'm in that range of continuous load, I'd rather look at more traditional means of provisioning and hosting…
PC is a good way to keep certain amount of performance guarantees. It seems like you’re solution fits into the free tier, so probably a bad idea for you.
Re: AWS Lambda Cold Start Times
#186Earlier quoted context omitted.
7s cold start is weird. I think there has to be more to the story than that.
Same intuition. We run ~all our back on firebase cloud functions since Beta in 2017 and cold start has never been such a problem.
I'm extremely surprised to hear that. I know that there can be implementation differences, but on the level of application-code, this stuff is super simple. Create a javascript function then upload it. Not really much else to it, so I can't fathom what the difference is between your project and my own.
We're not doing anything crazy, it's just a basic CRUD application with minimal data (entire DB is less than 100MB at this point). And yet we're seeing constant, constant lags of several seconds for almost every single request. I can't explain it.
Re: AWS Lambda Cold Start Times
#187Earlier quoted context omitted.
>It has automatic scaling, so it can scale up and down with traffic (all of which is configured in ECS) Doesn't scaling take time, though? Doesn't downloading a new docker container definition and starting it take at least as long as initializing a new lambda function? Also with lambda there's no configuring to do for scaling. If anything lambda gives you tools to limit the concurrency.
I think it's just the trade off between these two scenarios. - Relatively poor amortized scale out time with good guarantees in the worst case. - Good amortized scale out time with dropped requests / timeouts in the worst case. With lambda, it doesn't really matter how spiky the traffic is. Users will see the cold start latency, albeit more often. With Fargate, users won't run into the cold start latencies - until th…
> With Fargate, users won't run into the cold start latencies - until they do, and the whole request may timeout waiting for that new server to spin up.
In practice that sort of setup is not trivial to accomplish with Fargate; normally while you are scaling up the requests get sent to the currently running tasks. There is no built-in ability to queue requests with Fargate(+ELB) so that they would then be routed to a new task. This is especially problematic if your application doesn't handle overloads very gracefully.
Re: AWS Lambda Cold Start Times
#188Earlier quoted context omitted.
Does your $250/mo include all the ops cost of other solutions?
Yes it does
You should look at offering this as a service perhaps. 2,500 250MB lambdas for $250/month with all AWS guarantees (ie, Multi-AZ, permissioning on every call etc etc) would be pretty compelling I think for folks running intermediate lambda workloads (ie, 5-10K lambadas at a time).
Re: AWS Lambda Cold Start Times
#189Earlier quoted context omitted.
Just use http functions in a Velo by Wix project - you will get 50ms cold start out of the box.
Thanks, will take a look. Can I run Java, or any kind of binary, there? That's why I'm not running this with CloudFlare workers, for instance. An alternative I'm actually considering is fly.io.
Velo runs only JavaScript, but unlike cloudflare, velo runs a full node.js.
With that in mind, you can a lot of things, from accessing your database or any other asset on aws (or gcp or azure). You can also use other wix apis, like payment api or the wix data database.
Re: AWS Lambda Cold Start Times
#190Earlier quoted context omitted.
Have you tried provision concurrency[1], this along with ARM based functions can help you with performance and reduce costs. 1 - https://aws.amazon.com/blogs/aws/new-provisioned-concurrency...
PC doesn't make that much sense for a use case like mine, where absolute cost minimization is the goal. My request volume (including ping requests) easily fits into the Lambda free tier, whereas with PC I'd be looking at some cost. More generally, I feel PC kinda defeats the purpose of Lambda to begin with. If I'm in that range of continuous load, I'd rather look at more traditional means of provisioning and hosting…
I think you are thinking of reserved concurrency - with reserved concurrency you can only run as many instances as you have reserved. With provisioned concurrency you can run as many as your quota allows but you are guaranteed to be able to handle as many concurrent instances have provisioned. Neither one of these are going to help you with cold start times, it is not defining the number of instances running, its just reserving a portion of quota so that they can run. Where provisioned/reserved concurrency comes in useful is keeping a run-away lambda function from starving out other functions either by guaranteeing a portion of the quota for the other functions or keeping a function from exceeding a number of concurrent executions.
The "ping" technique you mentioned is one way to keep a function warm but if lambda decides to start a second instance of the function because the hot one is handling a request, then that person is going to take a warm up hit and nothing you can do about that.
If you are really latency sensitive then lambda might not be the right choice for you. You can get a t4g.nano SPOT instance for about $3.50/month and you can keep that warm, but that is probably a whole lot more then you are paying for lambda.