AWS Lambda Cold Start Times • Hacker News Board

Re: AWS Lambda Cold Start Times

#201

Post by laurencerowe » Tue, Oct 12, 2021, 9:16 PM UTC

wpietri wrote: post #160

musingsole wrote: post #142

Earlier quoted context omitted.

You increase the usage of the lambda, sure, but not by double unless all this lambda does is respond to the search. There's a point where this is all semantics when running cloud services with regard to how you've deployed your code behind gateways and lambdas, but most applications have obvious triggers that could warm up a lambda at 10% or less overhead. And even then....lambdas are dirt cheap.

Lambdas can be dirt cheap if you use them well. But that's always what happens. I recently saw a report about contractor who used them for crawling websites. Their client saw a surprise $12k bill for Lambda, when a simple Scrapy crawler on a low-end instance would have cost them Why? Because if you spin up a ton of Lambda invocations and have them sit around waiting for the network, you pay for each CPU to sit idle,…

Couldn't this also have been handled efficiently in Lambda had it been configured to use batching? (Assuming it was implemented as async invocations driven by a queue of some sort.)

It does seem unfortunate that Lambda cannot be configured to support concurrent synchronous invocations for serving HTTP requests concurrently.

Re: AWS Lambda Cold Start Times

#202

Post by [deleted] » Tue, Oct 12, 2021, 9:26 PM UTC

Zababa wrote: post #200

rough-sea wrote: post #199

Earlier quoted context omitted.

> just As if everything else is the same. It is a completely independent code base.

Well, the performance seems to be mostly the same https://mayankchoubey.github.io/Deno-vs-Node-Performance/ . > It is a completely independent code base. Not exactly, the heavy lifting is done by v8 on both sides. Deno can do lots of things around the ergonomics, and switch around the event loop (though libuv is already pretty good), but outside of that they are mostly equivalent.

[deleted]

Re: AWS Lambda Cold Start Times

#203

Post by asdev » Tue, Oct 12, 2021, 9:58 PM UTC

This guy has such a rudimentary understanding that he can't point to real examples of Ethereum applications that support his point. Most smart contracts are immutable and non-upgradeable, which nullifies this entire blog post. Many protocols are also moving to DAO governance for even further decentralization of power. This reflects a highly pervasive issue as people with a somewhat technical background but no experience with crypto networks and applications think they are qualified to give opinions on whether the protocols will make it or not.

Re: AWS Lambda Cold Start Times

#204

Post by WatchDog » Tue, Oct 12, 2021, 10:17 PM UTC

015a wrote: post #170

websap wrote: post #157

Earlier quoted context omitted.

Have you tried provision concurrency[1], this along with ARM based functions can help you with performance and reduce costs. 1 - https://aws.amazon.com/blogs/aws/new-provisioned-concurrency...

Provisioned concurrency is a bit of a non-starter for cold start latency reduction in user-facing applications. Its a tool in the toolbox, but the problem is: Let's say you set the provisioned concurrency at 10, then you have 11 concurrent requests come in. That 11th request will still get a cold start. PC doesn't scale automatically. The ping architecture of warming up functions does scale (better) in this setup. It…

Hit the nail on the head with this one. The pinging approach incredibly more cost-effective than provisioned concurrency.

Re: AWS Lambda Cold Start Times

#205

Post by WatchDog » Tue, Oct 12, 2021, 10:38 PM UTC

wmfiv wrote: post #76

VWWHFSfQ wrote: post #15

Earlier quoted context omitted.

Provisioned concurrency is insanely expensive. If you have any kind of a thundering herd access pattern then Lambda is a complete non-starter because of the warm-up and scaling characteristics. We eventually just put an nginx/openresty server on a regular medium EC2 instance and got rid of Lambda from our stack completely and now we're paying about 1/300th the cost we were previously and the performance is infinitely…

Insanely expensive is definitely a flexible term. I think numbers help here. Provisions Concurrency $8.64 / GB / month 256 MB per Lambda (Assuming Python, Ruby, NodeJS, or Rust) $2.16 per Lambda per month A lot of organizations can probably make a good business case for keeping 100s or even 1000s of Lambda's warm. You also don't need to keep them warm 24x7, can get an additional 12% discount using savings plans, and…

1 GB of lambda provisioned concurrency is $10.95 USD a month in us-east-1.

That's what you pay for your lambda sitting there doing nothing.

You can get an on-demand ec2 instance with 1 GB of RAM, for $9.13 USD a month, or $6.87 if you get a reserved instance.

You can fully utilize those instances the whole month.

Source: https://calculator.aws/#/estimate?id=7e1d1c2f32a2c63ba4ded19...

Re: AWS Lambda Cold Start Times

#206

Post by ignoramous » Tue, Oct 12, 2021, 10:42 PM UTC

Zababa wrote: post #200

rough-sea wrote: post #199

Earlier quoted context omitted.

> just As if everything else is the same. It is a completely independent code base.

Well, the performance seems to be mostly the same https://mayankchoubey.github.io/Deno-vs-Node-Performance/ . > It is a completely independent code base. Not exactly, the heavy lifting is done by v8 on both sides. Deno can do lots of things around the ergonomics, and switch around the event loop (though libuv is already pretty good), but outside of that they are mostly equivalent.

That blog post is damning! Thanks for sharing it. btw, deno publishes its own benchmarks: https://deno.land/benchmarks

Re: AWS Lambda Cold Start Times

#207

Post by pugz » Tue, Oct 12, 2021, 10:49 PM UTC

davidjfelix wrote: post #109

You can shave even more coldstart time off the golang version by building with `go build -tags lambda.norpc` and deploying it as a custom runtime.

Is that build tag necessary? I should check out what it does, because I've been deploying Go functions to custom runtimes without it. Does it skip some kind of "determine if Go-specific RPC or custom runtime" check?

EDIT: I found it. Thanks for the tip! https://github.com/aws/aws-lambda-go/blob/1f782848d89c02bf80...

Re: AWS Lambda Cold Start Times

#208

Post by aahortwwy » Tue, Oct 12, 2021, 10:50 PM UTC

gunnarmorling wrote: post #117

The best cold starts are those which aren't noticed by the user. For my blog search (which runs on Lambda), I found a nice way of achieving that [1]: as soon as a user puts the focus to the input field for the search text, this will already submit a "ping" request to Lambda. Then, when they submit the actual query itself, they will hit the already running Lambda most of the times. And, as others said, assigning more…

> assigning more RAM to your Lambda than it actually may need [...] increases the assigned CPU shares

"serverless" indeed

Re: AWS Lambda Cold Start Times

#209

Post by mulmboy » Tue, Oct 12, 2021, 11:06 PM UTC

A pattern I have implemented is to have my API code on both ECS/Fargate and Lambda at the same time, and send traffic to the appropriate one using an Elastic Load Balancer. I flag specific endpoints as "cpu intensive" and have them run on lambda.

Implemented by

- Duplicating all routes in the API with the "/sls/" prefix (this is a couple of lines in FastAPI)

- Setting up a rule in ELB to route to Lambda if the route starts with /sls, or to ECS otherwise.

- Set up the CPU intensive routes to automatically respond with a 307 to the same route but prefixed with /sls.

Boom, with that the system can handle bursts of CPU intensive traffic (e.g. data exports) while remaining responsive to the simple 99% of requests all on one vCPU.

And the same dockerfile, with just a tiny change, can be used both in ECS and Lambda.

Re: AWS Lambda Cold Start Times

#210

Post by tiew9Vii » Tue, Oct 12, 2021, 11:32 PM UTC

The Rust metrics are interesting.

I've been getting around 20ms cold starts, 1ms warm exec on the 128MB ARM Graviton2 using Rust for the most basic test cases. Graviton2 was slightly slower on cold starts than X86 for me (1-2ms) but who doesn't want to save $0.0000000004 per execution? Adding calls to parameter store/dynamo DB bumps it up a little but still Memory usage is 20-30MB, and I haven't done anything to optimise memory. I know I can get rid of a few allocations I'm doing for simplicity if I want to.

I've not always been the greatest fan of Lambdas seeing it has hidden complexity orchestrating and a blackbox for debugging. Re-visiting a few years on and with Rust, you get an excellent language, excellent runtime characteristics and substantial cost savings unless you really need more than 128MB memory, i.e. processing large volumes of data per execution in memory or transcoding. Any asynchronous/event-driven service I write, I'll just package as a Rust lambda going forward and pay fractions of a cent per month. I am still on-the-wall with HTTP exposed services as that's a big plumbing exercise and hidden gateway costs but not as adverse to it as I was.