Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS Lambda – Best practices

cloudncode.blog

Post reply on HN
Print view View original

51–60 of 68 posts

Re: AWS Lambda – Best practices

#51

Post by kevan »

alexvy86 wrote: post #40
gamache wrote: post #19

Earlier quoted context omitted.

AWS will sometimes reuse the container in which your Lambda runs, so if you set up e.g., database pools during load time, they're often still there the next time. Related: https://aws.amazon.com/blogs/compute/container-reuse-in-lamb...

I'm not familiar at all with Amazon's cloud, so I might be misinterpreting... but when you say " sometimes reuse..." and "they're often still there..." it sounds to me like they might as well not be? How would that be a good foundation for an app deployed in Lambda?

Think of Lambdas like autoscaling containers. If you're operating with decent load, there's a good chance your container will stay up and be reused for multiple requests. But if load drops, your containers might be shut down so they aren't taking up resources while they're not being used. When load increases again, more instances will be spun up.

It's the same approach as autoscaling VMs, but at a function level.

Re: AWS Lambda – Best practices

#52

Post by 013a »

After using lambda for a while, the opinion I've landed on is that the concept of serverless is a powerful one in specific domains, but (1) being the zeitgeist it is encouraged for use in domains where it doesn't belong, and (2) AWS Lambda itself isn't very good, for reasons that are specific to Lambda, not to the concept of serverless.

Re: AWS Lambda – Best practices

#53

Post by ahallock »

joecot wrote: post #25
ahallock wrote: post #24

> My opinion, when going serverless, I wouldn’t think of using RDBMS’s (although you can) and instead use databases like Elasticsearch, DynamoDB, Mongo etc. This is a vague statement that's not really based on evidence, and I'm not sure why using Postgres, for example, is going to be so much worse for most use cases than these NoSQL solutions.

As I explain over here ( https://news.ycombinator.com/item?id=13774041 ), trying to use Lambdas with your existing servers, like Postgres, can cause some serious complications.

Thanks for the explanation. So it isn't really RDBMS vs NoSQL inside Lambda, but issues with using Lambda on your own VPC.

Re: AWS Lambda – Best practices

#54

Post by jedberg »

013a wrote: post #52

After using lambda for a while, the opinion I've landed on is that the concept of serverless is a powerful one in specific domains, but (1) being the zeitgeist it is encouraged for use in domains where it doesn't belong, and (2) AWS Lambda itself isn't very good, for reasons that are specific to Lambda, not to the concept of serverless.

I'm curious if you could expand on:

> AWS Lambda itself isn't very good, for reasons that are specific to Lambda

Re: AWS Lambda – Best practices

#55

Post by stlava »

I run a fairly high scale pipeline on lambda. The post touched briefly on logging but wanted to chime in with a few helpful points:

- Format all logs in json. This allows you to use the json path filtering option on CW.

- Add function version & tag to the log lines. CW doesn't allow you to filter based on version.

- Don't expect CW log querying to work all the time.

- Output runtime stats to CW metrics.

edit: formatting

Re: AWS Lambda – Best practices

#56

Post by giaour »

pastaking wrote: post #50
gamache wrote: post #19

Earlier quoted context omitted.

AWS will sometimes reuse the container in which your Lambda runs, so if you set up e.g., database pools during load time, they're often still there the next time. Related: https://aws.amazon.com/blogs/compute/container-reuse-in-lamb...

Yes, but there's no guarantee that lambda will reuse a connection / won't continue to request new connections against your database :( Currently, it seems like there's no "concurrency limit" or "per-function throttling". It would be amazing if requests are just magically queued up in SQS and lambda just processes them... like a traditional queue-worker system except now we don't have to manage the infra.

That's essentially how asynchronously invoked (e.g., triggered by an event in S3 or DynamoDB) lambda tasks work. They're retried if the original invocation doesn't execute successfully, and if the third try fails, the triggering event is sent to a dead letter queue in SQS.

Re: AWS Lambda – Best practices

#57

Post by breandr »

throwaway2016a wrote: post #37
wiredfool wrote: post #35

Every time I look at Lambda+API Gateway for a web api, there's always been one little thing that makes things way harder than they should be. A while ago (pre proxy), it was nearly impossible to send back 302 redirects cleanly. With LAMBDA_PROXY, it's not clear that you can actually return binary data through the Api Gateway. There's the undocumented isBase64Encoded parameter, but as far as I can tell, it's ignored.…

Binary data is tricky with Lambda proxy (and by tricky I mean... I haven't gotten it to work). But I also haven't tried much. Typically best practice for me has been: - When dealing with uploads have an API call to get a pre-signed S3 upload URL - When dealing with downloads using HATEOAS and/or the Location header to send the client to S3 to grab the actual file. It requires some client-side designing around eventua…

You need to specify your MIME type (most commonly, application/json) to be treated as binary (from console: API -> Binary Support). In your Lambda function: `callback(null, { body: base64EncodedBody, status: 200, headers: {}, isBase64Encoded: true })`

Re: AWS Lambda – Best practices

#58

Post by brilliantcode »

I gave up on Lambda when I realized it was cheaper to use a $5/month VPS or EC2 which is cheaper and more responsive without cold startup boot time.

There will be niche use cases for Lambda but Serverless(TM) is just another passing buzzword. It won't see wide adoption unless the cold startup and costs are significantly lowered than current solutions.

Re: AWS Lambda – Best practices

#59

Post by wiredfool »

breandr wrote: post #57
throwaway2016a wrote: post #37

Earlier quoted context omitted.

Binary data is tricky with Lambda proxy (and by tricky I mean... I haven't gotten it to work). But I also haven't tried much. Typically best practice for me has been: - When dealing with uploads have an API call to get a pre-signed S3 upload URL - When dealing with downloads using HATEOAS and/or the Location header to send the client to S3 to grab the actual file. It requires some client-side designing around eventua…

You need to specify your MIME type (most commonly, application/json) to be treated as binary (from console: API -> Binary Support). In your Lambda function: `callback(null, { body: base64EncodedBody, status: 200, headers: {}, isBase64Encoded: true })`

The mimetype of the response from lambda? Or the mimetype in the headers?

Re: AWS Lambda – Best practices

#60

Post by collyw »

joecot wrote: post #21

I'm amused by the off-hand note about maybe using DynamoDB instead of a relational database, because the biggest issue with connecting your Lambda functions to MySQL is that using Lambdas within a VPC is absolutely terrible right now. Sure, you can make lambda IP subnets in your VPC, assign your lambda to a subnet, give it a security group, and then you can easily setup your security group rules for your lambdas to c…

I am curious about the type of workloads that you do that will require serverless architecture yet are backed by a MySQL database. It was only 5 or so years ago that we were told that relational databases didn't scale so we all needed NoSQL.
Post reply on HN