AWS Lambda – Best practices • Hacker News Board

Re: AWS Lambda – Best practices

#41

Post by joecot » Thu, Mar 02, 2017, 3:59 PM UTC

bpicolo wrote: post #39

joecot wrote: post #25

Earlier quoted context omitted.

As I explain over here ( https://news.ycombinator.com/item?id=13774041 ), trying to use Lambdas with your existing servers, like Postgres, can cause some serious complications.

If that's the case, seems like a great argument against using lambda ;) Side note, is that just startup time for a single lambda `node`? Won't you be running >>1 node and not really care about startup time for any single node?

It's every time you have to start up a lambda instance.

So if you're low, continuous traffic, it'll happen once and you won't really care. If you're high traffic, occasionally different instances will get started up, will also need their node network interface, and also take 6+ seconds. And if you're low traffic but not continuous, if there's a pause for more than around 15 minutes (entirely depends on how much demand there is for lambda and how rapidly they clean up old containers), for the next request you get you'll have to cold start again.

Re: AWS Lambda – Best practices

#42

Post by mwcampbell » Thu, Mar 02, 2017, 4:09 PM UTC

Can a given Lambda container process ever be used for multiple concurrent function invocations? If not, then we're dealing with at least 128 MB of RAM per concurrent request. Multithreaded and event-driven servers can do much better than that.

Re: AWS Lambda – Best practices

#43

Post by Just1689 » Thu, Mar 02, 2017, 4:10 PM UTC

mwcampbell wrote: post #42

Can a given Lambda container process ever be used for multiple concurrent function invocations? If not, then we're dealing with at least 128 MB of RAM per concurrent request. Multithreaded and event-driven servers can do much better than that.

Yes

See: https://aws.amazon.com/blogs/compute/container-reuse-in-lamb... http://stackoverflow.com/questions/37523128/how-aws-lambda-c...

Re: AWS Lambda – Best practices

#44

Post by [deleted] » Thu, Mar 02, 2017, 4:12 PM UTC

Just1689 wrote: post #43

mwcampbell wrote: post #42

Can a given Lambda container process ever be used for multiple concurrent function invocations? If not, then we're dealing with at least 128 MB of RAM per concurrent request. Multithreaded and event-driven servers can do much better than that.

Yes See: https://aws.amazon.com/blogs/compute/container-reuse-in-lamb... http://stackoverflow.com/questions/37523128/how-aws-lambda-c...

[deleted]

Re: AWS Lambda – Best practices

#45

Post by EnFinlay » Thu, Mar 02, 2017, 4:16 PM UTC

throwaway2016a wrote: post #3

As someone who uses Lambda heavily I find this post somewhat disappointing from a "best practices" standpoint. No mention of Lambda best practices like: - Using CloudFormation - IAM policies - Managing config - How to handle databases and connection pools Instead we get a logging suggestion that while interesting (I too prefer ElasticSearch to CloudWatch logs) is definitely not Lambda best practice. And suggestion ab…

I agree. This is less a description of best practices and more of a stream of consciousness about Lambda.

Re: AWS Lambda – Best practices

#46

Post by grzm » Thu, Mar 02, 2017, 4:25 PM UTC

throwaway2016a wrote: post #30

foota wrote: post #6

Earlier quoted context omitted.

Your throwaway has 1500 karma?

Yeah. I guess anonymous is a better word than "throw away" at this point.

Or even pseudonymous. You've posted enough to have established a reputation and personality. It's a name, just not your real one.

Re: AWS Lambda – Best practices

#47

Post by IanCal » Thu, Mar 02, 2017, 4:31 PM UTC

hendry wrote: post #10

throwaway2016a wrote: post #3

As someone who uses Lambda heavily I find this post somewhat disappointing from a "best practices" standpoint. No mention of Lambda best practices like: - Using CloudFormation - IAM policies - Managing config - How to handle databases and connection pools Instead we get a logging suggestion that while interesting (I too prefer ElasticSearch to CloudWatch logs) is definitely not Lambda best practice. And suggestion ab…

I like in https://serverless.com/ 's serverless.yml how it provisions the CloudFormation stuff and gives you a way to manage IAM permissions.

I tried to use serverless, but I don't have full admin permissions and the site is unable to say what permissions it actually requires to work. I'll swing back round to it later in the year and see how it's progressing.

Re: AWS Lambda – Best practices

#48

Post by talawahdotnet » Thu, Mar 02, 2017, 4:42 PM UTC

hendry wrote: post #10

throwaway2016a wrote: post #3

As someone who uses Lambda heavily I find this post somewhat disappointing from a "best practices" standpoint. No mention of Lambda best practices like: - Using CloudFormation - IAM policies - Managing config - How to handle databases and connection pools Instead we get a logging suggestion that while interesting (I too prefer ElasticSearch to CloudWatch logs) is definitely not Lambda best practice. And suggestion ab…

I like in https://serverless.com/ 's serverless.yml how it provisions the CloudFormation stuff and gives you a way to manage IAM permissions.

Agreed, at first I was skeptical because the marketing message on the site seemed a bit over-hyped, but it turns out it is a really valuable tool for streamlining both testing (invoke local) and deployment when working with Lambda. New users also benefit for the basic scaffolding (severless create) and the documentation is pretty decent.

I have tried kappa (Python) and node-lambda (Node.js), and serverless works better than both of them, is more frequently updated (Lambda and API Gateway themselves change frequently), supports more languages (I have used it with Java, Python and Node.js) and has a much larger, more active community.

The API gateway support is pretty simple and solid, and they also allow you to use cloud formation rules directly so it is very flexible.

Re: AWS Lambda – Best practices

#49

Post by eof » Thu, Mar 02, 2017, 4:43 PM UTC

Maybe I am too late too this party but I have a question:

Is there any framework for handling routing and deployments?

What I mean is, is there ane xisting solution where I can make JS files to routes, and have those routes and js files be loaded into Lambda and configure the API end point; so that if I need to make an update to the lambda function i can just update the file (and re run the framework)

Re: AWS Lambda – Best practices

#50

Post by pastaking » Thu, Mar 02, 2017, 4:45 PM UTC

gamache wrote: post #19

pastaking wrote: post #18

Earlier quoted context omitted.

How do you use database connection pools with lambda? I thought it wasn't possible.

AWS will sometimes reuse the container in which your Lambda runs, so if you set up e.g., database pools during load time, they're often still there the next time. Related: https://aws.amazon.com/blogs/compute/container-reuse-in-lamb...

Yes, but there's no guarantee that lambda will reuse a connection / won't continue to request new connections against your database :( Currently, it seems like there's no "concurrency limit" or "per-function throttling". It would be amazing if requests are just magically queued up in SQS and lambda just processes them... like a traditional queue-worker system except now we don't have to manage the infra.