AWS Lambda – Best practices • Hacker News Board

Re: AWS Lambda – Best practices

#21

Post by joecot » Thu, Mar 02, 2017, 2:07 PM UTC

I'm amused by the off-hand note about maybe using DynamoDB instead of a relational database, because the biggest issue with connecting your Lambda functions to MySQL is that using Lambdas within a VPC is absolutely terrible right now.

Sure, you can make lambda IP subnets in your VPC, assign your lambda to a subnet, give it a security group, and then you can easily setup your security group rules for your lambdas to connect. It's quite snazzy! Until you realize that cold startup times for your lambda normally are around a second, and when you have it running in your VPC, it takes 5 more seconds for it to make the VPC network interface. If you're using lambdas to make a REST API, this turns an occasional slight slowdown to a complete show stopper for your users.

There doesn't appear to be any good solution to this right now. You can host your Lambda outside the VPC, but then you essentially need to open your MySQL security group to the world (or every AWS Subnet, which is about the same thing). You can make a request to your Lambda function every 15 minutes, which is a terrible hack but works, except when you start approaching 50+ lambda functions you have to make heartbeat requests to every 15 minutes.

The issue is somewhat mitigated if, instead of leaning towards the natural inclination to make a separate Lambda function for each possible API request, you group API requests together, or use one Lambda function for your entire need. If your service is small in scope anyway, it won't particularly affect performance, and there will be less likelihood the lambda will go dormant/less functions you need to make heartbeat requests to. Serverless Framework has a good overview of the coding pattern options: https://serverless.com/blog/serverless-architecture-code-pat...

And yes, if you just jump straight from traditional server architectures straight to Lambda functions and Dynamodb NoSQL, you don't need VPC connections to connect to Database servers. But some of us change architecture paradigms one step at a time, Amazon!

Re: AWS Lambda – Best practices

#22

Post by optimuspaul » Thu, Mar 02, 2017, 2:21 PM UTC

throwaway2016a wrote: post #3

As someone who uses Lambda heavily I find this post somewhat disappointing from a "best practices" standpoint. No mention of Lambda best practices like: - Using CloudFormation - IAM policies - Managing config - How to handle databases and connection pools Instead we get a logging suggestion that while interesting (I too prefer ElasticSearch to CloudWatch logs) is definitely not Lambda best practice. And suggestion ab…

Lambda on Cloudformation is just sad. Unless you don't care about versioning, then by all means jump on that ship. I personally don't want to spin up a new stack to deploy a new version safely.

Re: AWS Lambda – Best practices

#23

Post by joecot » Thu, Mar 02, 2017, 2:28 PM UTC

danielhunt wrote: post #12

Unfortunately, there's absolutely nothing here regarding things like git, or CI/CD, or how-to-use-it-when-a-medium-to-large-team-is-involved If anyone has any research, thoughts, or comments on this side of Lambda, I'd love to learn more

IMO one of the better ways to handle managing config and functions is using the Serverless Framework: https://serverless.com/

Your configuration and your code both live together, and can be stored in git together. You can create separate environments -- dev, staging, production, etc -- and deploy to them separately.

There aren't great options for testing in Lambda right now, and for the most part the advice I've seen is to make testable libraries, and then hook them into lambda.

Re: AWS Lambda – Best practices

#24

Post by ahallock » Thu, Mar 02, 2017, 2:33 PM UTC

> My opinion, when going serverless, I wouldn’t think of using RDBMS’s (although you can) and instead use databases like Elasticsearch, DynamoDB, Mongo etc.

This is a vague statement that's not really based on evidence, and I'm not sure why using Postgres, for example, is going to be so much worse for most use cases than these NoSQL solutions.

Re: AWS Lambda – Best practices

#25

Post by joecot » Thu, Mar 02, 2017, 2:36 PM UTC

ahallock wrote: post #24

> My opinion, when going serverless, I wouldn’t think of using RDBMS’s (although you can) and instead use databases like Elasticsearch, DynamoDB, Mongo etc. This is a vague statement that's not really based on evidence, and I'm not sure why using Postgres, for example, is going to be so much worse for most use cases than these NoSQL solutions.

As I explain over here (https://news.ycombinator.com/item?id=13774041), trying to use Lambdas with your existing servers, like Postgres, can cause some serious complications.

Re: AWS Lambda – Best practices

#26

Post by zurn » Thu, Mar 02, 2017, 2:41 PM UTC

jaxondu wrote: post #14

hendry wrote: post #10

Earlier quoted context omitted.

I like in https://serverless.com/ 's serverless.yml how it provisions the CloudFormation stuff and gives you a way to manage IAM permissions.

AWS has its own Serverless Application Model (SAM) https://github.com/awslabs/serverless-application-model released last Oct. Rather than learning another framework, you'd use the same CloudFormation yaml syntax to define your stack and apps. It simplifies calling Lambda functions by providing a higher level abstraction triggers called Events. This makes it simpler to create Lambda functions for Rest API, Cron schedu…

Serverless supports other providers besides AWS, so it's better if your aren't centered around AWS.

Re: AWS Lambda – Best practices

#27

Post by throwaway2016a » Thu, Mar 02, 2017, 2:42 PM UTC

jaxondu wrote: post #14

hendry wrote: post #10

Earlier quoted context omitted.

I like in https://serverless.com/ 's serverless.yml how it provisions the CloudFormation stuff and gives you a way to manage IAM permissions.

AWS has its own Serverless Application Model (SAM) https://github.com/awslabs/serverless-application-model released last Oct. Rather than learning another framework, you'd use the same CloudFormation yaml syntax to define your stack and apps. It simplifies calling Lambda functions by providing a higher level abstraction triggers called Events. This makes it simpler to create Lambda functions for Rest API, Cron schedu…

Personally I use the Serverless framework which supports other cloud providers and is generally more on the cutting edge than SAM (and pre-dates SAM).

But SAM is certainly a valid option if you are OK with vendor lock in. (which many people are)

Re: AWS Lambda – Best practices

#28

Post by throwaway2016a » Thu, Mar 02, 2017, 2:43 PM UTC

optimuspaul wrote: post #22

throwaway2016a wrote: post #3

As someone who uses Lambda heavily I find this post somewhat disappointing from a "best practices" standpoint. No mention of Lambda best practices like: - Using CloudFormation - IAM policies - Managing config - How to handle databases and connection pools Instead we get a logging suggestion that while interesting (I too prefer ElasticSearch to CloudWatch logs) is definitely not Lambda best practice. And suggestion ab…

Lambda on Cloudformation is just sad. Unless you don't care about versioning, then by all means jump on that ship. I personally don't want to spin up a new stack to deploy a new version safely.

There are ways to handle versioning of Lambda and CloudFormation. Serverless framework does it.

With creative use of deletion policy you can do pretty well.

CloudFormation is pretty terrible about managing changes on DynamoDb databases, though.

Re: AWS Lambda – Best practices

#29

Post by throwaway2016a » Thu, Mar 02, 2017, 2:45 PM UTC

hn_user2 wrote: post #15

cel1ne wrote: post #8

Earlier quoted context omitted.

Off-topic question: Why do people use throwaway accounts here on HN? Why is it so important that nobody knows who the real person behind posts is?

Can't speak for parent. For me, I don't care if people know the real person behind the post. But I don't want someone to be able to see a list of everything I have ever said. And unfortunately there is no way to do one without the other. Just like everyday conversation. I can speak without anonymity. But I would hate for someone to go look at some of the things I said 20 years ago.

Same here.

Re: AWS Lambda – Best practices

#30

Post by throwaway2016a » Thu, Mar 02, 2017, 2:46 PM UTC

foota wrote: post #6

throwaway2016a wrote: post #5

Earlier quoted context omitted.

I've been using Lambda since the beginning (was an early private beta user). I am actually considering writing a book or perhaps some courseware. I don't blog. Unfortunately, if I do I cannot post it here as it would defeat the purpose of using a throw-away account :(

Your throwaway has 1500 karma?

Yeah. I guess anonymous is a better word than "throw away" at this point.