Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Behind the scenes, AWS Lambda

bschaatsbergen.com

Post reply on HN
Print view View original

31–40 of 89 posts

Re: Behind the scenes, AWS Lambda

#31

Post by cloakandswagger »

carlosf wrote: post #21

Really cool post! From the architecture, it's not really clear to me why Lambdas have the 15 min limitation. It seems to me AWS could use the same infrastructure to make a product that competes with Google Cloud Run. Maybe it's a businesses thing?

I can't think of any reason outside of product positioning.

A lot of the novelty of Lambda is its identity as a function: small units of execution run on-demand. A Lambda that can run perpetually is made redundant by EC2, and the opinionated time limit informs a lot of design.

Re: Behind the scenes, AWS Lambda

#32

Post by rejectedandsad »

abarrak wrote: post #25
ec109685 wrote: post #18

Earlier quoted context omitted.

This is a good paper that talks about Aurora and provides some insight into how RDS operates: https://www.allthingsdistributed.com/files/p1041-verbitski.p... It’s nice that AWS builds their own higher level abstractions on the same primitives outside developers use. Feels like they eat their own dogfood much more than Google where they bypass GCP and instead utilize underlying Borg primitives for many services.

Nice. I wonder if the stateful merits provided and marketed by containers orchestrates (e.g. K8S) is something they will consider in the future? ..

To build a new service at Amazon, the general path of least resistance these days is to use Lambda. If not Lambda, then ECS. If not ECS (or if it requires bare metal) then EC2.

Re: Behind the scenes, AWS Lambda

#33

Post by rejectedandsad »

ec109685 wrote: post #18
abarrak wrote: post #3

A couple of days ago, I tried to search on how AWS operates RDS behind the scenes, since it is a managed stateful service I was wondering whether it runs in a traditional way VM-based or in a fully containerized environment? .. Unfortunately, a simple search will lead you to the consumer/customer resources out there only.

This is a good paper that talks about Aurora and provides some insight into how RDS operates: https://www.allthingsdistributed.com/files/p1041-verbitski.p... It’s nice that AWS builds their own higher level abstractions on the same primitives outside developers use. Feels like they eat their own dogfood much more than Google where they bypass GCP and instead utilize underlying Borg primitives for many services.

This is a new thing really - it used to be that you'd use a different system that's in many ways better integrated with how the rest of development works but far worse in terms of UX and capacity planning, etc. Now many of the tools are basically frankenstein transformation from the old way into the Amazon-specific way AWS is used via the multi-account pattern.

Re: Behind the scenes, AWS Lambda

#34

Post by aeyes »

navaati wrote: post #16
StratusBen wrote: post #15

Earlier quoted context omitted.

Agreed. AWS RDS instance types are just EC2 instance types prefixed with "db." and you're choosing either single-AZ or multi-AZ deployments so presumably AWS is just spinning up 1 to 3 EC2 instances with some preconfigured software on them.

From what I know there is a secret sauce beyond a mere AMI and a control plane, based on some EBS volumes magic. I may be mixing things up with Aurora though.

There were some comments in the early days that the Multi-AZ magic for classic RDS was just drbd on top of EBS.

Aurora is a completely different approach where the RDBMS code is modified to directly interface with EBS instead of going through a traditional OS filesystem layer.

Re: Behind the scenes, AWS Lambda

#37

Post by bschaatsbergen »

dr_kretyn wrote: post #6

Is this write up correct? How do they know that? I don't see any references on info source except a talk at re:invent.

Both Marc Brooker (lead developer on the AWS Lambda team) giving the talks at Re:Invent as I mentioned in the footnotes, and the official documentation that's out there will provide you with a lot of information.

Re: Behind the scenes, AWS Lambda

#39

Post by simonw »

kolanos wrote: post #30
carlosf wrote: post #27

Earlier quoted context omitted.

Oof Makes sense! I wish Fargate was easier to use and had a scale to 0 feature. If App Runner ends up supporting private deployments then we can have a true Cloud Run competitor.

> I wish Fargate was easier to use and had a scale to 0 feature. Fargate can be scaled to zero. Also, have you tried the CLI? [0] [0]: https://github.com/aws/copilot-cli

When I say "scale to zero" I mean like Cloud Run or AWS Lambda: I define it as the service automatically scaling to zero (and hence costing nothing to run) in between requests, but automatically starting up again when a new request comes in - so the request still gets served, it just suffers from a few seconds of cold-start time.

I'm pretty sure Fargate doesn't offer this. It sounds like you're talking about the ability to manually (or automatically through scripting) turn off your Fargate containers, then manually turn them back on again - but not in a way that an incoming request still gets served even though the container wasn't running when the request first arrived.

Re: Behind the scenes, AWS Lambda

#40

Post by simonw »

This is a great article - I really appreciate when people take the time to assemble details from a bunch of different sources (Firecracker paper, re:Invent talks) and turn them into a useful overview like this.

Clearly Bruno got a lot of the details right, Jeff Barr tweeted a link to this a few weeks ago: https://twitter.com/jeffbarr/status/1404512248152825857

Post reply on HN