Live data from Hacker News

Behind the scenes, AWS Lambda

bschaatsbergen.com

21–30 of 89 posts

Re: Behind the scenes, AWS Lambda

#21
Really cool post!

From the architecture, it's not really clear to me why Lambdas have the 15 min limitation. It seems to me AWS could use the same infrastructure to make a product that competes with Google Cloud Run. Maybe it's a businesses thing?

Re: Behind the scenes, AWS Lambda

#22
post #21

Really cool post! From the architecture, it's not really clear to me why Lambdas have the 15 min limitation. It seems to me AWS could use the same infrastructure to make a product that competes with Google Cloud Run. Maybe it's a businesses thing?

This service exists, it's called AWS Fargate [0].

[0]: https://read.iopipe.com/how-far-out-is-aws-fargate-a2409d2f9...

Re: Behind the scenes, AWS Lambda

#23

Is this write up correct? How do they know that? I don't see any references on info source except a talk at re:invent.

When I was at re:invent 2019 I joined some chalk talks which weren't recorded (or not published). Some of the hosts told lot of details of their internal infrastructure.

Re: Behind the scenes, AWS Lambda

#24
post #16

Earlier quoted context omitted.

Agreed. AWS RDS instance types are just EC2 instance types prefixed with "db." and you're choosing either single-AZ or multi-AZ deployments so presumably AWS is just spinning up 1 to 3 EC2 instances with some preconfigured software on them.

From what I know there is a secret sauce beyond a mere AMI and a control plane, based on some EBS volumes magic. I may be mixing things up with Aurora though.

this is very, very safe to assume. There is probably a hundred engineer's worth of "secret sauce" for an entire managed DB product line.

Re: Behind the scenes, AWS Lambda

#25
post #3

A couple of days ago, I tried to search on how AWS operates RDS behind the scenes, since it is a managed stateful service I was wondering whether it runs in a traditional way VM-based or in a fully containerized environment? .. Unfortunately, a simple search will lead you to the consumer/customer resources out there only.

This is a good paper that talks about Aurora and provides some insight into how RDS operates: https://www.allthingsdistributed.com/files/p1041-verbitski.p... It’s nice that AWS builds their own higher level abstractions on the same primitives outside developers use. Feels like they eat their own dogfood much more than Google where they bypass GCP and instead utilize underlying Borg primitives for many services.

Nice. I wonder if the stateful merits provided and marketed by containers orchestrates (e.g. K8S) is something they will consider in the future? ..

Re: Behind the scenes, AWS Lambda

#26

Earlier quoted context omitted.

We surely picked up some baggage. Much of it vendor specific. But also we jettison some? You stick a lambda into API gateway and you're on the internet. No servers. No linux setup. No apache conf. I'd encourage you to dive in for 20 hours in pure curiosity mode and see what you find.

Lambda is a proprietary solution that only works for people on AWS. Linux is open, and I just need to put it on a box. How do I install the AWS Lambda stack on a standard Linux box?

Openstack?

https://www.openstack.org/

Re: Behind the scenes, AWS Lambda

#27
post #22
post #21

Really cool post! From the architecture, it's not really clear to me why Lambdas have the 15 min limitation. It seems to me AWS could use the same infrastructure to make a product that competes with Google Cloud Run. Maybe it's a businesses thing?

This service exists, it's called AWS Fargate [0]. [0]: https://read.iopipe.com/how-far-out-is-aws-fargate-a2409d2f9...

Oof

Makes sense!

I wish Fargate was easier to use and had a scale to 0 feature.

If App Runner ends up supporting private deployments then we can have a true Cloud Run competitor.

Re: Behind the scenes, AWS Lambda

#28
post #3

A couple of days ago, I tried to search on how AWS operates RDS behind the scenes, since it is a managed stateful service I was wondering whether it runs in a traditional way VM-based or in a fully containerized environment? .. Unfortunately, a simple search will lead you to the consumer/customer resources out there only.

This is a good paper that talks about Aurora and provides some insight into how RDS operates: https://www.allthingsdistributed.com/files/p1041-verbitski.p... It’s nice that AWS builds their own higher level abstractions on the same primitives outside developers use. Feels like they eat their own dogfood much more than Google where they bypass GCP and instead utilize underlying Borg primitives for many services.

Which gcp services run directly on borg? My understanding is at least bigtable, cloud sql and other dbs are within “hidden” VMs. I think loadbalancers and storage are exceptions but same is true for aws (except the classic elb probably)

Re: Behind the scenes, AWS Lambda

#29
post #22
post #21

Really cool post! From the architecture, it's not really clear to me why Lambdas have the 15 min limitation. It seems to me AWS could use the same infrastructure to make a product that competes with Google Cloud Run. Maybe it's a businesses thing?

This service exists, it's called AWS Fargate [0]. [0]: https://read.iopipe.com/how-far-out-is-aws-fargate-a2409d2f9...

Fargate isn't a competitor to Cloud Run (I wish it was) because it doesn't scale to zero in between requests and scale back up again when new traffic arrives.

Re: Behind the scenes, AWS Lambda

#30
post #27
post #22

Earlier quoted context omitted.

This service exists, it's called AWS Fargate [0]. [0]: https://read.iopipe.com/how-far-out-is-aws-fargate-a2409d2f9...

Oof Makes sense! I wish Fargate was easier to use and had a scale to 0 feature. If App Runner ends up supporting private deployments then we can have a true Cloud Run competitor.

> I wish Fargate was easier to use and had a scale to 0 feature.

Fargate can be scaled to zero. Also, have you tried the CLI? [0]

[0]: https://github.com/aws/copilot-cli

Post reply on HN