Everyone complains about the 15 minutes problem on lambda, but am I the only that have a problem with the 250MB deployment size limit?
What would make AWS even better
11–20 of 70 posts
Re: What would make AWS even better
#12Lambda is insanely expensive is why they don’t allow long-running jobs. A 1GB allocation is $43/mo. And most lambda users are running 1 single task/process per lambda invocation.
Wait, so AWS does not allow long running lambdas because that would make too much money for them?
The actual reason is that long running invocations run synchronous workflows, typically requiring holding threads and sockets open for the entire duration of execution.
Lambda is a complex system, and holding those sockets for long times across many services could cause resource starvation issues. You've got load balancers, data plane, control plane, tenant vms, and a whole bunch of caches, and support services that all need to be ready to roll over the lifetime of the invocation.
And you have to consider the use case for draining and patching lambda pools. If someone is running a two hour function and you need to take down any server that's currently holding a thread or socket for it, you need to wait for the function to complete. You can't start a new load, so you are really inefficiently using resources until the function completes.
Re: What would make AWS even better
#13Everyone complains about the 15 minutes problem on lambda, but am I the only that have a problem with the 250MB deployment size limit?
What are you publishing that has a binary that big but can't use a container image?
Re: What would make AWS even better
#14Long running lambda functions are called AWS Batch. It’s a relatively unknown service but pretty decent if you need something like a GPU or long running jobs and can tolerate a 90 second cold start.
Re: What would make AWS even better
#15What about, "treat AWS workers better"? Pay your people for their on call hours! Let them work on side projects and games in their spare time! Give them more than seven paid holidays. Give them more than two weeks vacation!
Only six weeks of paid parental leave?
I would absolutely be willing to pay more for AWS if I knew that amount was going to treating the poor folks who built it all better.
Re: What would make AWS even better
#16I am pretty surprised they don't compete with Stripe. They have some Amazon pay thing I'd never use, but competing with Stripe seems obvious.
Same with Twilio. They do kind of compete with them, but not really.
Their managed airflow is insanely basically unusably expensive, I don't get that.
Re: What would make AWS even better
#17Long running lambdas would be sick for infrequent / low concurrency data pipelines. My wet dream is "bidirectional IaC". Let me make changes using the GUI, commit to repo automatically.
In most cases, I think you can do fine with just having a proper async interface to the data pipeline, where you trigger something, and then you respond to the trigger that it has completed.
Re: What would make AWS even better
#18Long running lambda functions are called AWS Batch. It’s a relatively unknown service but pretty decent if you need something like a GPU or long running jobs and can tolerate a 90 second cold start.
...and there's also AWS DataPipeline.
Re: What would make AWS even better
#19Everyone complains about the 15 minutes problem on lambda, but am I the only that have a problem with the 250MB deployment size limit?
Sure you can get around that with proper use of Layers.
Re: What would make AWS even better
#20Yes, it can be improved
It feels things like the S3 API are design by committee. If you use tools like the cli you'll notice how clunky it is