I've found EFS enticing in theory but painfully slow and riddled with issues in practice. In the past I've tried it thinking "it's basically an EBS volume I can mount on > 1 EC2 instance," only to find terrible read performance and misc. low-level NFS errors.
Also insanely expensive. Do you know how much it costs to write 3 MB/second to EFS? How much does it cost to read 26 MB/second from the same EFS volume? That's $1,400/month.
A shared file system for lambda functions
41–50 of 113 posts
Re: A shared file system for lambda functions
#42I tried Lambda for a use-case that I had in 2018:
We published Polls and Predictions to people watching the 2018 World Cup. We set the vote callback URL to a function on AWS Lambda.
It failed spectacularly during our load-testing because the ramp-up period was far too slow. We needed to go from 0 to 100,000 incoming requests/second in about 20 seconds.
We had to switch to an Nginx/Lua/Redis endpoint because Lambda was just completely unusable. It would have cost us $27,000/month to pre-provision 10,000 concurrent executions...
What is it that people actually use Lambda for?
Re: A shared file system for lambda functions
#43Cool, so we're staring to curve more sharply around the full circle we'll eventually go on. So now lambda functions can mount persistent block storage. Next up: allow your lamda functions to run for longer Then: allow multiple lamda functions to execute concurrently, and indefinitely, as a group, while having block storage mounted And finally: use your EC2 instances as lambda functions
Re: A shared file system for lambda functions
#44I'm curious what people actually use Lambda for? I tried Lambda for a use-case that I had in 2018: We published Polls and Predictions to people watching the 2018 World Cup. We set the vote callback URL to a function on AWS Lambda. It failed spectacularly during our load-testing because the ramp-up period was far too slow. We needed to go from 0 to 100,000 incoming requests/second in about 20 seconds. We had to switch…
Re: A shared file system for lambda functions
#45I've found EFS enticing in theory but painfully slow and riddled with issues in practice. In the past I've tried it thinking "it's basically an EBS volume I can mount on > 1 EC2 instance," only to find terrible read performance and misc. low-level NFS errors.
Re: A shared file system for lambda functions
#46I'm curious what people actually use Lambda for? I tried Lambda for a use-case that I had in 2018: We published Polls and Predictions to people watching the 2018 World Cup. We set the vote callback URL to a function on AWS Lambda. It failed spectacularly during our load-testing because the ramp-up period was far too slow. We needed to go from 0 to 100,000 incoming requests/second in about 20 seconds. We had to switch…
We use it as a cron substitute quite often, which I guess isn’t the serverless use case, but it’s quite handy
Re: A shared file system for lambda functions
#47Earlier quoted context omitted.
We use it as a cron substitute quite often, which I guess isn’t the serverless use case, but it’s quite handy
Cron is a scheduler that executes shell commands. Lambda represents the shell command itself right? What are you using to specify when something should run?
https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/S...
Re: A shared file system for lambda functions
#48Earlier quoted context omitted.
Cron is a scheduler that executes shell commands. Lambda represents the shell command itself right? What are you using to specify when something should run?
Lambda can trigger off a bunch of AWS events. One of those events types are CloudWatch events, mostly used for triggering in response to things like CPU activity or other metrics. One of the CloudWatch events you can specify are plain old Cron events: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/S...
Re: A shared file system for lambda functions
#49I'm curious what people actually use Lambda for? I tried Lambda for a use-case that I had in 2018: We published Polls and Predictions to people watching the 2018 World Cup. We set the vote callback URL to a function on AWS Lambda. It failed spectacularly during our load-testing because the ramp-up period was far too slow. We needed to go from 0 to 100,000 incoming requests/second in about 20 seconds. We had to switch…
1) AWS administration-related actions (required for non-trivial mgmt.)
2) end-user actions (optional).
AWS throttles and limits everything, so expecting 100,000 requests in 20 seconds on a default AWS account is unreasonable.
The Cloud doesn't mean unlimited capacity.
Also, I haven't seen anybody version control lambda code, even in compliance environments, so something to investigate.
Re: A shared file system for lambda functions
#50I'm curious what people actually use Lambda for? I tried Lambda for a use-case that I had in 2018: We published Polls and Predictions to people watching the 2018 World Cup. We set the vote callback URL to a function on AWS Lambda. It failed spectacularly during our load-testing because the ramp-up period was far too slow. We needed to go from 0 to 100,000 incoming requests/second in about 20 seconds. We had to switch…
AWS lambda is used in 2 categories: 1) AWS administration-related actions (required for non-trivial mgmt.) 2) end-user actions (optional). AWS throttles and limits everything, so expecting 100,000 requests in 20 seconds on a default AWS account is unreasonable. The Cloud doesn't mean unlimited capacity. Also, I haven't seen anybody version control lambda code, even in compliance environments, so something to investig…
Edit: my AWS support rep explained it: Without pre-provisioned capacity, it would take 34 minutes to scale our Lambda functions to the peak concurrency we needed. And it would cost several thousand dollars.
Completely insane cost. Also, EFS is ridiculously expensive.