Live data from Hacker News

A shared file system for lambda functions

aws.amazon.com

41–50 of 113 posts

Re: A shared file system for lambda functions

#41

I've found EFS enticing in theory but painfully slow and riddled with issues in practice. In the past I've tried it thinking "it's basically an EBS volume I can mount on > 1 EC2 instance," only to find terrible read performance and misc. low-level NFS errors.

Also insanely expensive. Do you know how much it costs to write 3 MB/second to EFS? How much does it cost to read 26 MB/second from the same EFS volume? That's $1,400/month.

[deleted]

Re: A shared file system for lambda functions

#42
I'm curious what people actually use Lambda for?

I tried Lambda for a use-case that I had in 2018:

We published Polls and Predictions to people watching the 2018 World Cup. We set the vote callback URL to a function on AWS Lambda.

It failed spectacularly during our load-testing because the ramp-up period was far too slow. We needed to go from 0 to 100,000 incoming requests/second in about 20 seconds.

We had to switch to an Nginx/Lua/Redis endpoint because Lambda was just completely unusable. It would have cost us $27,000/month to pre-provision 10,000 concurrent executions...

What is it that people actually use Lambda for?

Re: A shared file system for lambda functions

#43

Cool, so we're staring to curve more sharply around the full circle we'll eventually go on. So now lambda functions can mount persistent block storage. Next up: allow your lamda functions to run for longer Then: allow multiple lamda functions to execute concurrently, and indefinitely, as a group, while having block storage mounted And finally: use your EC2 instances as lambda functions

cron+perl on a shared hosting is a lambda from 90s, cheap, fast, and it just works

Re: A shared file system for lambda functions

#44

I'm curious what people actually use Lambda for? I tried Lambda for a use-case that I had in 2018: We published Polls and Predictions to people watching the 2018 World Cup. We set the vote callback URL to a function on AWS Lambda. It failed spectacularly during our load-testing because the ramp-up period was far too slow. We needed to go from 0 to 100,000 incoming requests/second in about 20 seconds. We had to switch…

We use it as a cron substitute quite often, which I guess isn’t the serverless use case, but it’s quite handy

Re: A shared file system for lambda functions

#45

I've found EFS enticing in theory but painfully slow and riddled with issues in practice. In the past I've tried it thinking "it's basically an EBS volume I can mount on > 1 EC2 instance," only to find terrible read performance and misc. low-level NFS errors.

I found it adequate to work around limitations of another system I was using. We were using a container coordinator thing (Convox) that couldn't attach EBS volumes, nor could it limit concurrent access to exactly one replica. So I used EFS which worked OK. I kept an eye on how we were using the burst credits, and picked a filesystem size that gave us enough IOPS. All in all, it worked fine (but I was perfectly happy to move off of it to EBS, of course).

Re: A shared file system for lambda functions

#46
post #44

I'm curious what people actually use Lambda for? I tried Lambda for a use-case that I had in 2018: We published Polls and Predictions to people watching the 2018 World Cup. We set the vote callback URL to a function on AWS Lambda. It failed spectacularly during our load-testing because the ramp-up period was far too slow. We needed to go from 0 to 100,000 incoming requests/second in about 20 seconds. We had to switch…

We use it as a cron substitute quite often, which I guess isn’t the serverless use case, but it’s quite handy

Cron is a scheduler that executes shell commands. Lambda represents the shell command itself right? What are you using to specify when something should run?

Re: A shared file system for lambda functions

#47
post #46
post #44

Earlier quoted context omitted.

We use it as a cron substitute quite often, which I guess isn’t the serverless use case, but it’s quite handy

Cron is a scheduler that executes shell commands. Lambda represents the shell command itself right? What are you using to specify when something should run?

Lambda can trigger off a bunch of AWS events. One of those events types are CloudWatch events, mostly used for triggering in response to things like CPU activity or other metrics. One of the CloudWatch events you can specify are plain old Cron events:

https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/S...

Re: A shared file system for lambda functions

#48
post #46

Earlier quoted context omitted.

Cron is a scheduler that executes shell commands. Lambda represents the shell command itself right? What are you using to specify when something should run?

Lambda can trigger off a bunch of AWS events. One of those events types are CloudWatch events, mostly used for triggering in response to things like CPU activity or other metrics. One of the CloudWatch events you can specify are plain old Cron events: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/S...

[deleted]

Re: A shared file system for lambda functions

#49

I'm curious what people actually use Lambda for? I tried Lambda for a use-case that I had in 2018: We published Polls and Predictions to people watching the 2018 World Cup. We set the vote callback URL to a function on AWS Lambda. It failed spectacularly during our load-testing because the ramp-up period was far too slow. We needed to go from 0 to 100,000 incoming requests/second in about 20 seconds. We had to switch…

AWS lambda is used in 2 categories:

1) AWS administration-related actions (required for non-trivial mgmt.)

2) end-user actions (optional).

AWS throttles and limits everything, so expecting 100,000 requests in 20 seconds on a default AWS account is unreasonable.

The Cloud doesn't mean unlimited capacity.

Also, I haven't seen anybody version control lambda code, even in compliance environments, so something to investigate.

Re: A shared file system for lambda functions

#50

I'm curious what people actually use Lambda for? I tried Lambda for a use-case that I had in 2018: We published Polls and Predictions to people watching the 2018 World Cup. We set the vote callback URL to a function on AWS Lambda. It failed spectacularly during our load-testing because the ramp-up period was far too slow. We needed to go from 0 to 100,000 incoming requests/second in about 20 seconds. We had to switch…

AWS lambda is used in 2 categories: 1) AWS administration-related actions (required for non-trivial mgmt.) 2) end-user actions (optional). AWS throttles and limits everything, so expecting 100,000 requests in 20 seconds on a default AWS account is unreasonable. The Cloud doesn't mean unlimited capacity. Also, I haven't seen anybody version control lambda code, even in compliance environments, so something to investig…

I guess that I was expecting AWS Lambda to scale up immediately in the same way the ALB or SQS scales up with demand. That assumption was very wrong.

Edit: my AWS support rep explained it: Without pre-provisioned capacity, it would take 34 minutes to scale our Lambda functions to the peak concurrency we needed. And it would cost several thousand dollars.

Completely insane cost. Also, EFS is ridiculously expensive.

Post reply on HN