A shared file system for lambda functions
aws.amazon.com
A shared file system for lambda functions
1–10 of 113 posts
Re: A shared file system for lambda functions
#2Pretty sweet.
Is anybody using Lambda to run huge MapReduce jobs? Do people still use Hadoop?
Doesn't this basically just let you have something like HDFS for running large distributed computations with some shared state, without having to reach for S3 or redis?
Re: A shared file system for lambda functions
#3Can we please change the title? I was really afraid that AWS might have lost it and called the new file system "new", which must be the least practical name for anything from an SEO standpoint.
Re: A shared file system for lambda functions
#4I've found EFS enticing in theory but painfully slow and riddled with issues in practice. In the past I've tried it thinking "it's basically an EBS volume I can mount on > 1 EC2 instance," only to find terrible read performance and misc. low-level NFS errors.
Re: A shared file system for lambda functions
#5Can we please change the title? I was really afraid that AWS might have lost it and called the new file system "new", which must be the least practical name for anything from an SEO standpoint.
Perhaps:
> AWS Lambda functions can now mount an Elastic File System
from the opening paragraph.
Re: A shared file system for lambda functions
#6Can we please change the title? I was really afraid that AWS might have lost it and called the new file system "new", which must be the least practical name for anything from an SEO standpoint.
I second this motion. I was going to make the title more sensible when I submitted it, but given this is discouraged and often rolled back nowadays (as I often see on https://hackernewstitles.netlify.app/), I decided I'd let it happen in due course.
Re: A shared file system for lambda functions
#7Can we please change the title? I was really afraid that AWS might have lost it and called the new file system "new", which must be the least practical name for anything from an SEO standpoint.
With how Amazon's naming spree for AWS products goes, it's not that far off.
Re: A shared file system for lambda functions
#8I've found EFS enticing in theory but painfully slow and riddled with issues in practice. In the past I've tried it thinking "it's basically an EBS volume I can mount on > 1 EC2 instance," only to find terrible read performance and misc. low-level NFS errors.
Dunno your exact requirements or when you last tried it, but they did boost EFS's read speed (they claim by 400% [1]) as of this April, so it might be worth looking into again if you're still trying to find a solution.
1: https://aws.amazon.com/about-aws/whats-new/2020/04/amazon-el...
Re: A shared file system for lambda functions
#9Pretty sweet. Is anybody using Lambda to run huge MapReduce jobs? Do people still use Hadoop? Doesn't this basically just let you have something like HDFS for running large distributed computations with some shared state, without having to reach for S3 or redis?
I know at least one team at Amazon that does runs map-reduce style jobs with Lambda, but now that Athena supports user-defined functions, I'd personally be inclined to use it instead over EFS or S3 + Lambda.
Re: A shared file system for lambda functions
#10Can we please change the title? I was really afraid that AWS might have lost it and called the new file system "new", which must be the least practical name for anything from an SEO standpoint.
Amazon New will be the worst name for a AWS product, since they announce TONS of products every year, nothing is really new.