Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon Elastic File System – Production-Ready in Three Regions

aws.amazon.com

Post reply on HN
Print view View original

91–100 of 122 posts

Re: Amazon Elastic File System – Production-Ready in Three Regions

#91

Post by acdha »

merpnderp wrote: post #87
KaiserPro wrote: post #25

YES Finally we can start using standard posix semantics for things. Yes people might say its primitive, but currently all I can see is people trying to re-create shared files systems over protocols not designed for it cough cough HTTP Finally I can have a shared home, with a shared environment. A single readonly binary source (great for docker by the way.) also great for ensuring one version of scripts, without havin…

At 10x the price for S3.

That's not a valid comparison for many applications: if you need POSIX filesystem semantics, it doesn't matter how much S3 costs because it cannot do the job, any more than you'd say S3 costs more than a USB key.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#92

Post by jedberg »

boucher wrote: post #66
jedberg wrote: post #9

Earlier quoted context omitted.

One use case would be one that came up a while back here on HN -- The person had a large binary blob that they needed to load up every time their Lambda function ran. If they could mount EFS from Lambda, they could put the large binary blob there. Depending on how they wrote their code, if the Lambda could seek to the right part of the file then this would work, but if it has to load the entire file into RAM anyway,…

Couldn't you just download the portion of the one file you want from S3? Unless EFS is significantly faster... From a caching perspective, you wouldn't seem to gain much in lamda. You can cache in memory/tmpfs until your container eventually dies, and then you likely aren't on the same machine so there's no EFS level caching to take advantage of.

Right that was my point. You most likely have to load the whole thing into memory, but in the rare case that you can seek directly to the right spot in the file, ECS might have an advantage for you in Lambda.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#93

Post by boucher »

jedberg wrote: post #92
boucher wrote: post #66

Earlier quoted context omitted.

Couldn't you just download the portion of the one file you want from S3? Unless EFS is significantly faster... From a caching perspective, you wouldn't seem to gain much in lamda. You can cache in memory/tmpfs until your container eventually dies, and then you likely aren't on the same machine so there's no EFS level caching to take advantage of.

Right that was my point. You most likely have to load the whole thing into memory, but in the rare case that you can seek directly to the right spot in the file, ECS might have an advantage for you in Lambda.

Yeah, I just meant that S3 has partial downloads, so you can still seek.

EFS/NFS makes a lot of sense I think when you have a lot of random/unpredictable access across large numbers of files. You could do the same thing with FUSE/S3 but across lots of smaller files/accesses I would think the overhead adds up faster.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#94

Post by boucher »

jedberg wrote: post #23
oconnore wrote: post #16

Earlier quoted context omitted.

Isn't the whole point of AWS that you can blame AWS engineers for downtime instead of your own (in this case) code deployment thingy?

I'm not sure if you're being sarcastic or not, but no, it isn't. It's to provide building blocks for infrastructure. Down time is still on you. Only a lazy engineer blames their tools for their failures.

Do you think that's still true in a "serverless" architecture?

Re: Amazon Elastic File System – Production-Ready in Three Regions

#95

Post by nemaar »

jedberg wrote: post #10

This is really cool but also kind of a shame. I feel like it will encourage bad behavior. I can see people using this to deploy code by deploying onto the shared volume so that all their instances get an "instant update". Which is great when it works, but lord help you when EFS goes down and every app server you have is hung on a broken NFS connection.

The instance can fetch the upgrade from the NFS mount and also work using the previous version when NFS is down. I see no problem here.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#96

Post by omni »

willejs wrote: post #83
jdub wrote: post #76

Earlier quoted context omitted.

Congratulations, you've won a single point of failure!

HAHAHA. YES! So much this. THere are legit use cases for this, but there are many use cases that will cause you much pain.

I suspect you're being downvoted because you've described every piece of technology ever created.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#97

Post by cvik »

boucher wrote: post #94
jedberg wrote: post #23

Earlier quoted context omitted.

I'm not sure if you're being sarcastic or not, but no, it isn't. It's to provide building blocks for infrastructure. Down time is still on you. Only a lazy engineer blames their tools for their failures.

Do you think that's still true in a "serverless" architecture?

Surprise! Serverless runs on... gasp servers!

Re: Amazon Elastic File System – Production-Ready in Three Regions

#98

Post by objectivefs »

arqq wrote: post #52
objectivefs wrote: post #17

Cool! If you are looking at shared filesystems on EC2, you might also want to take a look at ObjectiveFS[1]. Works in all regions and on GCS, and you can mount your filesystem securely (end-to-end encryption) between regions. [1]: https://objectivefs.com

ObjectiveFS looks awesome and I've been wanting to use it, but using non-S3 storage is locked to "enterprise, I can't afford it". :( But I also can't afford S3's bandwidth pricing.. I could, however afford to run an S3-API compatible service myself with bandwidth pricing I could afford. But then you wouldn't let me use it because then I'd be "enterprise".

Great that you have been wanting to use ObjectiveFS. We are happy to talk with you about your non-S3 storage use case and see if we can find a plan that works for you.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#99

Post by KaiserPro »

pmalynin wrote: post #86
KaiserPro wrote: post #45

Earlier quoted context omitted.

You are right for small scale things. I'll have to double check my working out, but you're only paying for what you use, not provision, which is one big things. Also, if you have > 3 machines working from the same dataset (ie docker image/video/large binaryblob) there is an instant saving, without factoring in provisioned vs useable space with EBS but, your original point, as a 1-1 replacement of a properly sized EBS…

Nitpick "large binaryblob" is literally "large binary binary large object"

Haha, point taken.
Post reply on HN