Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon Elastic File System – Production-Ready in Three Regions

aws.amazon.com

Post reply on HN
Print view View original

71–80 of 122 posts

Re: Amazon Elastic File System – Production-Ready in Three Regions

#71

Post by rincebrain »

Sanddancer wrote: post #19

This feels rather...primitive for a network filesystem introduced in 2016. Where are the snapshots? Where are the subvolumes? Where's the ability to send/receive volumes/subvolumes to filesystems in other availability zones? There are also I/O limits if you have a smaller filesystem with frequently accessed data, an inability to mount on machines through vpn gateways, and a lot of other seriously rough edges. It's an…

It seems like this is more of a glue layer between a more interesting backend service (EFS) and clients, but I don't know that I'd expect Amazon to do something like write a native client, versus just probably improving the NFS client behavior on Windows/Linux/...

The neat trick would be if NFS 4.X grew support for communicating interesting operations like snapshotting.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#72

Post by deskamess »

Can you expose the EFS file systems to CloudFront? For a future migration (local to cloud) I would like a backend application running on EC2 to use standard path names, but public access would be through CloudFront. It would be easier to migrate existing apps to the cloud with that approach, and then slowly work on changing the apps to use S3 (for cost reasons).

Re: Amazon Elastic File System – Production-Ready in Three Regions

#73

Post by dexterdog »

dunkelheit wrote: post #27
jedberg wrote: post #23

Earlier quoted context omitted.

I'm not sure if you're being sarcastic or not, but no, it isn't. It's to provide building blocks for infrastructure. Down time is still on you. Only a lazy engineer blames their tools for their failures.

"Down time is still on you" - well put. And it is actually great when you can only blame yourself because then you can fix the issue right away. If someone other is to blame, you need to a) convince that person that there is a problem and b) wait for the issue to rise to the top of their queue. Which can be painful.

But in the case of Amazon they will likely know about the problem before you would have anyway. Sure, their problems tend to have a much larger scale, but their ops teams are pretty good.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#74

Post by jabl »

pyre wrote: post #64
zeisss wrote: post #60

Earlier quoted context omitted.

Can't you measure and timeout on the file-descriptor writes too?

Do you want to start wrapping your system calls to read files in code that forks to periodically check for a hung system call?

It's worse that that, actually. With hard mounts (the default), the only way to interrupt is with SIGKILL, which if my memory serves me correctly, is a process-wide signal. So you'd have to do all NFS I/O in a separate child process.

With soft mounts (does EFS support this?) you can exchange SIGKILL for religiously checking return values for all I/O syscalls, retrying partial writes and whatnot.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#75

Post by ceejayoz »

riquito wrote: post #40
oconnore wrote: post #37

Earlier quoted context omitted.

Not sarcastic: aws is not a tool, it's a service. You're exchanging dollars for SLA. The SLA is built in: you're sometimes (rarely) down, and someone else gets called about it.

Your customers will blame you: you don't want it to happen. Even if amazon pay you some damage you may lose trust or your whole business in the meantime.

With a major AWS outage, customers will think "the Internet is having trouble today." Half the sites they visit will be down or broken in some way.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#76

Post by jdub »

KaiserPro wrote: post #25

YES Finally we can start using standard posix semantics for things. Yes people might say its primitive, but currently all I can see is people trying to re-create shared files systems over protocols not designed for it cough cough HTTP Finally I can have a shared home, with a shared environment. A single readonly binary source (great for docker by the way.) also great for ensuring one version of scripts, without havin…

Congratulations, you've won a single point of failure!

Re: Amazon Elastic File System – Production-Ready in Three Regions

#77

Post by sheraz »

I think it is worth mentioning that Azure offer something similar with Samba / CIFS/ SMB protocol. [1]

This is the underlying tech that powered their Docker volume plugin as well [2]

I'm using in production for serving up small images to some web servers, and I'm currently in playground with docker stuff.

So far, so good. I'm impressed.

[1] https://azure.microsoft.com/en-us/documentation/articles/sto...

[2] https://azure.microsoft.com/en-us/blog/persistent-docker-vol...

edit: footnotes

Re: Amazon Elastic File System – Production-Ready in Three Regions

#79

Post by takeda »

jedberg wrote: post #50
Corrado wrote: post #47

Earlier quoted context omitted.

I think that deploying onto a shared volume is exactly the workload that EFS encourages. That things will stop functioning can be said about any (AWS) technology. "Lord help you when (S3|RDS|SQS|etc.) goes down and every (thing) you have is hung up on a broken (thing) connection." The rational argument here is that if uptime is important to you, the solution is to utilize multiple regions, of which EFS is now in 4. Y…

The difference is that all of those services are accessed via an API, which means your client can do clever things when it fails, like timeout, give a fallback, find an equivalent resource in another zone or region, etc. With EFS, it's exposed as NFS, which hooks in much deeper. If it goes down, there isn't anything you can do to work around the problem, unless you start hacking your own file system kernel modules.

Can't you just use soft mounts? Especially since in the scenario you described you can use read only mounts.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#80

Post by Thaxll »

jedberg wrote: post #10

This is really cool but also kind of a shame. I feel like it will encourage bad behavior. I can see people using this to deploy code by deploying onto the shared volume so that all their instances get an "instant update". Which is great when it works, but lord help you when EFS goes down and every app server you have is hung on a broken NFS connection.

Most app / language load everything at startup.
Post reply on HN