Live data from Hacker News

Amazon Elastic File System – Production-Ready in Three Regions

aws.amazon.com

81–90 of 122 posts

Re: Amazon Elastic File System – Production-Ready in Three Regions

#81
post #23

Earlier quoted context omitted.

I'm not sure if you're being sarcastic or not, but no, it isn't. It's to provide building blocks for infrastructure. Down time is still on you. Only a lazy engineer blames their tools for their failures.

> Down time is still on you If you can say "It's not our fault, it's Amazon's", there are plenty of boards and customers who are fine with that, in my experience.

Yep. "Nobody ever got fired for buying IBM" applies here.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#82
post #22
post #14

Earlier quoted context omitted.

If files are immutable, isn't s3 much better fit for it? s3 main weaknesses is that you can't modify part of a file, which makes it unsuitable for a filesystem, but looks like it satisfies your use case.

Maybe but unfortunately the database doesn't support "backup store" that we can use directly S3, it just writes the data to specified directory in a filesystem. We should either fork the database or implement "S3 backed" filesystem and we picked the latter one. Unfortunately the performance FUSE is not that good for read and writing data to S3 is somehow expensive.

Why not have a service watch for filesystem changes and then upload to S3?

Re: Amazon Elastic File System – Production-Ready in Three Regions

#83
post #76

YES Finally we can start using standard posix semantics for things. Yes people might say its primitive, but currently all I can see is people trying to re-create shared files systems over protocols not designed for it cough cough HTTP Finally I can have a shared home, with a shared environment. A single readonly binary source (great for docker by the way.) also great for ensuring one version of scripts, without havin…

Congratulations, you've won a single point of failure!

HAHAHA. YES! So much this. THere are legit use cases for this, but there are many use cases that will cause you much pain.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#84
post #76

YES Finally we can start using standard posix semantics for things. Yes people might say its primitive, but currently all I can see is people trying to re-create shared files systems over protocols not designed for it cough cough HTTP Finally I can have a shared home, with a shared environment. A single readonly binary source (great for docker by the way.) also great for ensuring one version of scripts, without havin…

Congratulations, you've won a single point of failure!

It's a single point of failure that's highly available across multiple availability zones. I'm not sure I would even call it a single point of failure when it's running in multiple datacenters. If you need geo-redundancy, rsync it to another region nightly.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#85
post #6

Can EFS be used as a volume for ECS tasks?

If you use the Convoy NFS volume driver with EFS, your containers can use any NFS server (including EFS) as highly available persistent volume storage: https://github.com/rancher/convoy

Re: Amazon Elastic File System – Production-Ready in Three Regions

#86

Price is a bit high $0.3 GB / month vs. $0.1 GB / month for SSD EBS and even less for local storage. S3 is even cheaper $0.03 GB / month. So EFS is 10 times more expensive than S3. On I/O front also EBS (gp2) and (st1) are several times cheaper. So most of EFS advantages are in flexibility and sharing same file system across many instances. This may be great e.g. for sharing readonly binaries, but large scale data pi…

You are right for small scale things. I'll have to double check my working out, but you're only paying for what you use, not provision, which is one big things. Also, if you have > 3 machines working from the same dataset (ie docker image/video/large binaryblob) there is an instant saving, without factoring in provisioned vs useable space with EBS but, your original point, as a 1-1 replacement of a properly sized EBS…

Nitpick

"large binaryblob" is literally "large binary binary large object"

Re: Amazon Elastic File System – Production-Ready in Three Regions

#87

YES Finally we can start using standard posix semantics for things. Yes people might say its primitive, but currently all I can see is people trying to re-create shared files systems over protocols not designed for it cough cough HTTP Finally I can have a shared home, with a shared environment. A single readonly binary source (great for docker by the way.) also great for ensuring one version of scripts, without havin…

At 10x the price for S3.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#89
post #23

Earlier quoted context omitted.

I'm not sure if you're being sarcastic or not, but no, it isn't. It's to provide building blocks for infrastructure. Down time is still on you. Only a lazy engineer blames their tools for their failures.

"Down time is still on you" - well put. And it is actually great when you can only blame yourself because then you can fix the issue right away. If someone other is to blame, you need to a) convince that person that there is a problem and b) wait for the issue to rise to the top of their queue. Which can be painful.

Just because it's your fault, that doesn't mean that you can quickly and easily fix the problem yourself. Amazon has a team of people to keep EFS up and running, and AWS owns the entire storage, server, and network stack. When your own NFS server starts timing out, you may have only a few people to diagnose the problem.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#90
post #40

Earlier quoted context omitted.

Not sarcastic: aws is not a tool, it's a service. You're exchanging dollars for SLA. The SLA is built in: you're sometimes (rarely) down, and someone else gets called about it.

Your customers will blame you: you don't want it to happen. Even if amazon pay you some damage you may lose trust or your whole business in the meantime.

I think this perspective really depends on who you are supporting. This is my experience:

1) Actual customers: They want you to fix it. They don't care who's fault it is, they are dependent on you.

2) Internal customers: They want someone to blame. They don't really care that it's down, they just want to point the finger at someone when their report goes out late.

Post reply on HN