Amazon Elastic File System – Production-Ready in Three Regions • Hacker News Board

Re: Amazon Elastic File System – Production-Ready in Three Regions

#11

Post by Cidan » Wed, Jun 29, 2016, 6:33 AM UTC

We've been in the EFS preview for quite a while now, and we are incredibly pleased with it. We do a few different things using EFS, including using it as an output for some of our Spark jobs.

It's really a top notch product; couldn't be happier.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#12

Post by kldavenport » Wed, Jun 29, 2016, 6:38 AM UTC

Pricing looks competitive https://aws.amazon.com/efs/pricing/ I wonder if this helps those who perceive S3 as vendor lock-in since it replicates a traditional file system.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#13

Post by fulafel » Wed, Jun 29, 2016, 6:39 AM UTC

Any ideas what they are using for NFS servers software?

Re: Amazon Elastic File System – Production-Ready in Three Regions

#14

Post by takeda » Wed, Jun 29, 2016, 6:55 AM UTC

buremba wrote: post #3

We use a columnar database and we insert data to it periodically (30s to 90s) using Kinesis as middleware. Files are immutable and each insert creates [count of columns] * [count of tables] files. The database also periodically merges the batch files in background. Since EFS was not available, I was trying to develop a FUSE filesystem that writes data to both S3 and local filesystem (EBS) for durability and read only…

If files are immutable, isn't s3 much better fit for it? s3 main weaknesses is that you can't modify part of a file, which makes it unsuitable for a filesystem, but looks like it satisfies your use case.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#15

Post by boraturan » Wed, Jun 29, 2016, 6:56 AM UTC

Seems like no Windows support?

Re: Amazon Elastic File System – Production-Ready in Three Regions

#16

Post by oconnore » Wed, Jun 29, 2016, 7:10 AM UTC

jedberg wrote: post #10

This is really cool but also kind of a shame. I feel like it will encourage bad behavior. I can see people using this to deploy code by deploying onto the shared volume so that all their instances get an "instant update". Which is great when it works, but lord help you when EFS goes down and every app server you have is hung on a broken NFS connection.

Isn't the whole point of AWS that you can blame AWS engineers for downtime instead of your own (in this case) code deployment thingy?

Re: Amazon Elastic File System – Production-Ready in Three Regions

#17

Post by objectivefs » Wed, Jun 29, 2016, 7:10 AM UTC

Cool! If you are looking at shared filesystems on EC2, you might also want to take a look at ObjectiveFS[1]. Works in all regions and on GCS, and you can mount your filesystem securely (end-to-end encryption) between regions.

[1]: https://objectivefs.com

Re: Amazon Elastic File System – Production-Ready in Three Regions

#18

Post by merb » Wed, Jun 29, 2016, 7:16 AM UTC

I wonder if some can run a postgresql database on top of that. This would make it relativly easy to have a master without downtime and be way cheaper than RDS.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#19

Post by Sanddancer » Wed, Jun 29, 2016, 7:19 AM UTC

This feels rather...primitive for a network filesystem introduced in 2016. Where are the snapshots? Where are the subvolumes? Where's the ability to send/receive volumes/subvolumes to filesystems in other availability zones? There are also I/O limits if you have a smaller filesystem with frequently accessed data, an inability to mount on machines through vpn gateways, and a lot of other seriously rough edges.

It's an interesting start to a project, but the public features feel very much lacking.

Re: Amazon Elastic File System – Production-Ready in Three Regions

#20

Post by madbiz » Wed, Jun 29, 2016, 7:23 AM UTC

bbgm wrote: post #7

madbiz wrote: post #6

Can EFS be used as a volume for ECS tasks?

Here's a post we wrote earlier in the year about EFS and ECS: https://aws.amazon.com/blogs/compute/using-amazon-efs-to-per...

In the blog post the EFS is mounted by the EC2 instances from the start but i expected to be dynamic depending on the scheduled tasks on the instance. I dont want to couple EFS volumes which might be used by the containers if they might get scheduled on an instance. I would like to have a general purpose cluster and let the scheduler instrument the EC2 instance to mount an EFS volume if its needed by a scheduled task dynamically.