Amazon Elastic File System • Hacker News Board

Re: Amazon Elastic File System

#71

Post by b1twise » Thu, Apr 09, 2015, 7:21 PM UTC

joemccall86 wrote: post #19

We're contemplating moving our static image files (JPG/PNG) from an EBS volume to serving them from an S3 bucket (so we can deploy a HA setup). It sounds like it would be a lot less code if we used EFS instead. Would you guys recommend S3 or EFS for this scenario?

S3 is going to be a lot cheaper. It isn't perfect, but it's pretty reliable. Then you can look at cloudfront. And they did recently release cross-region S3 replication. Then you can be really safe and keep a backup copy at a whole other location.

Re: Amazon Elastic File System

#72

Post by jude- » Thu, Apr 09, 2015, 7:23 PM UTC

jtchang wrote: post #36

Does anyone else cringe when someone suggestions using NFS in production? I can't be the only one that has been woken up at 2am because of an NFS outage.

Maybe it only speaks the NFSv4 protocol, but the implementation is different?

Re: Amazon Elastic File System

#73

Post by toomuchtodo » Thu, Apr 09, 2015, 7:23 PM UTC

biot wrote: post #54

All services should eventually be like this. Just as you don't want to deal with provisioning BTUs of air conditioning or watts of power needed for your cloud infrastructure, why should you concern yourself about allocating a certain number of bytes of storage?

> All services should eventually be like this. Just as you don't want to deal with provisioning BTUs of air conditioning or watts of power needed for your cloud infrastructure, why should you concern yourself about allocating a certain number of bytes of storage?

Infrastructure guy here. Abstraction is to reduce workload; you still need to understand the underlying concepts. Otherwise, you're just the guy who freaks out when their DB is at 100% CPU utilization or hours of replica lag without knowing why.

Re: Amazon Elastic File System

#74

Post by click170 » Thu, Apr 09, 2015, 7:23 PM UTC

jeffbarr wrote: post #31

chuckcode wrote: post #16

I wish AWS would be a little more technical in their product descriptions and announcements. If there was ever an audience that wants technical data about how this type of technology will scale and compares to existing technology like glusterfs it would be the AWS users. Instead the performance specifications are ".. and is designed to provide the throughput, IOPS, and low latency needed for a broad range of workload…

We'll be providing a lot more details later.

Fair enough, but one of the biggest reasons I ignore these types of posts is because too often they lack these critical details, they come off as puff pieces to me. IMO these posts would hold value for a larger audience if that info was included in them.

Re: Amazon Elastic File System

#75

Post by jstoiko » Thu, Apr 09, 2015, 7:29 PM UTC

DenisM wrote: post #66

Salient points: - NFS (v4). - Supports petabyte-scale file systems, thousands of concurrent NFS connections. - Automatically grows/shrinks in size. - Multi-zone storage and access. - $0.30 / (Gigabyte * month). - (Not mentioned) Both Linux and Windows have built-in NFS clients.

Thanks for the time-saving description. Osx also has a built-in NFS client.

Re: Amazon Elastic File System

#76

Post by graceofs » Thu, Apr 09, 2015, 7:32 PM UTC

netcraft wrote: post #4

We have been needing this very thing for hosting a large number of assets across multiple ec2 instances. Hope they roll this out quickly.

If you need something like EFS today, there is ObjectiveFS (https://ObjectiveFS.com), a posix cloud file system with S3 backend. Disclaimer: I am a co-founder :-)

Re: Amazon Elastic File System

#77

Post by click170 » Thu, Apr 09, 2015, 7:32 PM UTC

toomuchtodo wrote: post #69

alexchamberlain wrote: post #50

Earlier quoted context omitted.

May I ask why? My experience with NFS is pretty bad performance wise.

Depending on how you tune it, it can be a monster. Several years ago I was managing a cluster with ~5K linux instances all mounted to ~4PB of spinning disk served with NFS. Worked very well.

Can you speak to any stale NFS handle problems?

I've used NFS at home and have had NFS file handle problems but IIRC that was only when there were problems like kernel faults or network partitions.

However several of my colleagues at work have many NFS horror stories and are adamant that NFS does not scale well.

Is NFS stability at scale simply a function of your underlying network and infrastructure stability in your opinion?

Re: Amazon Elastic File System

#78

Post by alexchamberlain » Thu, Apr 09, 2015, 7:33 PM UTC

azinman2 wrote: post #58

alexchamberlain wrote: post #50

Earlier quoted context omitted.

May I ask why? My experience with NFS is pretty bad performance wise.

Ya. I'm wondering how you'd back say a PostgreSQL instance over nfs? It is just different needs that require different solutions.

Is anyone suggesting that? It is widely considered best practice to back your databases by SSDs.

Re: Amazon Elastic File System

#79

Post by rcoder » Thu, Apr 09, 2015, 7:35 PM UTC

tszming wrote: post #40

Congratulation on launching a killer feature once you addicted to it, very hard to move out from AWS :) (but honestly, this is a very awesome technology we've been dreaming for years)

How does this make it harder to move out of AWS?

If anything, their use of NFSv4 means there are plenty of competitive offerings if you decide that performance, security, or physical access constraints dictate migrating off their service.

If you don't want to manage your own Linux/BSD/etc. NFS infrastructure, Oracle, Netapp, and EMC will all happily sell you a storage appliance that supports it. I don't see much lock-in here.

Re: Amazon Elastic File System

#80

Post by notacoward » Thu, Apr 09, 2015, 7:35 PM UTC

evol262 wrote: post #64

acdha wrote: post #11

Earlier quoted context omitted.

At least the Linux NFS client made some big improvements in stability by the mid-2000s. FreeBSD took longer but I've heard they've fixed the kernel deadlocks as well. The other interesting note is that they apparently only support NFSv4, which has some welcome improvements: it uses TCP over a single port, avoids the entire portmap/statd/lockd train-wreck, UTF-8 everywhere, etc. One of the more interesting ones is tha…

It's almost certainly NFSv4 only so they can utilize pNFS

Not necessarily. IIRC, pNFS is an optional feature of NFSv4.1, so they might not have implemented that flavor. If they had, I'm pretty sure they'd be advertising it. They're not shy.