Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon Elastic File System

aws.amazon.com

Post reply on HN
Print view View original

151–160 of 217 posts

Re: Amazon Elastic File System

#151

Post by ploxiln »

eknkc wrote: post #127
matheweis wrote: post #113

Earlier quoted context omitted.

Not sure why NFS gets such a bad rap. On a low-latency network, properly tuned NFS has very few, if any performance issues. I've personally seen read/write rates exceeding 800MByte/sec on more or less white-box hardware, at which point it was limited by the underlying storage infrastructure (8Gbit fiber), not the NFS protocols. Dell has a 2013 white paper (I'm not affiliated with them, fwiw) about their fairly white…

Is it really that good? My only test case has been a VMWare virtual machine, mounting an nfs share from the host so I could work on my local filesystem and execute within the VM. Switched to a filesystem wather + rsync combo after struggling with poor random read performance. Maybe it was due to bad configuration but always thought it would be a poor choice for anything serius.

That might be an issue with scheduling of the virtual kernel and the host kernel.

I've found nfs to be much faster and more reliable than sshfs or smbfs for VMs, using either qemu-kvm on linux or virtualbox on OS X.

Re: Amazon Elastic File System

#152

Post by tracker1 »

616c wrote: post #132
DenisM wrote: post #66

Salient points: - NFS (v4). - Supports petabyte-scale file systems, thousands of concurrent NFS connections. - Automatically grows/shrinks in size. - Multi-zone storage and access. - $0.30 / (Gigabyte * month). - (Not mentioned) Both Linux and Windows have built-in NFS clients.

> - (Not mentioned) Both Linux and Windows have built-in NFS clients. Does anyone use the NFS for Windows? Is it reliable enough I can move servers away from CIFS alogether if I want to access a Linux box from Windows. For instance if I click the folder's Properties in Windows, will I see the proper Unix perms and metadata and be able to edit them?

FYI: In windows (desktop versions) it's only now in the "Enterprise" version, and apparently will be removed in Windows 10. Not sure if there are limitations to which server versions are currently supported.

CIFS/SAMBA in Linux has been pretty good, imho for the past couple years... though I still get some occasional wonky behavior from my NAS box.

Re: Amazon Elastic File System

#153

Post by hariramshankar »

jeffbarr wrote: post #31
chuckcode wrote: post #16

I wish AWS would be a little more technical in their product descriptions and announcements. If there was ever an audience that wants technical data about how this type of technology will scale and compares to existing technology like glusterfs it would be the AWS users. Instead the performance specifications are ".. and is designed to provide the throughput, IOPS, and low latency needed for a broad range of workload…

We'll be providing a lot more details later.

Yea and when is that? If a product is announced, even if its a preview version, shouldn't the details be sorted out by then? This is like a 10000ft view of a product, a great product at that. But numbers to back your claims of scalability and performance would have been best.

Re: Amazon Elastic File System

#154

Post by jordanthoms »

andrew311 wrote: post #138

EFS is a great addition to AWS. We have SAN as a service via EBS, now we get NFS as a service. Great. The question (for me) now becomes "where do we go from here?" Infinite NFS is great, but what I've always wanted is infinite EBS that is fully integrated from file system to SAN. In other words, something that behaves like a local file system (without the gotchas of NFS like a lack of delete on close), but I don't ha…

That would be awesome. A huge engineering effort - I imagine that would require building a radically different filesystem more or less from scratch - but AWS has the resources to do that sort of thing.

Re: Amazon Elastic File System

#155

Post by Nexxxeh »

rcoder wrote: post #79
tszming wrote: post #40

Congratulation on launching a killer feature once you addicted to it, very hard to move out from AWS :) (but honestly, this is a very awesome technology we've been dreaming for years)

How does this make it harder to move out of AWS? If anything, their use of NFSv4 means there are plenty of competitive offerings if you decide that performance, security, or physical access constraints dictate migrating off their service. If you don't want to manage your own Linux/BSD/etc. NFS infrastructure, Oracle, Netapp, and EMC will all happily sell you a storage appliance that supports it. I don't see much lock…

I think parent meant it in a positive way, as opposed to negative. "So good you don't want to leave" as opposed to proprietary tech results in vendor lock-in.

Re: Amazon Elastic File System

#156

Post by jonahx »

chuckcode wrote: post #16

I wish AWS would be a little more technical in their product descriptions and announcements. If there was ever an audience that wants technical data about how this type of technology will scale and compares to existing technology like glusterfs it would be the AWS users. Instead the performance specifications are ".. and is designed to provide the throughput, IOPS, and low latency needed for a broad range of workload…

As a developer (but not a sysadmin) I feel the opposite. Not saying your criticism is wrong, just giving another view from a AWS newbie perspective.

When I read most AWS product descriptions, I cannot understand what real world situations they are for. I either could not say, or could only vaguely say, what the product is in my own words. There seems to be a great scaffolding of assumed knowledge about the AWS system and distributed computing. Maybe that's intended, but it's intimidating and keeps me in simpler places like Heroku land.

Re: Amazon Elastic File System

#157

Post by jsmeaton »

throwaway1979 wrote: post #56
wildlogic wrote: post #44

Earlier quoted context omitted.

Although skeptical about it going into the project, I've deployed a virtualized oracle rac environment over 10G NFS and with some tuning it was stable and performant. If it is good enough for rac, which has some of the most stringent latency / performance requirements that I've seen, I'd say that it is probably good enough for quite a few production use cases, although to be fair this was only an 8 node cluster.

Some naive questions if you don't mind... (I'm really curious about RAC .. my only DB experience has been small MySQL and MS SQL Server clusters). 1) I thought the really high end DBs like to manage their own block storage. Your NFS comment suggests that the database data files were running on an NFS mount, and you had a 10 gig Ethernet connection to the file server. 2) What would you say is the average size of a RAC…

Just to offer a little bit of information, we're currently running a 2 node RAC cluster. I'm not entirely sure about the storage mechanism though.

Re: Amazon Elastic File System

#158

Post by Gigablah »

rjurney wrote: post #129
vially wrote: post #94

Earlier quoted context omitted.

Would you mind explaining how exactly "Apple consistently leads and innovates ahead of other companies and open source"?

Are you kidding? Apple reinvents every space they enter with profound invention. Do you remember computers before the Mac? Notebooks before the Macbook? Music players before the iPod? Phones before the iPhone? Tablets before the iPad? And in a month you'll think: watches before the Apple Watch?

> watches before the Apple Watch

Yes, I remember the iPod nano

Re: Amazon Elastic File System

#159

Post by hrez »

e40 wrote: post #93
DenisM wrote: post #66

Salient points: - NFS (v4). - Supports petabyte-scale file systems, thousands of concurrent NFS connections. - Automatically grows/shrinks in size. - Multi-zone storage and access. - $0.30 / (Gigabyte * month). - (Not mentioned) Both Linux and Windows have built-in NFS clients.

For comparison, EBS is $0.10 / GB / Month for SSDs and $0.05 / GB / Month for regular disks.

For comparison, keep in mind that EBS is $0.10/Gb per provisioned storage, which is often over-provisioned. Versus EFS $0.30/Gb per utilized storage.

Re: Amazon Elastic File System

#160

Post by seanwilson »

jonahx wrote: post #156
chuckcode wrote: post #16

I wish AWS would be a little more technical in their product descriptions and announcements. If there was ever an audience that wants technical data about how this type of technology will scale and compares to existing technology like glusterfs it would be the AWS users. Instead the performance specifications are ".. and is designed to provide the throughput, IOPS, and low latency needed for a broad range of workload…

As a developer (but not a sysadmin) I feel the opposite. Not saying your criticism is wrong, just giving another view from a AWS newbie perspective. When I read most AWS product descriptions, I cannot understand what real world situations they are for. I either could not say, or could only vaguely say, what the product is in my own words. There seems to be a great scaffolding of assumed knowledge about the AWS system…

I feel similarly about AWS and I've been using it for a while. AWS is very powerful and flexible but the number of options and different services can be bewildering. I've suggested AWS to several developers I know that have never used it before and they were turned off from not being able to understand what was being offered and how much it would cost them.

For example, compare the Heroku pricing page and the AWS pricing page for EC2:

https://www.heroku.com/pricing

http://aws.amazon.com/ec2/pricing/

The Heroku one is significantly easier to understand in my opinion. I understand AWS has more options and is therefore harder to summarise though.

Post reply on HN