Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon Elastic File System

aws.amazon.com

Post reply on HN
Print view View original

141–150 of 217 posts

Re: Amazon Elastic File System

#141

Post by jsprogrammer »

vacri wrote: post #134
click170 wrote: post #74

Earlier quoted context omitted.

Fair enough, but one of the biggest reasons I ignore these types of posts is because too often they lack these critical details, they come off as puff pieces to me. IMO these posts would hold value for a larger audience if that info was included in them.

It's a upcoming product announcement page from a vendor asking for preview registration, how can it be anything other than a puff piece?

The trick is, based on the title and other marketing tricks, one doesn't find out it's an actual puff piece until one has already spent time reading at least a portion of the puff. By then, you just feel used.

Re: Amazon Elastic File System

#142

Post by andrewchambers »

rjurney wrote: post #129
vially wrote: post #94

Earlier quoted context omitted.

Would you mind explaining how exactly "Apple consistently leads and innovates ahead of other companies and open source"?

Are you kidding? Apple reinvents every space they enter with profound invention. Do you remember computers before the Mac? Notebooks before the Macbook? Music players before the iPod? Phones before the iPhone? Tablets before the iPad? And in a month you'll think: watches before the Apple Watch?

Yes I do remember all those things, they were pretty good.

Re: Amazon Elastic File System

#143

Post by georgerobinson »

rectang wrote: post #121
click170 wrote: post #77

Earlier quoted context omitted.

Can you speak to any stale NFS handle problems? I've used NFS at home and have had NFS file handle problems but IIRC that was only when there were problems like kernel faults or network partitions. However several of my colleagues at work have many NFS horror stories and are adamant that NFS does not scale well. Is NFS stability at scale simply a function of your underlying network and infrastructure stability in you…

Unlike typical local Unix file systems, NFS does not support "delete on last close" semantics. Ordinarily, even if you unlink a file, the operating system keeps the inode around until the last filehandle referencing it goes away. But an NFS mount cannot know when all filehandles on all networked systems have closed. When you attempt to read from an NFS file handle whose underlying file has been deleted out from under…

Doesn't NFS rename deleted files to some temp name so other clients can still read and write to it using their existing file handle?

Re: Amazon Elastic File System

#144

Post by zippergz »

wmf wrote: post #106
echeese wrote: post #96

Earlier quoted context omitted.

I think it means you can mount an EFS instance on OS X

Yes, but NFS over the Internet may not be fast enough to be usable.

We used to do it over 10Base-T, which is slower than a lot of residential connections now. Well, latency will be higher over the internet, but bandwidth will be ok.

Re: Amazon Elastic File System

#145

Post by tracker1 »

mrmondo wrote: post #133
chuckcode wrote: post #16

I wish AWS would be a little more technical in their product descriptions and announcements. If there was ever an audience that wants technical data about how this type of technology will scale and compares to existing technology like glusterfs it would be the AWS users. Instead the performance specifications are ".. and is designed to provide the throughput, IOPS, and low latency needed for a broad range of workload…

Until you see technical details on any file, network or storage system you should always assume your writes (or data for that matter) are/is not safe. It's disappointing to see a product launched and hyped without the appropriate details required to make an informed decision around its use.

I think that you are correct, that some technical details are definitely needed for comparison... To me, this matches up against Azure Files (which is CIFS/SAMBA based)...

Though, even if redundancy isn't a factor, it's nice that you can have network file shares without having to run your own dedicated instance in a given cloud. There are plenty of situations where having a common networked filesystem makes sense across a few servers for purposes of sharing some information, without it being physically on all of them... seeding static content, or user-uploaded files for example. It makes a given solution simpler to implement initially (though other considerations may take hold as a site/application grows).

Typically Azure seems to perform better in terms of storage i/o over AWS. Though, if you are disk constrained in ways you can't reasonable scale horizontally, then you may be better off with something using local disks, and your own backup strategy on something cheaper (Linode DigitalOcean, Joyent, etc).

Re: Amazon Elastic File System

#146

Post by discodave »

pyre wrote: post #70
biot wrote: post #54

All services should eventually be like this. Just as you don't want to deal with provisioning BTUs of air conditioning or watts of power needed for your cloud infrastructure, why should you concern yourself about allocating a certain number of bytes of storage?

> Just as you don't want to deal with provisioning BTUs of air conditioning or watts of power needed for your cloud infrastructure Maybe you don't want to, but there is definitely someone out there dealing with these issues. E.g. during a heat-wave (100 F+) a transformer on top of the building (at a previous employer) started on fire. When the dust settled, we found out that the person in charge of it had not upgrade…

That is a reasonable analogy but the question is whether or not your in-house support will do a better job than Amazon.

Re: Amazon Elastic File System

#147

Post by dekhn »

georgerobinson wrote: post #143
rectang wrote: post #121

Earlier quoted context omitted.

Unlike typical local Unix file systems, NFS does not support "delete on last close" semantics. Ordinarily, even if you unlink a file, the operating system keeps the inode around until the last filehandle referencing it goes away. But an NFS mount cannot know when all filehandles on all networked systems have closed. When you attempt to read from an NFS file handle whose underlying file has been deleted out from under…

Doesn't NFS rename deleted files to some temp name so other clients can still read and write to it using their existing file handle?

some implementations of NFS use "silly rename" to get delete on last close semantics. http://nfs.sourceforge.net/#faq_d2

I think that's limited to v2/v3 and not fully general or reliable.

Re: Amazon Elastic File System

#148

Post by smarterclayton »

tristanz wrote: post #107
notacoward wrote: post #85

As a GlusterFS developer, and furthermore the founder of a project to create a "cloud" version of GlusterFS aimed at exactly this use case, this is pretty darn interesting to me. I guess I'm supposed to pick away at all the feature differences between EFS and GlusterFS-on-EC2, or something like that, but for now I'm more pleased to see that this use case is finally being addressed and the solution seems well integrat…

I'd love to see an easy to manage version for containerized cloud stacks like Kubernetes and Mesos. I think the AWS move here validates that NFS is still an ok pattern to use for some applications. I've had mixed experiences with Gluster a year ago, including lost files, so something that was rock solid and easy to manage would be a great product.

Yeah, that's something the Ceph and Gluster teams have been working on to integrate with Kubernetes seamlessly (or at least, easily). The gluster core drivers for volumes landed recently, and self service service on demand FS is a goal.

Re: Amazon Elastic File System

#149

Post by mikey_p »

joemccall86 wrote: post #19

We're contemplating moving our static image files (JPG/PNG) from an EBS volume to serving them from an S3 bucket (so we can deploy a HA setup). It sounds like it would be a lot less code if we used EFS instead. Would you guys recommend S3 or EFS for this scenario?

Depends, I work with a number of HA Drupal sites, and Drupal 7 can do most files in S3 but it still likes to put generated CSS/JS files and tmp files on local disk. In most of these cases (or for already existing sites) it's usually easier to just use NFS or Gluster instead of trying to force everything to S3.

upstream issue: https://www.drupal.org/node/2044509

Re: Amazon Elastic File System

#150

Post by mhuffman »

jeffbarr wrote: post #31
chuckcode wrote: post #16

I wish AWS would be a little more technical in their product descriptions and announcements. If there was ever an audience that wants technical data about how this type of technology will scale and compares to existing technology like glusterfs it would be the AWS users. Instead the performance specifications are ".. and is designed to provide the throughput, IOPS, and low latency needed for a broad range of workload…

We'll be providing a lot more details later.

I am a huge AWS fan, and user. Look at my accounts for the sums I have spent over the years. I lover new AWS services. But, this is my main complaint with AWS -- don't announce until you have usable numbers and workable examples for early adopters!
Post reply on HN