This could be a byproduct of what they developed for Aurora (their upcoming redesigned mysql) shared storage. Probably the other way around.
Amazon Elastic File System
111–120 of 217 posts
Re: Amazon Elastic File System
#112Earlier quoted context omitted.
Until Amazon starts offering EC2 OS X instances, I'm not sure that's as relevant.
I think it means you can mount an EFS instance on OS X
No, it doesn't. From the announcement:
"Only Amazon EC2 instances within the Amazon Virtual Private Cloud (Amazon VPC) you specify can directly access your Amazon EFS file systems."
Re: Amazon Elastic File System
#113This is what should have existed instead of EBS all along. I'll never consider this to be as reliable as S3, but if I'm going to have a network filesystem I'd rather be dealing with NFS as my abstraction instead of virtualized network block devices.
May I ask why? My experience with NFS is pretty bad performance wise.
Not sure why NFS gets such a bad rap. On a low-latency network, properly tuned NFS has very few, if any performance issues.
I've personally seen read/write rates exceeding 800MByte/sec on more or less white-box hardware, at which point it was limited by the underlying storage infrastructure (8Gbit fiber), not the NFS protocols.
Dell has a 2013 white paper (I'm not affiliated with them, fwiw) about their fairly white box setup that achieved > 100,000 iops, 2.5GBbyte/s sequential write, and 3.5GBbyte/s sequential read: http://en.community.dell.com/techcenter/high-performance-com...
Re: Amazon Elastic File System
#114This is what should have existed instead of EBS all along. I'll never consider this to be as reliable as S3, but if I'm going to have a network filesystem I'd rather be dealing with NFS as my abstraction instead of virtualized network block devices.
May I ask why? My experience with NFS is pretty bad performance wise.
Yeah, if you need a performant file system you use an instance with a lot of local storage or you get out of EC2.
Re: Amazon Elastic File System
#115Re: Amazon Elastic File System
#116Earlier quoted context omitted.
Is anyone suggesting that? It is widely considered best practice to back your databases by SSDs.
Well he's pointing out EFS is not a replacement for EBS. You can run postgres on EBS volumes with PIOPS.
If EFS offers PIOPS, couldn't you run Postgres on it, too?
Re: Amazon Elastic File System
#117Does anyone else cringe when someone suggestions using NFS in production? I can't be the only one that has been woken up at 2am because of an NFS outage.
On a previous workplace, we had a pretty beefy VMware set up backed by NFS. Performance was excellent, and file level access offers a lot of functionality you can't have with for example iSCSI.
Re: Amazon Elastic File System
#118Earlier quoted context omitted.
Depending on how you tune it, it can be a monster. Several years ago I was managing a cluster with ~5K linux instances all mounted to ~4PB of spinning disk served with NFS. Worked very well.
Can you speak to any stale NFS handle problems? I've used NFS at home and have had NFS file handle problems but IIRC that was only when there were problems like kernel faults or network partitions. However several of my colleagues at work have many NFS horror stories and are adamant that NFS does not scale well. Is NFS stability at scale simply a function of your underlying network and infrastructure stability in you…
Having a low latency network is key (e.g. LAN across an office or maybe a city should be fine, but WAN across country would be bad news).
Hard coding the fsid's on the server side can help if you change your exports a lot.
Using the automounter can help, although it does not scale to large numbers of mounts well.
Re: Amazon Elastic File System
#119Salient points: - NFS (v4). - Supports petabyte-scale file systems, thousands of concurrent NFS connections. - Automatically grows/shrinks in size. - Multi-zone storage and access. - $0.30 / (Gigabyte * month). - (Not mentioned) Both Linux and Windows have built-in NFS clients.
Thanks for the time-saving description. Osx also has a built-in NFS client.
Unfortunately it's fairly flaky release to release.
Re: Amazon Elastic File System
#120Sounds like they are recommending to use an anti-pattern. Share your storage across instances.... You really don't want to do that.
It works for HPC/VFX and "big data"
/home thats on a network drive with an SLA....
concurrent versions of your app sitting next to each other somewhere in $PATH meaning you can roll forward/back by typing app-$version (or what ever your convention is)
Quickly and efficiently share files between instances.
It basically means you can treat a node group as single proper linux cluster, without having to buy GPFS licenses or indulge in the horror that is glusterfs.