Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon Elastic File System

aws.amazon.com

Post reply on HN
Print view View original

201–210 of 217 posts

Re: Amazon Elastic File System

#201

Post by rev_bird »

vizzah wrote: post #102
nkozyra wrote: post #42

Earlier quoted context omitted.

As has been mentioned, EBS can only be mounted to one instance at a time. If you could mount it to multiple it would effectively be the same thing, but then you have all sorts of write-locking issues. As the rest of the comments also allude, a lot of the cloud-entrenched world has abandoned NFS, or at least in AWS circles. I'm not one of these people. Rather than relying solely on puppet->all instances to handle mult…

Would it be a good idea to store Mysql db on EFS to be shared among load-balanced instances?

This idea got shot down pretty quick, but I love that it was asked. Coming up with potentially disastrous sideways use-cases for new toys is a favorite pastime of mine.

Re: Amazon Elastic File System

#202

Post by dekhn »

zippergz wrote: post #144
wmf wrote: post #106

Earlier quoted context omitted.

Yes, but NFS over the Internet may not be fast enough to be usable.

We used to do it over 10Base-T, which is slower than a lot of residential connections now. Well, latency will be higher over the internet, but bandwidth will be ok.

That's the whole point. Since NFS is such a round-trip oriented protocol, it's very sensitive to latency. You'll never fill up the bandwidth unless you do large bulk transfers in parallel.

Re: Amazon Elastic File System

#203

Post by dekhn »

buster wrote: post #192
matheweis wrote: post #113

Earlier quoted context omitted.

Not sure why NFS gets such a bad rap. On a low-latency network, properly tuned NFS has very few, if any performance issues. I've personally seen read/write rates exceeding 800MByte/sec on more or less white-box hardware, at which point it was limited by the underlying storage infrastructure (8Gbit fiber), not the NFS protocols. Dell has a 2013 white paper (I'm not affiliated with them, fwiw) about their fairly white…

(me: Working in the Messaging Business for ISPs) Not sure how it would ever be technically possible for a networked filesystem to get even near directly attached storage. But, for sure, the typical carrier grade EMC or Netapp is MUCH slower then a good SAN. I'm talking about petabytes of very small (average maybe 20kB) files with lots of _random_ sync writes and reads. NFS has a lot of other benefits, but it surely i…

Petabytes of 20k files?

I think you might want to use your filesystem more effectively.

Re: Amazon Elastic File System

#204

Post by ceejayoz »

goldenkey wrote: post #171
jonahx wrote: post #156

Earlier quoted context omitted.

As a developer (but not a sysadmin) I feel the opposite. Not saying your criticism is wrong, just giving another view from a AWS newbie perspective. When I read most AWS product descriptions, I cannot understand what real world situations they are for. I either could not say, or could only vaguely say, what the product is in my own words. There seems to be a great scaffolding of assumed knowledge about the AWS system…

heroku uses ec2 under the hood. heroku is heresy for any competent programmer. despite aws being packaged services at least both managed db and ec2 show you your actual servers and let you choose specs.

Heroku is used by a wide variety of competent programmers. Not everyone wants or needs to choose specs or see actual servers.

Re: Amazon Elastic File System

#205

Post by ceejayoz »

tw04 wrote: post #162
rjurney wrote: post #131

Earlier quoted context omitted.

Says the guy who will own one within a year. Or the Android knock off.

PSSST: Android watches were a thing long before Apple pre-announced theirs.

MP3 players were available long before the iPod. Apple has a pretty good track record of turning niche markets into mainstream ones - I'll be interested to see if they can do it with the smartwatch category.

Re: Amazon Elastic File System

#206

Post by Terretta »

mhuffman wrote: post #150
jeffbarr wrote: post #31

Earlier quoted context omitted.

We'll be providing a lot more details later.

I am a huge AWS fan, and user. Look at my accounts for the sums I have spent over the years. I lover new AWS services. But, this is my main complaint with AWS -- don't announce until you have usable numbers and workable examples for early adopters!

On the contrary, the earlier AWS can let you know they're working on a general kind of thing, the better you can plan your own roadmap.

Re: Amazon Elastic File System

#207

Post by efsavage »

rjurney wrote: post #131
efsavage wrote: post #126

Earlier quoted context omitted.

Maybe you didn't notice, but they just invented the Watch. Imagine...if you could put a whole entire clock...on your wrist. Almost everybody has a wrist!

Says the guy who will own one within a year. Or the Android knock off.

I've actually long advocated that the pocket brick is a temporary form factor for a phone/computer. The watch will soon be the brain of your computing, to which you will be able to pair any number of dumb displays, keyboards, earpieces, etc.

But you're right, I will likely get an iWatch. It will go in my QA toybox with my macbooks and ipads and iphones.

Re: Amazon Elastic File System

#208

Post by azinman2 »

dekhn wrote: post #137
azinman2 wrote: post #58

Earlier quoted context omitted.

Ya. I'm wondering how you'd back say a PostgreSQL instance over nfs? It is just different needs that require different solutions.

You run PostgreSQL or any other DB server with its DB data dir on the NFS mount. Oracle supports this- and they even wrote a user-space NFS client to "get the highest level of performance" (because they thought the kernel NFS implementation sucked). The important bit is to ensure the NFS client and server implementation handle whatever POSIX features are required by the DB server.

Why would you want to? You can't share across two instances at the same time anyway, it's going to be slower/more edge case-y, and the cost with Amazon is higher?

Re: Amazon Elastic File System

#209

Post by istvan__ »

notacoward wrote: post #200
istvan__ wrote: post #165

NFS? I just simply don't see the place for this product. The world moved towards the use of smart data formats on platforms like S3 and HDFS. Where would be a NFS service better? It is kind of hard to see for me to use a distributed filesystem over a distributed datastore. Wondering about the outage scenarios with that. Historically speaking, the drivers that deal with this in the linux kernel are not the best in ter…

There are two things wrong with the "nobody should use a distributed file system" meme. (Disclaimer: I'm a distributed file system developer.) (1) Not all development is green-field and thus subject to using this week's fashionable data store. A lot of applications already use file systems and rely on their semantics. Ripping up the entire storage layer of a large and complex application would be irresponsible, as it…

Funny you mentioned that this is a meme to you, while it is really a technical consideration to me, and I supplied some details about my concerns.

Answers following your numbering:

(1) Calling S3 "this week's fashionable data store" is like saying that an elephant is an interesting microbe. The rest of your points about "please do not innovate, we have filesystems for 40 years and this is how you store your data". I do not agree. Disclaimer, I was member of the team that moved amazon.com from using an NFS based storage to S3. It was a great success, and it solved many of our problems including dealing with the insane amount of issues that was introduced by running an NFS cluster at that scale. And I would like to emphasize on scale, because your operational problems are quite often increase worse than linear with you scale.

I know about legacy code, and running several legacy services in production as of now. I can tell you one thing. There is a point when it is not financially viable to keep rolling with the legacy code. This point is very different based on your actual use case, banks tend to run "legacy code" while web2.0 companies tend to innovate and replace systems with a faster peace. I don't see any conflict here. We even did a compatibility layer for the new solution and it was possible to run your legacy code using the new system and your software was untouched.

(2) Nested directories is a logical layer on the top of how the data is stored, aka a view, your are a distributed FS developer so I guess you understand that. S3 also supports nested directories no biggie here. Security. Well this is kind of weird because last time I checked S3 had an extensive security http://aws.amazon.com/s3/faqs/#security_anchor Now the rest of your question can be re-phrased: "I am used to X why isn't there X with this new thing???" I am not sure how many of the file system users use SElinux my educated guess is that roughly around 1-10%. It is a very complex system that not too many companies invest using. For our use cases the fine grained ACLs were good enough so we are using those. File system durability: yes it is very important, this why I was kind of shocked about this bug: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/317781/...

I guess you are right about the overhead of reading and writing, dealing with http headers etc. If the systems that benefit the most from S3 where single node systems it would be silly to use S3 at the first place. We are talking about 1000 - 10000 computers using the same data storage layer. And you can tell me if I am wrong but if you would like to access the same files on these nodes using a FS than you are going to end up with a locking hell. This is why modern software that is IO heavy moved away from in-place edits towards the "lock free" data access. Look at the implementation of Kafka log files or how Aeron writes files. This is exactly the same schematics how use use S3. Accident? ;)

I would like to repeat my original question: I don't see huge market for a distributed FS. I might be wrong, but this is how I see it.

http://kafka.apache.org/ https://github.com/real-logic/Aeron

Re: Amazon Elastic File System

#210

Post by notacoward »

istvan__ wrote: post #209
notacoward wrote: post #200

Earlier quoted context omitted.

There are two things wrong with the "nobody should use a distributed file system" meme. (Disclaimer: I'm a distributed file system developer.) (1) Not all development is green-field and thus subject to using this week's fashionable data store. A lot of applications already use file systems and rely on their semantics. Ripping up the entire storage layer of a large and complex application would be irresponsible, as it…

Funny you mentioned that this is a meme to you, while it is really a technical consideration to me, and I supplied some details about my concerns. Answers following your numbering: (1) Calling S3 "this week's fashionable data store" is like saying that an elephant is an interesting microbe. The rest of your points about "please do not innovate, we have filesystems for 40 years and this is how you store your data". I…

"please do not innovate, we have filesystems for 40 years and this is how you store your data"

Please don't put words in my mouth like that. It's damn rude. I never said anything that was even close.

"S3 also supports nested directories no biggie here."

Not according to the API documentation I've seen. There are buckets, and there are objects within buckets. Nothing about buckets within buckets. Sure, there are umpteen different ways to simulate nested directories using various naming conventions recognized by an access library, but there's no standard and thus no compatibility. You also lose some of the benefits of true nested directories, such as combining permissions across different levels of the hierarchy. Also no links (hard or soft) which many people find useful, etc. Your claim here is misleading at best.

"last time I checked S3 had an extensive security"

Yes, it has its very own permissions system, fundamentally incompatible with any other and quite clunky to use. That still doesn't answer the question of how you'd do anything like SELinux with it.

"File system durability: yes it is very important, this why I was kind of shocked about this bug: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/317781/...

Open up your bug list and we can have that conversation. Throwing stones from behind a proprietary wall is despicable.

"you can tell me if I am wrong but if you would like to access the same files on these nodes using a FS than you are going to end up with a locking hell."

You're wrong. Maybe you've only read about distributed file systems (or databases which have to deal with similar problems) from >15 years ago, but things have changed a bit since then. In fact, if you were at Amazon you might have heard of a little thing called Dynamo which was part of that evolution. Modern distributed systems, including distributed file systems, don't have that locking hell. That's just FUD.

"I don't see huge market for a distributed FS."

Might want to tell that to the EFS team. Let me know how that goes. In fact you might be right, but whether there's a market has little to do with your pseudo-technical objections. Many technologies are considered uncool long before they cease being useful.

Post reply on HN