61–70 of 217 posts
Earlier quoted context omitted.
9p is a better network filesystem protocol, and it's available on Linux.
That's enticing, are there any references/benchmarks to back it up?
Compare https://ericvh.github.io/9p-rfc/rfc9p2000.html vs https://tools.ietf.org/html/rfc5661
I think this is a great product, but I've been actively avoiding NFS for a while now. It's a real shame that there isn't a better network FS protocol standardized already - NFS is complex, is usually a single point of failure (would be interesting to know if EFS isn't...) and comes with a whole set of cruft.
I understand why they did it, though. It's either this (everyone supports NFS, even Windows) or get crucified for vendor lock-in. Just like Github did yesterday ( https://news.ycombinator.com/item?id=9343021 ), even when they released their product as an open standard with an open source reference implementation.
Git was not designed for large files. But what github released yesterday primarily serves to promote github's central-server model for git. Moreover, it seems that it could have been better done within the git protocol itself (modify git to do more sparse pulls, and then try to fetch on a checkout when it is missing blobs, rather than erroring immediately).
I suspect AWS chose to use NFS for expedience, the net effect is positive, but I don't think it would have much mattered anyway.
Github is trying to inject their own server-model into the git protocol, with an extension that is only half thought through; that is a huge step backwards, open-source or not.
We're contemplating moving our static image files (JPG/PNG) from an EBS volume to serving them from an S3 bucket (so we can deploy a HA setup). It sounds like it would be a lot less code if we used EFS instead. Would you guys recommend S3 or EFS for this scenario?
EFS first and foremost will take away a huge amount of pain when trying to make filesystem-dependent legacy applications more reliable.
On the one hand, I've been wanting something like this for a while. On the other hand, I have so many bad memories of problems caused by NFS in production from the 90s that I'm leery.
At least the Linux NFS client made some big improvements in stability by the mid-2000s. FreeBSD took longer but I've heard they've fixed the kernel deadlocks as well. The other interesting note is that they apparently only support NFSv4, which has some welcome improvements: it uses TCP over a single port, avoids the entire portmap/statd/lockd train-wreck, UTF-8 everywhere, etc. One of the more interesting ones is tha…
- NFS (v4).
- Supports petabyte-scale file systems, thousands of concurrent NFS connections.
- Automatically grows/shrinks in size.
- Multi-zone storage and access.
- $0.30 / (Gigabyte * month).
- (Not mentioned) Both Linux and Windows have built-in NFS clients.
Earlier quoted context omitted.
As always, it depends on your use case. Just because it can be slow doesn't mean it's not a viable (and in some cases superior) option. We use it to store petabytes of large video files and our system is structured such that no folder ever has more than a couple files in it (>20 is rare). With properly tuned caching this works fantastically well for our use case and I would take the simpler code and reduced points of…
Which s3fs are you using? The fuse-based ones that I've tried were ridden with problems and poor error handling. Hangs and truncated files were the rule rather than the exception.
I think this is a great product, but I've been actively avoiding NFS for a while now. It's a real shame that there isn't a better network FS protocol standardized already - NFS is complex, is usually a single point of failure (would be interesting to know if EFS isn't...) and comes with a whole set of cruft.
I understand why they did it, though. It's either this (everyone supports NFS, even Windows) or get crucified for vendor lock-in. Just like Github did yesterday ( https://news.ycombinator.com/item?id=9343021 ), even when they released their product as an open standard with an open source reference implementation.
This is what should have existed instead of EBS all along. I'll never consider this to be as reliable as S3, but if I'm going to have a network filesystem I'd rather be dealing with NFS as my abstraction instead of virtualized network block devices.
May I ask why? My experience with NFS is pretty bad performance wise.
All services should eventually be like this. Just as you don't want to deal with provisioning BTUs of air conditioning or watts of power needed for your cloud infrastructure, why should you concern yourself about allocating a certain number of bytes of storage?
Maybe you don't want to, but there is definitely someone out there dealing with these issues.
E.g. during a heat-wave (100 F+) a transformer on top of the building (at a previous employer) started on fire. When the dust settled, we found out that the person in charge of it had not upgraded it as our power requirements increased. It was over-taxed and the heat-wave put it over the edge.