I appreciate the desire for stripped-down websites, but "body { max-width: 800px; }" makes a world of difference for readability.
Firefox reader mode is pretty useful in these cases, and can also save you from websites with the opposite problem.
Files Are Fraught with Peril
51–60 of 79 posts
Re: Files Are Fraught with Peril
#52Earlier quoted context omitted.
I write database engines for Linux and the filesystem situation really is a train wreck. It isn't anyone's fault per se, design choices and standards were accreted over many decades that in isolation made sense in some context but which in aggregate have many poorly defined interactions and create conflicting requirements. And you can't change any of it easily because there are several decades of software built using…
Very few applications need random read/write access to files. Most of the time, you need to read an entire file in, or write an entire file out via the streaming access APIs. This core fact of typical usage is why I think so many application developers have naive expectations about filesystem behavior. Read-only random-access is well served by mmap() and pread(). For random writes access within a file, preadv2() and…
I would only add that mmap should be used with care[0] and pread should be preferred.
[0]: https://www.sublimetext.com/blog/articles/use-mmap-with-care
Re: Files Are Fraught with Peril
#53The suggestion to "use SQLite" to write files should come with a caveat. Such files can still become corrupted, and cannot be inspected nor amended with a text editor. Any truly robust file-on-local-disk storage scheme for a non-cloud system should allow for manual diagnostics and repair.
Text-based formats are the best for repairability but not so efficient for structuring, querying and storing information. It’s a tradeoff, as always!
A search index can be maintained in binary format; it can be rebuilt in the event of corruption.
Re: Files Are Fraught with Peril
#54Earlier quoted context omitted.
> From the filesystem point of view, I don't think that Dropbox is so much concerned about you losing your data because of corruption. As, anyway they are supposed to be safe and versioned in the cloud. They could get to the Cloud corrupted, and then continue replicating in your devices, so no...
That's where the "and versioned" comes in handy
Versioned wont help if it's the first copy that got into dropbox.
Re: Files Are Fraught with Peril
#55Earlier quoted context omitted.
> filesystem API design is obviously a problem It's an API that's so critical it's almost impossible to change or rewrite. The API itself has barely moved in 30 years, whether it's POSIX or Win32. Possibly the only widely adopted change in filesystem API has been "S3" and compatibles, which provide a completely different set of atomicity semantics as well as being network-native.
The typical solution is to provide a new, safe API while supporting the broken stuff forever.
Except if the API's have different semantics and you can't reconcile them. Then your new API can't be significantly different from your old API or your old API gets broken on the edge cases.
That's the situation we are in with file systems API's now.
Re: Files Are Fraught with Peril
#56I appreciate the desire for stripped-down websites, but "body { max-width: 800px; }" makes a world of difference for readability.
Firefox reader mode is pretty useful in these cases, and can also save you from websites with the opposite problem.
True, but sadly Chrome still lacks a reader mode for whatever reason
Re: Files Are Fraught with Peril
#57There's also this from 2015: https://news.ycombinator.com/item?id=10725859
Re: Files Are Fraught with Peril
#58Earlier quoted context omitted.
Very few applications need random read/write access to files. Most of the time, you need to read an entire file in, or write an entire file out via the streaming access APIs. This core fact of typical usage is why I think so many application developers have naive expectations about filesystem behavior. Read-only random-access is well served by mmap() and pread(). For random writes access within a file, preadv2() and…
I would only add that mmap should be used with care[0] and pread should be preferred. [0]: https://www.sublimetext.com/blog/articles/use-mmap-with-care
It should be noted that much of the problems with mmap cited there aren't actually mmap's fault, but the fact that I/O errors become POSIX signals, and the API for POSIX signals really blows. (POSIX signals are probably even more ripe than filesystems for needing a different approach).
I'm surprised that page didn't mention the other issue with mmap, which is concurrent file access.
Re: Files Are Fraught with Peril
#59This is hard only on GNU/Linux because it's inherently broken because solving this once and for all proved too hard of a problem for volunteers working on the kernel and filesystem code. That's a hard to swallow fact if one's favorite operating system is GNU/Linux. There is one group of people who did solve it and they solved it correctly: the ZFS team under Jeff Bonwick.
If you care about data integrity, if you need an operating system where filesystem operations aren't leaky abstractions and where fsync(2) works as POSIX specifies, if correctness of operation is important to you, use an illumos-based operating system like SmartOS or any other based on the illumos codebase. Put some effort in and learn real UNIX and leave these long ago solved problems where they belong, back in the past century.
Re: Files Are Fraught with Peril
#60Earlier quoted context omitted.
Okay and? Why does userspace care about this? Should Emacs not run on btrfs?
This is rather like the bluescreen problem: if your application tries to open a file from the normal filesystem and it's corrupt, the user blames the application. If the user opens a file in the Dropbox folder, they blame Dropbox . So they end up engaging in heroics to not be blamed for it. (Windows has gone to increasing lengths to accomodate and contain badly written drivers, since most bluescreens are caused by dr…
Which prevents any CUDA kernels from running longer than like 5 seconds or so. Which means you can't use the GPU to spawn it's own kernels with no PCIe/driver latency in between, because this master has to finish before windows kills it.
Last I checked, providing callback function pointers to binary vendor libraries (read/write adapters for FFT come to mind, allowing on-the-fly metric computation or skipping an intermediate storage for FFT convolution) was only possible on Linux, and with statically linking said vendor library into the software (incidentally breaking binary distributability for GPL).