Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linux file write patterns: So you want to write to a file fast

blog.plenz.com

Post reply on HN
Print view View original

31–40 of 53 posts

Re: Linux file write patterns: So you want to write to a file fast

#31

Post by awda »

Phlarp wrote: post #29
masklinn wrote: post #26

Earlier quoted context omitted.

SSD controllers do write-leveling, the blocks a filesystem writes to is virtual and remapped (think VMEM)

wouldn't an ~8GB page/swap file being continuously rewritten on a 250GB drive still consume a non-negligible number of write cycles over several days / weeks at most?

It depends on how much free space you have on the SSD. But yes, especially because the swap file isn't ssd-aware, you get a high degree of write amplification which wears the disk more than necessary. That being said, newish SSDs can take a beating, even under these kinds of workloads.

Re: Linux file write patterns: So you want to write to a file fast

#32

Post by awda »

asveikau wrote: post #28
masklinn wrote: post #17

Earlier quoted context omitted.

> Assuming all I/O failures are EINTR is really, really odd, as if to say disks never fill up or fail and sockets never disconnect. The point is to retry on EINTR and to abort completely in case of other IO failures. assert(errno == EINTR); continue; is equivalent to if (errno == EINTR) continue; abort(); > But somebody somewhere is reading this and thinking this is a "semantically correct pattern" (as it is introduc…

You are wrong. assert is a no-op when NDEBUG is defined. Some compilers will set that for you in an optimized build. Using an assert in place of real error checking or otherwise relying on its side effects is consequently a huge wtf in C.

More like, assert() from assert.h is a huge wtf in C, because turning asserts off in optimized builds produces exactly these kinds of scenarios.

Re: Linux file write patterns: So you want to write to a file fast

#33

Post by dekhn »

Phlarp wrote: post #29
masklinn wrote: post #26

Earlier quoted context omitted.

SSD controllers do write-leveling, the blocks a filesystem writes to is virtual and remapped (think VMEM)

wouldn't an ~8GB page/swap file being continuously rewritten on a 250GB drive still consume a non-negligible number of write cycles over several days / weeks at most?

BTW I already have 16GB RAM on the machine. The swap file was 32GB.

Re: Linux file write patterns: So you want to write to a file fast

#34

Post by [deleted] »

maxhou wrote: post #9

actually since none of its tests ask to sync the data onto the disk, he might just be measuring each method efficiency in creating dirty pages. of course that depends on the amount of RAM the system has, and how the kernel VM parameters are tuned (sysctl vm.dirty_*) just add a fdatasync() call and you will take into account the time it takes to flush all dirty pages into the disk.

[deleted]

Re: Linux file write patterns: So you want to write to a file fast

#35

Post by cjensen »

The author has failed to account for command latency. If you write some bytes, there are a bunch of hardware buffering delays in getting bytes to disk including seek and rotational latency.

Async I/O avoids this. You can tell the I/O subsystem what you want to read next even while doing a write. The I/O is posted to the disk in modern systems, and the disk will begin seeking to the read site in parallel with informing the OS that the write has completed. Posting I/O even helps for SSDs to avoid the idle time on the SSD media between write done and read start.

Re: Linux file write patterns: So you want to write to a file fast

#36

Post by nhaehnle »

[deleted] wrote: post #34
maxhou wrote: post #9

actually since none of its tests ask to sync the data onto the disk, he might just be measuring each method efficiency in creating dirty pages. of course that depends on the amount of RAM the system has, and how the kernel VM parameters are tuned (sysctl vm.dirty_*) just add a fdatasync() call and you will take into account the time it takes to flush all dirty pages into the disk.

[deleted]

I am >99% confident that this is false for default setups. write() itself may return even before a command has been sent to the physical disk, and definitely before the disk reports that it is done. Waiting for the disk on every write() would kill the performance for most programs.

Re: Linux file write patterns: So you want to write to a file fast

#37

Post by Nican »

tytso wrote: post #11

It's 2014; why was the author using ext3 instead of ext4? Ext4 does have fallocate support. Also, if you use fallocate(2) instead of posix_fallocate(3), you don't have to worry about glibc trying to emulate fallocate() for those file systems which don't use it. Finally, it's a little surprising the author didn't try using O_DIRECT writes.

From a Boston Linux Usergroup discussion: https://www.mail-archive.com/discuss@blu.org/msg08490.html

Re: Linux file write patterns: So you want to write to a file fast

#38

Post by nkurz »

[deleted] wrote: post #34
maxhou wrote: post #9

actually since none of its tests ask to sync the data onto the disk, he might just be measuring each method efficiency in creating dirty pages. of course that depends on the amount of RAM the system has, and how the kernel VM parameters are tuned (sysctl vm.dirty_*) just add a fdatasync() call and you will take into account the time it takes to flush all dirty pages into the disk.

[deleted]

write() guarantees that the bits have been sent to the disk, and the disk reports that they have been written (or, if a nonvolatile cache is available, it is in the cache).

At least for Linux, I think that's dangerously untrue. On my machine, 'man write' even includes an explicit warning:

   A successful return from write() does not make any 
   guarantee that data has been  committed to  disk.
   In  fact, on some buggy implementations, it does not
   even guarantee that space has successfully been reserved
   for the data.  The only way to be sure is to call
   fsync(2) after you are done writing all your data.
There are ways to configure a file system so this is not the case, but they are rare. Is there a reference you could point to that would clarify what you are saying?

Re: Linux file write patterns: So you want to write to a file fast

#40

Post by mtdewcmu »

cjensen wrote: post #35

The author has failed to account for command latency. If you write some bytes, there are a bunch of hardware buffering delays in getting bytes to disk including seek and rotational latency. Async I/O avoids this. You can tell the I/O subsystem what you want to read next even while doing a write. The I/O is posted to the disk in modern systems, and the disk will begin seeking to the read site in parallel with informin…

I think there is some amount of write back caching in the kernel so that the application doesn't have to wait for each individual chunk to go to disk before it can submit the next chunk. I believe there's a sync on either file close or process termination, or some combination.
Post reply on HN