11–20 of 53 posts
Also, if you use fallocate(2) instead of posix_fallocate(3), you don't have to worry about glibc trying to emulate fallocate() for those file systems which don't use it.
Finally, it's a little surprising the author didn't try using O_DIRECT writes.
The first hit for "SSD write speeds" says ~500Mb/s (I hope I got the right b). I didn't bother clicking the links that just the blurb under it. He's dumping 128Mb in ~200 ms. I'm not sure there is much room for improvement.
Nope, it's MB not Mb.
The first hit for "SSD write speeds" says ~500Mb/s (I hope I got the right b). I didn't bother clicking the links that just the blurb under it. He's dumping 128Mb in ~200 ms. I'm not sure there is much room for improvement.
A bit surprising (considering he started off talking about coredumps) that he doesn't mention sparse files. Core dumps can be very sparse, and you might save time and definitely will save space by not writing out the all-zeroes parts.
to do that, wouldn't you have to look at every byte just to detect the runs of 0s, that would mean that you have to pull the whole file through the memory hierarchy of your system (rather than just passing chunks from syscall to syscall) wouldn't that alone slow you down significantly?
This could be done quickly in the kernel. RAID (which does pass the data through multiple transformations) subsystem metrics printed at boot demonstrate that.
The first hit for "SSD write speeds" says ~500Mb/s (I hope I got the right b). I didn't bother clicking the links that just the blurb under it. He's dumping 128Mb in ~200 ms. I'm not sure there is much room for improvement.
I've frequently observed sustained 500MB/sec writes and reads on my cheap ($250) 250GB SSDs. One of my favorite instances was running out of RAM while assembling a gigapan in Hugin. I added a swap file on my SSD and continued- it ran over night with nearly 500MB/sec reads and writes more or less continuously, but the job finished fine.
It's 2014; why was the author using ext3 instead of ext4? Ext4 does have fallocate support. Also, if you use fallocate(2) instead of posix_fallocate(3), you don't have to worry about glibc trying to emulate fallocate() for those file systems which don't use it. Finally, it's a little surprising the author didn't try using O_DIRECT writes.
Assuming all I/O failures are EINTR is really, really odd, as if to say disks never fill up or fail and sockets never disconnect. He does say: > in a real program you’d have to do real error handling instead of assertions, of course But somebody somewhere is reading this and thinking this is a "semantically correct pattern" (as it is introduced) and may just copy-paste it into their program. Especially when contrasti…
The point is to retry on EINTR and to abort completely in case of other IO failures.
assert(errno == EINTR);
continue;
if (errno == EINTR)
continue;
abort();
Even if they do, it likely will not actually do any harm, it'll just kill the program instead of gracefully handle error.
The first hit for "SSD write speeds" says ~500Mb/s (I hope I got the right b). I didn't bother clicking the links that just the blurb under it. He's dumping 128Mb in ~200 ms. I'm not sure there is much room for improvement.
M B . Bytes. nobody quotes disk speeds in bits (or if they do I typically ignore them). I've frequently observed sustained 500MB/sec writes and reads on my cheap ($250) 250GB SSDs. One of my favorite instances was running out of RAM while assembling a gigapan in Hugin. I added a swap file on my SSD and continued- it ran over night with nearly 500MB/sec reads and writes more or less continuously, but the job finished…
Assuming all I/O failures are EINTR is really, really odd, as if to say disks never fill up or fail and sockets never disconnect. He does say: > in a real program you’d have to do real error handling instead of assertions, of course But somebody somewhere is reading this and thinking this is a "semantically correct pattern" (as it is introduced) and may just copy-paste it into their program. Especially when contrasti…
> Assuming all I/O failures are EINTR is really, really odd, as if to say disks never fill up or fail and sockets never disconnect. The point is to retry on EINTR and to abort completely in case of other IO failures. assert(errno == EINTR); continue; is equivalent to if (errno == EINTR) continue; abort(); > But somebody somewhere is reading this and thinking this is a "semantically correct pattern" (as it is introduc…
Earlier quoted context omitted.
M B . Bytes. nobody quotes disk speeds in bits (or if they do I typically ignore them). I've frequently observed sustained 500MB/sec writes and reads on my cheap ($250) 250GB SSDs. One of my favorite instances was running out of RAM while assembling a gigapan in Hugin. I added a swap file on my SSD and continued- it ran over night with nearly 500MB/sec reads and writes more or less continuously, but the job finished…
I weep for the memory sectors that got re-written continuously for an entire night.