Linux file write patterns: So you want to write to a file fast • Hacker News Board

Re: Linux file write patterns: So you want to write to a file fast

#51

Post by vacri » Fri, May 02, 2014, 4:32 AM UTC

ef47d35620c1 wrote: post #24

dekhn wrote: post #22

Earlier quoted context omitted.

I used to, but given not a single one of my SSD drives (I have 4 deployed in my house) has so much as balked once in a year of continuous deployment, I am cautiously optimistic.

I've had very good reliability from my SSD drives as well. Some have been running almost continuously since 2009.

I ran a 60GB SSD as my Windows machine's system drive (with pagefile) for four years before it started showing problems, and that machine saw a few hours use almost every day. It was >90-95% full for most of that time.

Re: Linux file write patterns: So you want to write to a file fast

#52

Post by angry_octet » Fri, May 02, 2014, 11:22 AM UTC

"For simplicity I’ll try things on my laptop computer with Ext3+dmcrypt and an SSD. This is “read a 128MB file and write it out”, repeated for different block sizes, timing each" The whole thing is completely invalid for measuring actual I/O hierarchy efficiencies because of (a) write sizes too small, would be in buffer cache of unknown hotness, (b) dmcrypt introduces a whole layer of indirection and timing variability and (c) on an SSD, almost anything could be happening regarding cache and syncs. Also, mount options, % disk used, small sample sizes, unknown contention effects, etc. This is a good example of how to convince yourself of something and yet be less accurate than a divining rod.

Re: Linux file write patterns: So you want to write to a file fast

#53

Post by dekhn » Fri, May 02, 2014, 2:28 PM UTC

StillBored wrote: post #44

dekhn wrote: post #16

Earlier quoted context omitted.

Most people who use O_DIRECT writes stop quickly, thinking it's "slow". What's actually happening is you're seeing what the system is actually capable of in terms of write bandwidth, without any of the 'clever' optimizations like write caching.

I don't think this is accurate. We have a kernel bypass for disk operations. We use our own memory buffers, and bypass the filesystem, block, and SCSI midlayers. Our stuff is basically what O_DIRECT should be. There are cases where we are 50% faster than O_DIRECT without any "caching". Furthermore, in high bandwidth applications (>4GB/sec) without O_DIRECT its easy to become CPU limited in the blk/midlayer so again w…

Sure, if you're dealing with extremely high bandwidth apps (4GB/sec is pretty high bandwidth!) what I said doesn't apply.

THe number of people who are sustaining 4GB/sec (on a single machine/device array) is pretty small and they have a reason to go beyond the straightforward approaches the kernel makes available through a simple API (everything you described, like bypass, puts you in a rare category).

Anyway, when I was swapping to SSD, the kswapd process was using 100% of one core while swapping at 500MB/sec. I suspect many kernel threads haven't been CPU-optimized for high throughput.