I stopped reading this when it became evident that the author was just promoting a library he’d written. I might have been a tiny bit more interested if I were a Rust developer.
Modern storage is plenty fast, but the APIs are bad
161–168 of 168 posts
Re: Modern storage is plenty fast, but the APIs are bad
#162Nice article. This part puzzled me though: > The operating system reads data in page granularity, meaning it can only read at a minimum 4kB at a time. That means if you need to read read 1kB split in two files, 512 bytes each, you are effectively reading 8kB to serve 1kB, wasting 87% of the data read. SSDs (whether SATA or NVMe) all read and write whole sectors at a time, right? I'm not sure what the sector size is,…
All SATA SSDs and almost all NVMe SSDs use 512-byte LBAs by default, but many NVMe SSDs can be reconfigured to use 4096-byte LBAs by default. The underlying NAND flash memory these days typically has a native page size of 16kB.
Re: Modern storage is plenty fast, but the APIs are bad
#163I don't think that io_uring is a bad API, developers just have to start using it.
Re: Modern storage is plenty fast, but the APIs are bad
#164Earlier quoted context omitted.
They everywhere in NUCs. Small size but hey :-) that's how I reboot some apps ultrafast... Mmap my whole app-state and gooo
Please may I pray your forgiveness first for my naivety and 2nd forgive please my disarray because I am keeping the first question unchanged because I think it's the way a lot of people will be thinking about Optane and I realised that this duality the memory and storage modes and it's not finished there either, needs far better marketing than I think Intel capable of. I immediately assumed that you are running Optan…
Errr not sure what happened here... Thanks for the correction I guess?
I'm sure I'm using something 'Optane', very fast, on a NUC PC. I'd be happy to learn something new today, though.
Re: Modern storage is plenty fast, but the APIs are bad
#165Earlier quoted context omitted.
Optane has been commercially available for five years already and it's not used in any device I'm aware of. Assuming it will find broad adoption at this point seems like a bad bet.
Optane in consumer devices is in a weird place. Write durability does not matter, SSD manufacturers reduce rewrite count with every new generation in the name of the cheaper prices and users are happy to accept that downgrade (see Samsung 980 Pro vs Samsung 970 Pro for a recent occasion). Speed matters to some extent. Many users don't observe performance improvements in their typical tasks when upgrading from SATA SS…
>Optane in consumer devices is in a weird place. Write durability does not matter, SSD manufacturers reduce rewrite count with every new generation in the name of the cheaper prices and users are happy to accept that downgrade (see Samsung 980 Pro vs Samsung 970 Pro for a recent occasion).
The odd thing about rewrite count is that it shrinks very quickly the bigger your drive is. In the past you used to have a 128GB boot SSD and install a few games on it. This meant that you constantly deleted and reinstalled games on that low capacity SSD increasing the rewrite count.
Nowadays you can get 500GB SSDs for a decent price. If you can install every game then you might never have to rewrite data on it through uninstallation/installation. A lot of files are only written once and then never changed. With greater capacity you can have more "write once" files.
Re: Modern storage is plenty fast, but the APIs are bad
#166Nice article. This part puzzled me though: > The operating system reads data in page granularity, meaning it can only read at a minimum 4kB at a time. That means if you need to read read 1kB split in two files, 512 bytes each, you are effectively reading 8kB to serve 1kB, wasting 87% of the data read. SSDs (whether SATA or NVMe) all read and write whole sectors at a time, right? I'm not sure what the sector size is,…
All SATA SSDs and almost all NVMe SSDs use 512-byte LBAs by default, but many NVMe SSDs can be reconfigured to use 4096-byte LBAs by default. The underlying NAND flash memory these days typically has a native page size of 16kB.
Thanks. What are the tradeoffs with NVMe sector size? There must be some reason to want 4096 if they added that option. Is there less write amplification as you get closer to the native page size or does the firmware avoid that anyway with its write leveling?
Re: Modern storage is plenty fast, but the APIs are bad
#167Earlier quoted context omitted.
Optane in consumer devices is in a weird place. Write durability does not matter, SSD manufacturers reduce rewrite count with every new generation in the name of the cheaper prices and users are happy to accept that downgrade (see Samsung 980 Pro vs Samsung 970 Pro for a recent occasion). Speed matters to some extent. Many users don't observe performance improvements in their typical tasks when upgrading from SATA SS…
>Optane in consumer devices is in a weird place. Write durability does not matter, SSD manufacturers reduce rewrite count with every new generation in the name of the cheaper prices and users are happy to accept that downgrade (see Samsung 980 Pro vs Samsung 970 Pro for a recent occasion). The odd thing about rewrite count is that it shrinks very quickly the bigger your drive is. In the past you used to have a 128GB…
You the consumer might not feel the need to rewrite, but the drive itself has to do this periodically as a matter of internal Flash maintenance. Even Read Disturb (think ram RowHammer) is a thing for NAND. Then you have MLC/TLC technology in conjunction with modern small geometries leading to faster leakage (data persistence) and you end up with failures like Samsung 840 slowing down to a crawl, or older driver forgetting all data after being unplugged for a while, leading to Samsung 840 firmware update forcing the drive to rewrite itself periodically in the background. https://www.techspot.com/news/60362-samsung-fix-slow-840-evo...
Re: Modern storage is plenty fast, but the APIs are bad
#168Earlier quoted context omitted.
They everywhere in NUCs. Small size but hey :-) that's how I reboot some apps ultrafast... Mmap my whole app-state and gooo
Please may I pray your forgiveness first for my naivety and 2nd forgive please my disarray because I am keeping the first question unchanged because I think it's the way a lot of people will be thinking about Optane and I realised that this duality the memory and storage modes and it's not finished there either, needs far better marketing than I think Intel capable of. I immediately assumed that you are running Optan…
BTW you asked for sk-y- (sku?), I have a good bunch of NUC7i7BNHX1 and some i5 with similar Optane storage. Don't know whether they're byte addressable, I did talk about mmap and speed...