Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Does a Database for SSDs Look Like?

brooker.co.za

Post reply on HN
Print view View original

21–30 of 127 posts

Re: What Does a Database for SSDs Look Like?

#21

Post by PunchyHamster »

> WALs, and related low-level logging details, are critical for database systems that care deeply about durability on a single system. But the modern database isn’t like that: it doesn’t depend on commit-to-disk on a single system for its durability story. Commit-to-disk on a single system is both unnecessary (because we can replicate across storage on multiple systems) and inadequate (because we don’t want to lose writes even if a single system fails).

And then a bug crashes your database cluster all at once and now instead of missing seconds, you miss minutes, because some smartass thought "surely if I send request to 5 nodes some of that will land on disk in reasonably near future?".

I love how this industry invents best practices that are actually good then people just invent badly researched reasons to just... not do them.

Re: What Does a Database for SSDs Look Like?

#22

Post by dist1ll »

PunchyHamster wrote: post #21

> WALs, and related low-level logging details, are critical for database systems that care deeply about durability on a single system. But the modern database isn’t like that: it doesn’t depend on commit-to-disk on a single system for its durability story. Commit-to-disk on a single system is both unnecessary (because we can replicate across storage on multiple systems) and inadequate (because we don’t want to lose w…

> "surely if I send request to 5 nodes some of that will land on disk in reasonably near future?"

That would be asynchronous replication. But IIUC the author is instead advocating for a distributed log with synchronous quorum writes.

Re: What Does a Database for SSDs Look Like?

#23

Post by throw0101a »

Sesse__ wrote: post #16
londons_explore wrote: post #3

Median database workloads are probably doing writes of just a few bytes per transaction. Ie 'set last_login_time = now() where userid=12345'. Due to the interface between SSD and host OS being block based, you are forced to write a full 4k page. Which means you really still benefit from a write ahead log to batch together all those changes, at least up to page size, if not larger.

A write-ahead log isn't a performance tool to batch changes, it's a tool to get durability of random writes. You write your intended changes to the log, fsync it (which means you get a 4k write), then make the actual changes on disk just as if you didn't have a WAL. If you want to get some sort of sub-block batching, you need a structure that isn't random in the first place, for instance an LSM (where you write all o…

> A write-ahead log isn't a performance tool to batch changes, it's a tool to get durability of random writes.

¿Por qué no los dos?

Re: What Does a Database for SSDs Look Like?

#25

Post by Sesse__ »

throw0101a wrote: post #23
Sesse__ wrote: post #16

Earlier quoted context omitted.

A write-ahead log isn't a performance tool to batch changes, it's a tool to get durability of random writes. You write your intended changes to the log, fsync it (which means you get a 4k write), then make the actual changes on disk just as if you didn't have a WAL. If you want to get some sort of sub-block batching, you need a structure that isn't random in the first place, for instance an LSM (where you write all o…

> A write-ahead log isn't a performance tool to batch changes, it's a tool to get durability of random writes. ¿Por qué no los dos?

Because it is in addition to your writes, not instead of them. That's what “ahead” points to.

Re: What Does a Database for SSDs Look Like?

#26

Post by formerly_proven »

crazygringo wrote: post #12
mrkeen wrote: post #2

> Design decisions like write-ahead logs, large page sizes, and buffering table writes in bulk were built around disks where I/O was SLOW, and where sequential I/O was order(s)-of-magnitude faster than random. Overall speed is irrelevant, what mattered was the relative speed difference between sequential and random access. And since there's still a massive difference between sequential and random access with SSDs, I…

Can you clarify? I thought a major benefit of SSDs is that there isn't any difference between sequential and random access. There's no physical head that needs to move. Edit: thank you for all the answers -- very educational, TIL!

SSDs have three block/page sizes:

- The access block size (LBA size). Either 512 bytes or 4096 bytes modulo DIF. Purely a logical abstraction.

- The programming page size. Something in the 4K-64K range. This is the granularity at which an erased block may be programmed with new data.

- The erase block size. Something in the 1-128 MiB range. This is the granularity at which data is erased from the flash chips.

SSDs always use some kind of journaled mapping to cope with the actual block size being roughly five orders of magnitude larger than the write API suggests. The FTL probably looks something like an LSM with some constant background compaction going on. If your writes are larger chunks, and your reads match those chunks, you would expect the FTL to perform better, because it can allocate writes contiguously and reads within the data structure have good locality as well. You can also expect for drives to further optimize sequential operations, just like the OS does.

(N.b. things are likely more complex, because controllers will likely stripe data with the FEC across NAND planes and chips for reliability, so the actual logical write size from the controller is probably not a single NAND page)

Re: What Does a Database for SSDs Look Like?

#27

Post by formerly_proven »

dist1ll wrote: post #22
PunchyHamster wrote: post #21

> WALs, and related low-level logging details, are critical for database systems that care deeply about durability on a single system. But the modern database isn’t like that: it doesn’t depend on commit-to-disk on a single system for its durability story. Commit-to-disk on a single system is both unnecessary (because we can replicate across storage on multiple systems) and inadequate (because we don’t want to lose w…

> "surely if I send request to 5 nodes some of that will land on disk in reasonably near future?" That would be asynchronous replication. But IIUC the author is instead advocating for a distributed log with synchronous quorum writes.

But we know this is not actually robust because storage and power failures tend to be correlated. The most recent Jepsen analysis again highlights that it's flawed thinking: https://jepsen.io/analyses/nats-2.12.1

Re: What Does a Database for SSDs Look Like?

#28

Post by lazide »

PunchyHamster wrote: post #21

> WALs, and related low-level logging details, are critical for database systems that care deeply about durability on a single system. But the modern database isn’t like that: it doesn’t depend on commit-to-disk on a single system for its durability story. Commit-to-disk on a single system is both unnecessary (because we can replicate across storage on multiple systems) and inadequate (because we don’t want to lose w…

Happens all the time (the ignores best practices because it’s convenient or ‘just because’ to do something different), literally everywhere including normal society.

Frankly, it’s shocking anything works at all.

Re: What Does a Database for SSDs Look Like?

#30

Post by formerly_proven »

londons_explore wrote: post #3

Median database workloads are probably doing writes of just a few bytes per transaction. Ie 'set last_login_time = now() where userid=12345'. Due to the interface between SSD and host OS being block based, you are forced to write a full 4k page. Which means you really still benefit from a write ahead log to batch together all those changes, at least up to page size, if not larger.

WALs are typically DB-page-level physical logs, and database page sizes are often larger than the I/O page size or the host page size.
Post reply on HN