Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Does a Database for SSDs Look Like?

brooker.co.za

Post reply on HN
Print view View original

31–40 of 127 posts

Re: What Does a Database for SSDs Look Like?

#31

Post by sreekanth850 »

PunchyHamster wrote: post #21

> WALs, and related low-level logging details, are critical for database systems that care deeply about durability on a single system. But the modern database isn’t like that: it doesn’t depend on commit-to-disk on a single system for its durability story. Commit-to-disk on a single system is both unnecessary (because we can replicate across storage on multiple systems) and inadequate (because we don’t want to lose w…

The biggest lie we’ve been told is that databases require global consistency and a global clock. Traditional databases are still operating with Newtonian assumptions about absolute time, while the real world moves according to Einstein’s relativistic theory, where time is local and relative. You dont need global order, you dont need global clock.

Re: What Does a Database for SSDs Look Like?

#32

Post by _bohm »

Sesse__ wrote: post #25
throw0101a wrote: post #23

Earlier quoted context omitted.

> A write-ahead log isn't a performance tool to batch changes, it's a tool to get durability of random writes. ¿Por qué no los dos?

Because it is in addition to your writes, not instead of them. That's what “ahead” points to.

The actual writes don’t need to be persisted on transaction commit, only the WAL. In most DBs the actual writes won’t be persisted until the written page is evicted from the page cache. In this sense, writing WAL generally does provide better perf than synchronously doing a random page write

Re: What Does a Database for SSDs Look Like?

#33

Post by lazide »

crazygringo wrote: post #12
mrkeen wrote: post #2

> Design decisions like write-ahead logs, large page sizes, and buffering table writes in bulk were built around disks where I/O was SLOW, and where sequential I/O was order(s)-of-magnitude faster than random. Overall speed is irrelevant, what mattered was the relative speed difference between sequential and random access. And since there's still a massive difference between sequential and random access with SSDs, I…

Can you clarify? I thought a major benefit of SSDs is that there isn't any difference between sequential and random access. There's no physical head that needs to move. Edit: thank you for all the answers -- very educational, TIL!

It depends on the side of read - most SSD’s have internal block sizes much larger than a typical (actual) random read, so they internally have to do a lot more work for a given byte of output in a random read situation than they would in a sequential one.

Most filesystems read in 4K chunks (or sometimes even worse, 512 byes), and internally the actual block is often multiple MB in size, so this internal read multiplication is a big factor in performance in those cases.

Note the only real difference between a random read and a sequential one is the size of the read in one sequence before it switches location - is it 4K? 16mb? 2G?

Re: What Does a Database for SSDs Look Like?

#34

Post by szundi »

threeducks wrote: post #15
crazygringo wrote: post #12

Earlier quoted context omitted.

Can you clarify? I thought a major benefit of SSDs is that there isn't any difference between sequential and random access. There's no physical head that needs to move. Edit: thank you for all the answers -- very educational, TIL!

Lets take the Samsung 9100 Pro M.2 as an example. It has a sequential read rate of ~6700 MB/s and a 4k random read rate of ~80 MB/s: https://i.imgur.com/t5scCa3.png https://ssd.userbenchmark.com/ (click on the orange double arrow to view additional columns) That is a latency of about 50 µs for a random read, compared to 4-5 ms latency for HDDs.

[dead]

Re: What Does a Database for SSDs Look Like?

#35

Post by nly »

sreekanth850 wrote: post #24

Unpopular Opinion: Database were designed for 1980-90 mechanics, the only thing that never innovates is DB. It still use BTree/LSM tree that were optimized for spinning disc. Inefficiency is masked by hardware innovation and speed (Moores Law).

Optimising hardware to run existing software is how you sell your hardware.

The amount of performance you can extract from a modern CPU if you really start optimising cache access patterns is astounding

High performance networking is another area like this. High performance NICs still go to great lengths to provide a BSD socket experience to devs. You can still get 80-90% of the performance advantages of kernel bypass without abandoning that model.

Re: What Does a Database for SSDs Look Like?

#36

Post by pmontra »

A tangent:

> Companies are global, businesses are 24/7

Only a few companies are global, so only a few of them should optimize for those kind of workload. However maybe every startup in SV must aim to becoming global, so probably that's what most of them must optimize for, even the ones that eventually fail to get traction.

24/7 is different because even the customers of local companies, even B2B ones, mighty feel like doing some work at midnight once in a while. They'll be disappointed to find the server down.

Re: What Does a Database for SSDs Look Like?

#37

Post by lazide »

zokier wrote: post #5

Author could have started by surveying current state of art instead of just falsely assuming that DB devs have just been resting on the laurels for past decades. If you want to see (relational) DB for SSD just check out stuff like myrocks on zenfs+; it's pretty impressive stuff.

But then how would they have anything to do?

Re: What Does a Database for SSDs Look Like?

#39

Post by dotancohen »

sreekanth850 wrote: post #31
PunchyHamster wrote: post #21

> WALs, and related low-level logging details, are critical for database systems that care deeply about durability on a single system. But the modern database isn’t like that: it doesn’t depend on commit-to-disk on a single system for its durability story. Commit-to-disk on a single system is both unnecessary (because we can replicate across storage on multiple systems) and inadequate (because we don’t want to lose w…

The biggest lie we’ve been told is that databases require global consistency and a global clock. Traditional databases are still operating with Newtonian assumptions about absolute time, while the real world moves according to Einstein’s relativistic theory, where time is local and relative. You dont need global order, you dont need global clock.

That's why we use UUIDv7 primary keys. Relativity be damned, our replication strategy does not depend upon the timestamp factor.

Re: What Does a Database for SSDs Look Like?

#40

Post by gethly »

nly wrote: post #35
sreekanth850 wrote: post #24

Unpopular Opinion: Database were designed for 1980-90 mechanics, the only thing that never innovates is DB. It still use BTree/LSM tree that were optimized for spinning disc. Inefficiency is masked by hardware innovation and speed (Moores Law).

Optimising hardware to run existing software is how you sell your hardware. The amount of performance you can extract from a modern CPU if you really start optimising cache access patterns is astounding High performance networking is another area like this. High performance NICs still go to great lengths to provide a BSD socket experience to devs. You can still get 80-90% of the performance advantages of kernel bypas…

> The amount of performance you can extract from a modern CPU if you really start optimising cache access patterns is astounding

I think this was one, and I want to emphasise this, of the main points behind Odin programming language.

Post reply on HN