Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Does a Database for SSDs Look Like?

brooker.co.za

Post reply on HN
Print view View original

71–80 of 127 posts

Re: What Does a Database for SSDs Look Like?

#71

Post by evanelias »

saghm wrote: post #64
wpietri wrote: post #44

I should add that the bond between relational databases and spinning rust goes back further. My dad, who started working as a programmer in the 60s with just magtape as storage, talked about the early era of disks as a big step forward but requiring a lot of detailed work to decide where to put the data and how to find it again. For him, databases were a solution to the problems that that disks created for programmer…

I'm not sure I totally understand the timeline you're describing, but my understanding is that relational databases themselves were only invented in the 1970s. Is your reference to the 60s just giving context for when he started but before this link happened (with the idea that the problems predated the solution)?

Non-relational databases existed in the 60s, and many programmers who worked in the 60s presumably continued working into the 70s, so either way I don't see any problems with the timeline GP mentions.

Re: What Does a Database for SSDs Look Like?

#72

Post by evanelias »

pmontra wrote: post #36

A tangent: > Companies are global, businesses are 24/7 Only a few companies are global, so only a few of them should optimize for those kind of workload. However maybe every startup in SV must aim to becoming global, so probably that's what most of them must optimize for, even the ones that eventually fail to get traction. 24/7 is different because even the customers of local companies, even B2B ones, mighty feel lik…

> Only a few companies are global, so only a few of them should optimize for those kind of workload

A massive number of companies have global customers, regardless of where the company itself has employees.

For example my b2b business is relatively tiny, yet my customer base spans four continents. Or six continents if you count free users!

Re: What Does a Database for SSDs Look Like?

#73

Post by ayende »

amluto wrote: post #52
cornholio wrote: post #49

You know you need to be careful when an Amazon engineer will argue for a database architecture that fully leverages (and makes you dependent of) the strengths of their employer's product. In particular: > Commit-to-disk on a single system is both unnecessary (because we can replicate across storage on multiple systems) and inadequate (because we don’t want to lose writes even if a single system fails). This is surely…

Skipping flushing the local disk seems rather silly to me: - A modern high end SSD commits faster than the one way time to anywhere much farther than a few miles away. (Do the math. A few tens of microseconds specified write latency is pretty common. NVDIMMs (a sadly dying technology) can do even better. The speed of light is only so fast. - Unfortunate local correlated failures happen. IMO it’s quite nice to be able…

Committing to NVMe drive properly is really costly. I'm talking using O_DIRECT | OSYNC or fsync here. Can be in the order of whole milliseconds, easily. And it is much worse if you are using cloud systems.

Re: What Does a Database for SSDs Look Like?

#74

Post by ayende »

PunchyHamster wrote: post #69
sreekanth850 wrote: post #31

Earlier quoted context omitted.

The biggest lie we’ve been told is that databases require global consistency and a global clock. Traditional databases are still operating with Newtonian assumptions about absolute time, while the real world moves according to Einstein’s relativistic theory, where time is local and relative. You dont need global order, you dont need global clock.

Till the financial controller shows up at the very least. Also even if not required makes reasoning about how systems work a hell lot easier. So for vast majority that doesn't need massive throughtputs sacrificing some speed for easier to understand consistency model is worthy tradeoff

All financial systems don't care about time.

Prety much all financial transactions are settled with a given date, not instantly. Go sell some stocks, it takes 2 days to actually settle. (May be hidden by your provider, but that how it works).

For that matter, the ultimate in BASE for financial transactions is the humble check.

That is a great example of "money out" that will only be settled at some time in the future.

There is a reason there is this notion of a "business day" and re-processing transactions that arrived out of order.

Re: What Does a Database for SSDs Look Like?

#75

Post by evanelias »

nextaccountic wrote: post #47
zokier wrote: post #5

Author could have started by surveying current state of art instead of just falsely assuming that DB devs have just been resting on the laurels for past decades. If you want to see (relational) DB for SSD just check out stuff like myrocks on zenfs+; it's pretty impressive stuff.

> myrocks anything like this, but for postgres? actually, is it even possible to write a new db engine for postgres? like mysql has innodb, myisam, etc

Postgres's strategy has traditionally been to focus on pluggable indexing methods which can be provided by extensions, rather than completely replacing the core heap storage engine design for tables.

That said, there are a few alternative storage engines for Postgres, such as OrioleDB. However due to limitations in Postgres's storage engine API, you need to patch Postgres to be able to use OrioleDB.

MySQL instead focused on pluggable storage engines from the get-go. That has had major pros and cons over the years. On the one hand, MyISAM is awful, so pluggable engines (specifically InnoDB) are the only thing that "saved" MySQL as the web ecosystem matured. It also nicely forced logical replication to be an early design requirement, since with a multi-engine design you need a logical abstraction instead of a physical one.

But on the other hand, pluggable storage introduces a lot of extra internal complexity, which has arguably been quite detrimental to the software's evolution. For example: which layer implements transactions, foreign keys, partitioning, internal state (data dictionary, users/grants, replication state tracking, etc). Often the answer is that both the server layer and the storage engine layer would ideally need to care about these concerns, meaning a fully separated abstraction between layers isn't possible. Or think of things like transactional DDL, which is prohibitively complex in MySQL's design so it probably won't ever happen.

Re: What Does a Database for SSDs Look Like?

#76

Post by ffsm8 »

ayende wrote: post #73
amluto wrote: post #52

Earlier quoted context omitted.

Skipping flushing the local disk seems rather silly to me: - A modern high end SSD commits faster than the one way time to anywhere much farther than a few miles away. (Do the math. A few tens of microseconds specified write latency is pretty common. NVDIMMs (a sadly dying technology) can do even better. The speed of light is only so fast. - Unfortunate local correlated failures happen. IMO it’s quite nice to be able…

Committing to NVMe drive properly is really costly. I'm talking using O_DIRECT | OSYNC or fsync here. Can be in the order of whole milliseconds, easily. And it is much worse if you are using cloud systems.

Isn't that why a WAL exists, so you didn't actually need to do that with eg postgres and other rdbms?

Re: What Does a Database for SSDs Look Like?

#77

Post by cmrdporcupine »

sreekanth850 wrote: post #24

Unpopular Opinion: Database were designed for 1980-90 mechanics, the only thing that never innovates is DB. It still use BTree/LSM tree that were optimized for spinning disc. Inefficiency is masked by hardware innovation and speed (Moores Law).

There's plenty of innovation in DB storage tech, but the hardware interface itself is still page-based.

It turns out that btrees are still efficient for this work. At least until the hardware vendors deign to give us an interface to SSD that looks more like RAM.

Reading over https://www.cs.cit.tum.de/dis/research/leanstore/ and associated papers and follow up work is recommended.

In the meantime with RAM prices sky rocketing, work and research in buffer & page management for greater-than-main-memory-sized DBs is set to be Hot Stuff again.

I like working in this area.

Re: What Does a Database for SSDs Look Like?

#78

Post by cmrdporcupine »

sscdotopen wrote: post #29

Umbra: A Disk-Based System with In-Memory Performance, CIDR'20 https://db.in.tum.de/~freitag/papers/p29-neumann-cidr20.pdf

Yep, and the work on https://www.cs.cit.tum.de/dis/research/leanstore/ that preceded it.

And CedarDB https://cedardb.com/ the more commercialized product that is following up on some of this research, including employing many of the key researchers.

Re: What Does a Database for SSDs Look Like?

#79

Post by evanelias »

koverstreet wrote: post #61
zokier wrote: post #5

Author could have started by surveying current state of art instead of just falsely assuming that DB devs have just been resting on the laurels for past decades. If you want to see (relational) DB for SSD just check out stuff like myrocks on zenfs+; it's pretty impressive stuff.

bcachefs's btree still beats the pants off of the entire rocksdb lineage :)

Rocksdb / myrocks is heavily used by Meta at extremely massive scale. For sake of comparison, what's the largest real-world production deployment of bcachefs?

Re: What Does a Database for SSDs Look Like?

#80

Post by mgerdts »

threeducks wrote: post #15
crazygringo wrote: post #12

Earlier quoted context omitted.

Can you clarify? I thought a major benefit of SSDs is that there isn't any difference between sequential and random access. There's no physical head that needs to move. Edit: thank you for all the answers -- very educational, TIL!

Lets take the Samsung 9100 Pro M.2 as an example. It has a sequential read rate of ~6700 MB/s and a 4k random read rate of ~80 MB/s: https://i.imgur.com/t5scCa3.png https://ssd.userbenchmark.com/ (click on the orange double arrow to view additional columns) That is a latency of about 50 µs for a random read, compared to 4-5 ms latency for HDDs.

Datacenter storage will generally not be using M.2 client drives. They employ optimizations that win many benchmarks but sacrifice on consistency multiple dimensions (power loss protection, write performance degrades as they fill, perhaps others).

With SSDs, the write pattern is very important to read performance.

Datacenter and enterprise class drives tend to have a maximum transfer size of 128k, which is seemingly the NAND block size. A block is the thing that needs to be erased before rewriting.

Most drives seem to have an indirection unit size of 4k. If a write is not a multiple of the IU size or not aligned, the drive will have to do a read-modify-write. It is the IU size that is most relevant to filesystem block size.

If a small write happens atop a block that was fully written with one write, a read of that LBA range will lead to at least two NAND reads until garbage collection fixes it.

If all writes are done such that they are 128k aligned, sequential reads will be optimal and with sufficient queue depth random 128k reads may match sequential read speed. Depending on the drive, sequential reads may retain an edge due to the drive’s read ahead. My own benchmarks of gen4 U.2 drives generally backs up these statements.

At these speeds, the OS or app performing buffered reads may lead to reduced speed because cache management becomes relatively expensive. Testing should be done with direct IO using libaio or similar.

Post reply on HN