A couple points I would make with respect to the article: - In-memory databases offer few advantages over a disk-backed database with a properly designed I/O scheduler. In-memory databases are generally only faster if the disk-backed database uses mmap() for cache replacement or similarly terrible I/O scheduling. The big advantage of in-memory databases is that you avoid the enormously complicated implementation task…
Can you give some refs to make these statements more real? I'm trying to understand whether an in-memory db would be faster than postgres, for instance. What's the postgres I/O scheduler, and is it good? Are there benchmarks somewhere showing the difference? Why does the I/O scheduler make a difference for in-memory vs. disk databases? Is this a subsystem that caches the database in memory? Are you saying that, with…
With proper cache and I/O scheduler, the same workload will fit in memory, so the only way the disk gets in the way is if the I/O scheduling does something suboptimal with respect to writes (which happens a lot with the kernel caching behavior).
Modern database servers typically have more disk bandwidth than network bandwidth and therefore most write workloads should be able to go through storage at the network's wire speed, at least in theory. In practice, I/O scheduling behavior from memory to storage tends to be bursty or poorly timed. Consequently, the instantaneous I/O bandwidth requirements can exceed the effective disk bandwidth for brief periods and performance degrades.
A really good database I/O scheduler, cache, and execution engine work together to basically makes sure that peak bandwidth demands to the disk subsystem are never much worse than the network wire speed. Achieving this is not trivial and requires a lot of clever dynamic resource optimization but many sophisticated database engines implement this to some degree or another.