Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Databases in 2025: A Year in Review

cs.cmu.edu

Post reply on HN
Print view View original

181–190 of 196 posts

Re: Databases in 2025: A Year in Review

#181

Post by SahAssar »

radarroark wrote: post #65
beders wrote: post #21

While the author mentions that he just doesn't have the time to look at all the databases, none of the reviews of the last few years mention immutable and/or bi-temporal databases. Which looks more like a blind spot to me honestly. This category of databases is just fantastic for industries like fintech. Two candidates are sticking out. https://xtdb.com/blog/launching-xtdb-v2 (2025) https://blog.datomic.com/2023/04/d…

People are slow to realize the benefit of immutable databases, but it is happening. It's not just auditability; immutable databases can also allow concurrent reads while writes are happening, fast cloning of data structures, and fast undo of transactions. The ones you mentioned are large backend databases, but I'm working on an "immutable SQLite"...a single file immutable database that is embedded and works as a libr…

How do you deal with deletion requirements in a immutable database? Like how do you delete personal data when requested?

Re: Databases in 2025: A Year in Review

#182

Post by senderista »

nijave wrote: post #158
throw0101d wrote: post #102

Regarding distributed(-ish) Postgres, does anyone know if something like My/MariaSQL's multi-master Galera† is around for Pg: > MariaDB Galera Cluster provides a synchronous replication system that uses an approach often called eager replication. In this model, nodes in a cluster synchronize with all other nodes by applying replicated updates as a single transaction. This means that when a transaction COMMITs, all no…

Citus, sort of Cockroach For HA, Patroni, stolon, CNPG Multimaster doesn't necessarily buy you availability. Usually it trades performance and potentially uptime for data integrity.

And DSQL if you use a very loose definition of "Postgres"

Re: Databases in 2025: A Year in Review

#183

Post by mwarkentin »

SahAssar wrote: post #181
radarroark wrote: post #65

Earlier quoted context omitted.

People are slow to realize the benefit of immutable databases, but it is happening. It's not just auditability; immutable databases can also allow concurrent reads while writes are happening, fast cloning of data structures, and fast undo of transactions. The ones you mentioned are large backend databases, but I'm working on an "immutable SQLite"...a single file immutable database that is embedded and works as a libr…

How do you deal with deletion requirements in a immutable database? Like how do you delete personal data when requested?

Crypto shredding?

Re: Databases in 2025: A Year in Review

#184

Post by andersmurphy »

cmrdporcupine wrote: post #174
andersmurphy wrote: post #161

Earlier quoted context omitted.

Why? Sqlite and LMDB make fantastic use of it. For anyone doing a single writer db it's a no brainer. It does so much for you and it does it very well. All the things you don't have to implement because it does it for you: - Reading the data from disk - Concurrency between different threads reading the same data - Caching and buffer management - Eviction of pages from memory - Playing nice with other processes in the…

The strongest argument as far as I can see it is... the problem is you now lose control over all those things. It's a black box with effectively no knobs. Anyways, read for yourself, Pavlo & Leis get into it in detail, and there's benchmarks: https://db.cs.cmu.edu/papers/2022/cidr2022-p13-crotty.pdf https://db.cs.cmu.edu/mmap-cidr2022/

What am I missing? The transactional safety problem (the bulk of the paper) is solved simply with a single writer. Which is where you want to be anyway for efficient batching throughput (and isolation).

The other concerns seem to imply there are no other programs running on the same machine as the database. The minute that's not true (is it ever true?). Then OS will do a better job (as seen with LMDB etc).

I think it's telling that the paper focuses on mongoDB not LMDB.

Re: Databases in 2025: A Year in Review

#185

Post by menaerus »

Sesse__ wrote: post #176
menaerus wrote: post #167

Earlier quoted context omitted.

Wouldn't Oracle need those 80%+ devs if they wanted to shift their efforts into Heatwave? That percentage sounds too huge to me and if true I believe they won't be making any larger investments into Heatwave neither. There's several core teams in MySQL and if you let those people go ... I don't know, I am not sure what to make out of it but that Oracle is completely moving away from MySQL as a strategic component of…

> Wouldn't Oracle need those 80%+ devs if they wanted to shift their efforts into Heatwave? They would, so Heatwave is also going to suffer over this.

So, AI ate the cake ... I always thought that the investment that Oracle needs to make for MySQL is peanuts compared to the Oracle's total revenue and the revenue MySQL is generating. Perhaps the latter is not so true anymore.

Re: Databases in 2025: A Year in Review

#186

Post by hyc_symas »

mtndew4brkfst wrote: post #155
andersmurphy wrote: post #142

With a trend towards immutable single writer databases MMAP seems like a massive win.

Andy is very critical of using mmap in database implementations.

Andy's critiques are only valid on dedicated database servers.

https://www.symas.com/post/are-you-sure-you-want-to-use-mmap...

LMDB uses mmap and Andy recommends LMDB, in the very article this thread is about.

Re: Databases in 2025: A Year in Review

#187

Post by hyc_symas »

andersmurphy wrote: post #161
mtndew4brkfst wrote: post #155

Earlier quoted context omitted.

Andy is very critical of using mmap in database implementations.

Why? Sqlite and LMDB make fantastic use of it. For anyone doing a single writer db it's a no brainer. It does so much for you and it does it very well. All the things you don't have to implement because it does it for you: - Reading the data from disk - Concurrency between different threads reading the same data - Caching and buffer management - Eviction of pages from memory - Playing nice with other processes in the…

Fun footnote: SQLite only got on board with mmap after I demonstrated how slow their code was without it. I.e., getting a 22x speedup by replacing SQLite's btree code with LMDB https://github.com/LMDB/sqlightning

Re: Databases in 2025: A Year in Review

#188

Post by radarroark »

SahAssar wrote: post #181
radarroark wrote: post #65

Earlier quoted context omitted.

People are slow to realize the benefit of immutable databases, but it is happening. It's not just auditability; immutable databases can also allow concurrent reads while writes are happening, fast cloning of data structures, and fast undo of transactions. The ones you mentioned are large backend databases, but I'm working on an "immutable SQLite"...a single file immutable database that is embedded and works as a libr…

How do you deal with deletion requirements in a immutable database? Like how do you delete personal data when requested?

The fastest approach is to just zero out the data. Alternatively you can rebuild the entire database while preserving only the data accessible in the latest copy of the db (kinda similar to SQLite's VACUUM command).

Re: Databases in 2025: A Year in Review

#190

Post by ugamarkj »

Interesting article and I've enjoyed reading all the comments. The focus on Postgres is fascinating to me. From an analytics database perspective, we've had great success with Exasol, which isn't so well known in the US. It is very low overhead (no index management) and extremely fast and scalable. They have a free version as well as a licensed MPP version -- cloud hosted or on-prem. It is a blank slate, but it can do all the things.
Post reply on HN