Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Databases in 2025: A Year in Review

cs.cmu.edu

Post reply on HN
Print view View original

121–130 of 196 posts

Re: Databases in 2025: A Year in Review

#121

Post by mrtimo »

apavlo wrote: post #94
beders wrote: post #21

While the author mentions that he just doesn't have the time to look at all the databases, none of the reviews of the last few years mention immutable and/or bi-temporal databases. Which looks more like a blind spot to me honestly. This category of databases is just fantastic for industries like fintech. Two candidates are sticking out. https://xtdb.com/blog/launching-xtdb-v2 (2025) https://blog.datomic.com/2023/04/d…

> none of the reviews of the last few years mention immutable and/or bi-temporal databases. We hosted XTDB to give a tech talk five weeks ago: https://db.cs.cmu.edu/events/futuredata-reconstructing-histo... > Which looks more like a blind spot to me honestly. What do you want me to say about them? Just that they exist?

Nice work Andy. I'd love to hear about semantic layer developments in this space (e.g. Malloy etc.). Something to consider for the future. Thanks.

Re: Databases in 2025: A Year in Review

#122

Post by Tostino »

znpy wrote: post #107
danielfalbo wrote: post #4

Maybe off-topic but, If you're not familiar with the CMU DB Group you might want to check out their eccentric teaching style [1]. I absolutely love their gangsta intros like [2] and pre-lecture dj sets like [3]. I also remember a video where he was lecturing with someone sleeping on the floor in the background for some reason. I can't find that video right now. Not too sure about the context or Andy's biography, I'll…

I can't understand if their "intro to database systems" is an introductory (undergrad) level course or some advanced course (as in, introduction to database (internals)). Anyone willing to clarify this? I'm quite weak at database stuff, i'd love to find some undergrad-level proper course to learn and catch up.

It's the internals.

He is training up people to work on new features for existing databases, or build new ones.

Not application developers on how to use a database.

Knowing some of the internals can help application developers make better decisions when it comes to using databases though.

Re: Databases in 2025: A Year in Review

#123

Post by CyberDildonics »

odie5533 wrote: post #13
TekMol wrote: post #9

From my perspective on databases, two trends continued in 2025: 1: Moving everything to SQLite 2: Using mostly JSON fields Both started already a few years back and accelerated in 2025. SQLite is just so nice and easy to deal with, with its no-daemon, one-file-per-db and one-type-per value approach. And the JSON arrow functions make it a pleasure to work with flexible JSON data.

For as much talk as I see about SQLite, are people actually using it or does it just have good marketers?

It is fantastic software, have you ever used it?

Re: Databases in 2025: A Year in Review

#124

Post by dujuku »

jereze wrote: post #63

No mention of DuckDB? Surprising.

Also somewhat surprised. DuckDB traction is impressive and on par with vector databases in their early phases. I think there's a good chance it will earn an honorable mention next year if adoption holds and becomes more mainstream. But my impression is that it's still early in its adoption curve where only those "in the know" are using it as a niche tool. It also still has some quirks and foot-guns that need moderately knowledgeable systems people to operate (e.g. it will happily OOM your DB)

Re: Databases in 2025: A Year in Review

#125

Post by apavlo »

mrtimo wrote: post #121
apavlo wrote: post #94

Earlier quoted context omitted.

> none of the reviews of the last few years mention immutable and/or bi-temporal databases. We hosted XTDB to give a tech talk five weeks ago: https://db.cs.cmu.edu/events/futuredata-reconstructing-histo... > Which looks more like a blind spot to me honestly. What do you want me to say about them? Just that they exist?

Nice work Andy. I'd love to hear about semantic layer developments in this space (e.g. Malloy etc.). Something to consider for the future. Thanks.

> I'd love to hear about semantic layer developments in this space (e.g. Malloy etc.)

We also hosted Llyod to give a talk about Malloy in March 2025:

https://db.cs.cmu.edu/events/sql-death-malloy-a-modern-open-...

Re: Databases in 2025: A Year in Review

#126

Post by sanderjd »

dietr1ch wrote: post #120
sanderjd wrote: post #110

Earlier quoted context omitted.

Not all olap-like queries are for daily reporting. I agree that the basic architecture should be row order -> delay -> column order, but the question (in my mind) is balancing the length of that delay with the usefulness of column order queries for a given workload. I seem to keep running into workloads that do inserts very quickly and then batch reads on a slower cadence (either in lockstep with the writes, or concu…

> the question (in my mind) is balancing the length of that delay with the usefulness of column order queries for a given workload. I seem to keep running into workloads that do inserts very quickly and then batch reads on a slower cadence (either in lockstep with the writes, or concurrently) but not on the extremely slow cadence seen in the typical olap reporting type flow. Essentially, building up state and then qu…

Yes! We're definitely talking about the same thing here! Definitely not thinking of consistent writes to both views.

Now that you said this about watermarks, I realize that this is definitely the same idea as streaming systems like flink (which is where I'm familiar with watermarks from), but my use cases are smaller data and I'm looking for lower latency than distributed systems like that. I'm interested in delays that are on the order of double to triple digit milliseconds, rather than 15 minutes. (But also not microseconds.)

I definitely agree that it's difficult to justify building this, which is why I keep looking for a system that already exists :)

Re: Databases in 2025: A Year in Review

#127

Post by quotemstr »

Why does "database" is surveys like this not include DuckDB and SQLite, which are great [1] embedded answers to Clickhouse and PostgreSQL. Both are excellent and useful databases; DuckDB's reasonable syntax, fast vectorized everything, and support for ingesting the hairiest of data as in-DB ETL make me reach for it first these days, at least for the things I want to do.

Why is it that in "I'm a serious database person" circles, the popular embedded databases don't count?

[1] Yes, I know it's not an exact comparison.

Re: Databases in 2025: A Year in Review

#128

Post by quotemstr »

TekMol wrote: post #9

From my perspective on databases, two trends continued in 2025: 1: Moving everything to SQLite 2: Using mostly JSON fields Both started already a few years back and accelerated in 2025. SQLite is just so nice and easy to deal with, with its no-daemon, one-file-per-db and one-type-per value approach. And the JSON arrow functions make it a pleasure to work with flexible JSON data.

FWIW (and this is IMHO of course) DuckDB makes working with random JSON much nicer than SQLite, not least because I can extract JSON fields to dense columnar representations and do it in a deterministic, repeatable way.

The only thing I want out of DuckDB core at this point is support for overriding the columnar storage representation for certain structs. Right now, DuckDB decomposes structs into fields and stores each field in a column. I'd like to be able to say "no, please, pre-materialize this tuple subset and store this struct in an internal BLOB or something".

Re: Databases in 2025: A Year in Review

#129

Post by trevor-e »

TekMol wrote: post #86
DrBazza wrote: post #74

Earlier quoted context omitted.

From my perspective - do you even need a database? SQLite is kind-of the middle ground between a full fat database, and 'writing your own object storage'. To put it another way, it provides 'regularised' object access API, rather than, say, a variant of types in a vector that you use filter or map over.

If I would write my own data storage I would re-implement SQLite. Why would I want to do that?

Not sure if this is quite what you are getting at, but the SQLite folks even mention this as a great use-case: https://www.sqlite.org/appfileformat.html

Re: Databases in 2025: A Year in Review

#130

Post by quotemstr »

beders wrote: post #21

While the author mentions that he just doesn't have the time to look at all the databases, none of the reviews of the last few years mention immutable and/or bi-temporal databases. Which looks more like a blind spot to me honestly. This category of databases is just fantastic for industries like fintech. Two candidates are sticking out. https://xtdb.com/blog/launching-xtdb-v2 (2025) https://blog.datomic.com/2023/04/d…

XTDB addresses a real use-case. I wish we invested more in time series databases actually: there's a ton of potential in a GIS-style database, but 1D and oriented around regions on the timeline, not shapes in space.

That said, it's kind of frustrating that XTDB has to be its own top-level database instead of a storage engine or plugin for another. XTDB's core competence is its approach to temporal row tagging and querying. What part of this core competence requires a new SQL parser?

I get that the XTDB people don't want to expose their feature set as a bunch of awkward table-valued functions or whatever. Ideally, DB plugins for Postgres, SQLite, DuckDB, whatever would be able to extend the SQL grammar itself (which isn't that hard if you structure a PEG parser right) and expose new capabilities in an ergonomic way so we don't end up with a world of custom database-verticals each built around one neat idea and duplicating the rest.

I'd love to see databases built out of reusable lego blocks to a greater extent than today. Why doesn't Calcite get more love? Is it the Java smell?

Post reply on HN