Rich Hickey: Deconstructing the Database

I am not saying "Datomic is stupid" or anything so simple; I'm saying I was "disappointed" in this talk because it motivated Datomic against a strawman that mischaracterized the actual problems that people using "traditional databases" have sufficiently that it was no longer possible to determine what was actually being claimed as an advantage.

I realize that to many people it is impossible to dislike a presentation of something without disliking the thing being presented, the person making the presentation, and the entire ideology behind the presentation, but that is a horrible thing to assume and is unlikely to ever be the case to such a simple extreme.

I will even go so far as to say that watching this talk seems to be doing a disservice to many people on the road to doing them a legitimate service: some of the people commenting on this thread (or previous ones on HN about similar talks and articles about Datomic) actually do/did not realize that "traditional databases" can even do this at a transaction level, as the argument in the talk downright claims they can't.

The result is that when I bring up that you actually get even some of these advantages with off-the-shelf copies of PostgreSQL, I get comments of the form "I had no idea one could get a consistent read view across multiple queries within a transaction using most sql databases. That does poke a hole in a major benefit that I thought was unique to datomic, great to know!"; that can only happen when there is some serious misinformation (accidentally) being presented.

Now, does that mean that Datomic is something no one should use, and that it doesn't put things together in a really nice way, and that it doesn't have a single thing in it that is innovative, or that Rich is wasting his time working on it? No: certainly it does not. I did not claim that. I can't even claim that, as I gave up on the talk after the first half so I could spend my time attempting to clarify some of the things said in the first half that were confusing people.

Thanks for the reply. In the talk, I missed the point that transaction functions are able to implement "optimistic CAS" (which I assume is the same as what is more traditionally but inaccurately called "optimistic locking"). To clarify, though: Wouldn't precondition testing have to be done /inside/ the transaction, not before, in order to be effective?

That's a nice model, I suppose, but doesn't strike me as particularly novel. As has been pointed out elsewhere, the idea that read-only operations can avoid creating a transaction is not particularly important per se. What's more important is avoiding locking, and this is already available in several DBMSes. Indeed, in the sense that reads are consistent "as of" a certain point in time, Datomic really does have a notion of a read transaction. I saw in another of your replies here that you consider these to be more flexible than transactions because they are in effect immutable snapshots of the database state, and independent of a single process. I guess for some applications that could be important, but for many applications the process independence is irrelevant (perhaps it helps with scaling a middle tier?), and the eventual storage cost of true immutability will become intractable (although perhaps you'll eventually have ways to release state prior to a point in time). Coordinating different parts of code within a single process over a specific span of time doesn't strike me as any easier with Datomic's model than with traditional transactions.

This talk just didn't focus on the kinds of data modeling problems that are important to me and which I consider difficult, which mostly have to do with maintainability of the data model - except, possibly, for the ease of dealing with temporal queries. And I share others' unease with what seemed to be inaccurate characterizations of current DBMS implementations.