Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Datomic is Free

blog.datomic.com

Post reply on HN
Print view View original

171–180 of 436 posts

Re: Datomic is Free

#171

Post by brundolf »

augustl wrote: post #11

Datomic's is perfect for probably 90% of small-ish backoffice systems that never has to be web scale (i.e. most of what I do at work). Writing in a single thread removes a whole host of problems in understanding (and implementing) how data changes over time. (And a busy MVCC sql db spends 75% of its time doing coordination, not actual writes, so a single thread applying a queue of transactions in sequence can be fast…

Something I've been curious about: how well (or badly) would it scale to do something similar on a normal relational DB (say, Postgres)?

You could have one or more append-only tables that store events/transactions/whatever you want to call them, and then materialized-views (or whatever) which gather that history into a "current state" of "entities", as needed

If eventual-consistency is acceptable, it seems like you could aggressively cache and/or distribute reads. Maybe you could even do clever stuff like recomputing state only from the last event you had, instead of from scratch every time

How bad of an idea is this?

Re: Datomic is Free

#172

Post by hlship »

oh_sigh wrote: post #21
mqus wrote: post #15

Earlier quoted context omitted.

> perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. Doesn't this mean, that, as soon as I (somehow) get hold of the source code, I can distribute it as I want?

Probably not, because the source is not a derivative. Having said that, if you decompile the binary I bet you could distribute that source.

That would be very ugly source, as Datomic is written in Clojure and AOT compiled to Java bytecode. Due to the architecture of Clojure (especially, the use of macros) it is not exactly possible to work backwards from JVM bytecode to anything that looks like the original source code. It's not like Java where a clever decompiler can exploit output patterns generated by the Java compiler to make reasonable guesses at the structure of the source code.

But this is all besides the point; Datomic is now free (as in beer) with a great license (Apache 2.0). You can use this amazing tool for free, and you have as much need to look at the source to do so as you might need to look at PostgreSQL's source.

Some of us have been hoping for this day since Datomic was first announced, but even as an insider (I have been working at NuBank NA for less than a year) I was stunned at the speed with which this decision was made and implemented.

Re: Datomic is Free

#173

Post by augustl »

brundolf wrote: post #171
augustl wrote: post #11

Datomic's is perfect for probably 90% of small-ish backoffice systems that never has to be web scale (i.e. most of what I do at work). Writing in a single thread removes a whole host of problems in understanding (and implementing) how data changes over time. (And a busy MVCC sql db spends 75% of its time doing coordination, not actual writes, so a single thread applying a queue of transactions in sequence can be fast…

Something I've been curious about: how well (or badly) would it scale to do something similar on a normal relational DB (say, Postgres)? You could have one or more append-only tables that store events/transactions/whatever you want to call them, and then materialized-views (or whatever) which gather that history into a "current state" of "entities", as needed If eventual-consistency is acceptable, it seems like you c…

Datomic already sort of does this :) You configure a storage backend (Datomic does not write to disk directly) which can be dynamodb, riak, or any JDBC database including postgres. You won't get readable data in PG though, as Datomic stores opaque compressed chunks in a key/value structure. The chunks are adressable via the small handful of built-in indexes that Datomic provides for querying, and the indexes are covering, i.e. data is duplicated for each index.

Re: Datomic is Free

#174

Post by gleenn »

samuell wrote: post #148

Question to people having used Datomic: Based on experience with Prolog, I always thought using Datalog in a database like Datomic would mean being able to model your data model using stored queries as a very expressive way of creating "classes". And that by modeling your data model using nested such queries, you alleviate the need for an ORM, and all the boilerplate and duplication of defining classes both in SQL an…

Clojure in general is all about passing around little maps of data and in particular not using OO to model. So Datomic naturally continues that by returning maps of nested structures to represent your query results and does side-step the ORM completely.

Re: Datomic is Free

#175

Post by hlship »

ddellacosta wrote: post #71
endisneigh wrote: post #4

The price is certainly right, but has anyone used this in production? What was your experience like?

My personal experience was using Datomic backed by DynamoDB, at the second Clojure company I worked at. In particular I remember feeling like it was hard to anticipate and understand its performance characteristics in particular, and how indices can be leveraged effectively. Maybe if we had chosen Postgres as a backing store that would have been better? I dunno. Using it was pretty nice at the scale of a small startu…

Although it is true that "time traveling" queries are relatively rare for production needs, the basic architecture supports things that many applications really need:

- It is possible to make queries against the database PLUS additional data not yet added, that is, "what if" queries

- Having a stable database-as-value is really useful for paginating results; you don't have to worry about new values being inserted into your results during execution, the way you do with traditional databases no longer how long (minutes, hours, even days) you take to traverse the data

- Reified transactions makes it possible to store extra data with each transaction, trivially, such as who made the update and why

- Immutability is amazing for caching at all layers

Re: Datomic is Free

#176

Post by augustl »

slaymaker1907 wrote: post #169
alecco wrote: post #92

Earlier quoted context omitted.

Just the temporal properties alone make it very useful for anything where it matters like billing, finance, inventory. Else you are in views/schema/indexing hell to do it on top of SQL. There is some SQL temporal support but it's not great and varies a lot. Also since it's not native to the storage it has a lot of complexity issues under the rug making it not great. Many financial systems use Event Sourcing (OOP + OR…

The temporal support seems handy, but time is still going to be really tricky for financial systems. Datomic only covers what the physical state of the database was at a particular time, but there's also the effective legal time (maybe a payment was dated a day before the system actually processed it) as well as requirements to remove data after a period of time (including point in time stuff).

Indeed, it depends a lot on the domain. Datomic only has "technical" database time, and doesn't have any built-in way of modelling domain time. You can set the transaction timestamp manually when you write, but you can't set it to be earlier than the latest transaction that was committed. So, if you want your domain modelling to piggyback on Datomic time travelling, you can only do things like delaying writes for, say, an hour, and hope you have all the data by the time you commit to db.

Re: Datomic is Free

#177

Post by brundolf »

augustl wrote: post #173
brundolf wrote: post #171

Earlier quoted context omitted.

Something I've been curious about: how well (or badly) would it scale to do something similar on a normal relational DB (say, Postgres)? You could have one or more append-only tables that store events/transactions/whatever you want to call them, and then materialized-views (or whatever) which gather that history into a "current state" of "entities", as needed If eventual-consistency is acceptable, it seems like you c…

Datomic already sort of does this :) You configure a storage backend (Datomic does not write to disk directly) which can be dynamodb, riak, or any JDBC database including postgres. You won't get readable data in PG though, as Datomic stores opaque compressed chunks in a key/value structure. The chunks are adressable via the small handful of built-in indexes that Datomic provides for querying, and the indexes are cove…

Interesting! I assumed Datomic was entirely custom

Now I'm even more curious if you could skip Datomic and just do something like this directly with a relational DB in production

Re: Datomic is Free

#178

Post by croes »

remram wrote: post #154
elteto wrote: post #151

Earlier quoted context omitted.

They didn’t open source the DB, just the binaries.

How can you "open source" something that doesn't include the sources?

That's parent's point, they didn't. They only made the binaries available under the Apache 2.0 license

Re: Datomic is Free

#179

Post by nlitened »

slaymaker1907 wrote: post #169
alecco wrote: post #92

Earlier quoted context omitted.

Just the temporal properties alone make it very useful for anything where it matters like billing, finance, inventory. Else you are in views/schema/indexing hell to do it on top of SQL. There is some SQL temporal support but it's not great and varies a lot. Also since it's not native to the storage it has a lot of complexity issues under the rug making it not great. Many financial systems use Event Sourcing (OOP + OR…

The temporal support seems handy, but time is still going to be really tricky for financial systems. Datomic only covers what the physical state of the database was at a particular time, but there's also the effective legal time (maybe a payment was dated a day before the system actually processed it) as well as requirements to remove data after a period of time (including point in time stuff).

There's a bitemporal Datomic-like database from JUXT that does exactly that, I believe https://www.xtdb.com

Re: Datomic is Free

#180

Post by thewataccount »

electroly wrote: post #143
dimitar wrote: post #115

Earlier quoted context omitted.

Consider this use case - in addition to your web app, you have a reporting service that makes heavy duty reports; if you run one at a bad time, bad things might happen like users not being able to log in or do any other important work, because the database is busy with the reports. So in a traditional DB you might have a DBA set up a reporting database so the operational one is not affected. Using Datomic the reporti…

I appreciated this insight into other people's use cases, thank you for that! This architecture brings RethinkDB to mind, which also had some ability to run your client as a cluster node that you alone get to query. (Although there it was more about receiving the live feed than about caching a local working set.)

Is RethinkDB still around?

They actually have recent commits, and a release last year.

Post reply on HN