Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Datomic is Free

blog.datomic.com

Post reply on HN
Print view View original

261–270 of 436 posts

Re: Datomic is Free

#261

Post by ysleepy »

twic wrote: post #244
wwweston wrote: post #241

Earlier quoted context omitted.

Yeah, it seems you could be able to substitute thoughtful schema design avoiding updates/deletes for time-travel as a feature. I wonder if anyone has made a collection of reference examples implemented this way (and in general think that a substantial compendium good examples of DB schema and thinking behind them could be worthwhile).

I'm moderately confident you could mechanically transform a time-oblivious schema into a history-preserving one, and then write a view on top of it which gave a slice at a particular time. Moderately.

That is essentially what MVCC does.

Re: Datomic is Free

#262

Post by pjot »

wwweston wrote: post #241
bvanderveen wrote: post #236

Earlier quoted context omitted.

We implemented it in Postgres with 'created_at' and 'deleted_at' columns on everything and filtering to make sure that the object 'exists' at the time the query is concerned with. Changes in relationships between objects are modeled as join tables with a boolean indicating whether the relationship is made or broken and at what time. Our data model is not large and we had a very complete test suite already, so it was…

Yeah, it seems you could be able to substitute thoughtful schema design avoiding updates/deletes for time-travel as a feature. I wonder if anyone has made a collection of reference examples implemented this way (and in general think that a substantial compendium good examples of DB schema and thinking behind them could be worthwhile).

It’s called a slowly changing dimension. In this example, it’s a type-2.

https://en.m.wikipedia.org/wiki/Slowly_changing_dimension

Re: Datomic is Free

#263

Post by cellularmitosis »

ilrwbwrkhv wrote: post #105
hombre_fatal wrote: post #14

Aside, I remember HN in 2009 or so where Clojure was a daily homepage staple and Rich Hickey was putting out his talks about Clojure and code design. I watched a lot of that and used Clojure fulltime for five years. Wonder what he's up to these days.

The whole clojure ecosystem and the wonderful tools around it never really took off due to unclear documentation, poor onboarding and too few evangelists. Datomic and other products are really cool but are now being given away as scrapware due to this lack of effort to make the whole ecosystem more palatable, colorful and easy to get into for new audiences.

Even as a Clojure hobbyist I feel like all of these points are off? Between 4clojure, clojuredocs and the slack channel and the surprising number of books available, the onboarding and docs are great. And when I think of my favorite lang evangelists, hickey and nolen are absolutely #1 and #2, and have influenced me heavily, despite my day job not involving Clojure at all.

Re: Datomic is Free

#264

Post by koreth1 »

brundolf wrote: post #171
augustl wrote: post #11

Datomic's is perfect for probably 90% of small-ish backoffice systems that never has to be web scale (i.e. most of what I do at work). Writing in a single thread removes a whole host of problems in understanding (and implementing) how data changes over time. (And a busy MVCC sql db spends 75% of its time doing coordination, not actual writes, so a single thread applying a queue of transactions in sequence can be fast…

Something I've been curious about: how well (or badly) would it scale to do something similar on a normal relational DB (say, Postgres)? You could have one or more append-only tables that store events/transactions/whatever you want to call them, and then materialized-views (or whatever) which gather that history into a "current state" of "entities", as needed If eventual-consistency is acceptable, it seems like you c…

That's a pretty common pattern in event-sourcing architectures. It is a completely viable way to do things as long as "eventual-consistency is acceptable" is actually true.

Re: Datomic is Free

#266

Post by outworlder »

tyre wrote: post #206
gleenn wrote: post #193

Earlier quoted context omitted.

Rich Hickey has a great talk about how Objects are data structures with unique interfaces that are unnecessary complexity. He showed and example of a web server with a web request object with request headers etc etc. Doing simple things like collecting information out of that nested object structure is bespoke and harder than it should be for no real gain. If everything is a map or list or set, it becomes completely…

“No real gain” I don’t agree with this. iirc Rack ultimately uses and array to represent HTTP responses. It has three members: the status code, the headers, and the response body. If you’re shipping a new change, is it easier to mistake response[0] or response.headers? This is a trivial example, but the general class (ha) of trade-off is amplified with more complex objects. I love clojure and lisp but the blindness b…

> If you’re shipping a new change, is it easier to mistake response[0] or response.headers

False dichotomy. There are many options other than arrays. Clojure in particular is fond of hashmaps. You can have your response.headers without OOP.

Re: Datomic is Free

#267

Post by btown »

JulianWasTaken wrote: post #233
bvanderveen wrote: post #224

From experience: Datomic Cloud is slow, expensive, resource intensive, designed in the baroque style of massively over-complicated CloudFormation astronautics. Hard to diagnose performance issues. Impossible to backup. Ran into one scenario where apparently we weren't quick enough to migrate to the latest version, AWS had dropped support for $runtime in Lambda, and it became impossible to upgrade the CloudFormation t…

I don't doubt your story of course, and I love Postgres, but comparing apples to oranges no? Datomic's killer feature is time travel. Did you simply not use that feature once you moved off Datomic (and if so why'd you pick Datomic in the first place)? Or are you using Postgres using some extension to add in?

https://www.postgresql.org/docs/11/contrib-spi.html#id-1.11.... discusses a model for implementing time travel in Postgres https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit... discusses why it was removed in Postgres 12 - it seems logical that it's more maintainable to implement in plpgsql, though as far as I can tell there aren't off-the-shelf implementations of this.

We use https://django-simple-history.readthedocs.io/en/latest/ (with some custom tooling for diff generation) for audit logs and resettability, and while you can't move an entire set of tables back in time simultaneously, it's usually sufficient for understanding data history.

Re: Datomic is Free

#268

Post by outworlder »

ithrow wrote: post #245
Scarbutt wrote: post #104

Earlier quoted context omitted.

Datomic's is perfect for probably 90% of small-ish backoffice systems that never has to be web scale How do they scale it for Nubank? (millions of users)

I don't how they do it, but the obvious answer is probably sharding. Their cloud costs must be no joke. Peers require tons of memory and I can only guess they must have thousands of transactors to support that workload and who knows how many peers. Add to this that they probably need something like Kafka for integrating/pipelining all this data.

> Peers require tons of memory

As do most distributed databases. Even when you don't store your entire database (or working set) in memory, you'll likely still have to add quite a bit of memory to be used as I/O cache.

Re: Datomic is Free

#269

Post by elteto »

remram wrote: post #154
elteto wrote: post #151

Earlier quoted context omitted.

They didn’t open source the DB, just the binaries.

How can you "open source" something that doesn't include the sources?

Well, _technically_ you are now free to modify the binary and redistribute the result as per the Apache 2.0 license. That’s different than giving something as freeware, which would not allow/cover modification/redistribution.

Re: Datomic is Free

#270

Post by elteto »

Thorrez wrote: post #159
remram wrote: post #154

Earlier quoted context omitted.

How can you "open source" something that doesn't include the sources?

Yeah, they didn't open source anything. They just made it free.

Ah! Yes, but not quite! It’s not freeware. The binaries are technically open sourced, you can do with them as you please within the confines of the Apache license.
Post reply on HN