Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cognitect: Relevance merges with Metadata Partners (Datomic)

cognitect.com

Post reply on HN
Print view View original

21–27 of 27 posts

Re: Cognitect: Relevance merges with Metadata Partners (Datomic)

#21

Post by calibraxis »

drcode wrote: post #6
MrBuddyCasino wrote: post #5

Earlier quoted context omitted.

What is the big deal about Datomic? From their FAQ: "Datomic is not a good fit if you need unlimited write scalability, or have data with a high update churn rate (e.g. counters)." Don't you get most of that through... caching? Also, it seems to assume that the dataset will fit into RAM.

In short, you can ask a datomic database stuff like "Show me all things that are different for customer X from the database today versus the database one year ago on September 14th at 9:32 AM" and it can answer those types of queries with high performance. And no, the dataset does not need to fit in RAM.

You can also go forward in time, to a hypothetical future. (That is, you add data and get back a new DB value, which you can query against. But the DB's source isn't modified.) Can be useful in analytics which deal with what-if scenarios.

Re: Cognitect: Relevance merges with Metadata Partners (Datomic)

#22

Post by robertfw »

1qaz2wsx3edc wrote: post #17
mike_ivanov wrote: post #3

Any plans to opensource Datomic?

Follow up: What are some open-source alternatives or similar software?

I have done some searching and have not turned up anything. The functionality that datomic enables is really intriguing, but I have trouble bringing myself around to using a non-open source bit of tooling.

Re: Cognitect: Relevance merges with Metadata Partners (Datomic)

#23

Post by jasonwatkinspdx »

1qaz2wsx3edc wrote: post #17
mike_ivanov wrote: post #3

Any plans to opensource Datomic?

Follow up: What are some open-source alternatives or similar software?

I'm doing some preliminary work on one. But realistically, it's a lofty goal and it'll be hard to get going. My priorities are different and so I'm taking some different design paths than datomic as well (ie, no datalog).

Re: Cognitect: Relevance merges with Metadata Partners (Datomic)

#24

Post by jasonwatkinspdx »

zerr wrote: post #10
grayrest wrote: post #8

Earlier quoted context omitted.

Datomic is interesting because it's a different take on what a database should look like. The TLDR version by someone who's looked into it a bit but not actually used it: * Storage, Transactions, and Querying are separated as in different processes/machines separated. * Data is immutable. Storage is pluggable and has implementations on top of Dynamo/Riak. * Transaction semantics and ordering are controlled by a singl…

You mean the whole data is fetched to the client and only queried afterwards? Why did they choose this way?

Only the range of data the client is interested in is fetched from the storage layer.

As for why they chose this, you'd have to ask them to be sure.

But two reasonable assumptions are: 1. they wanted the storage layer to be "dumb", in particular so that they could use existing services like Dynamo. 2. they wanted reading processes to be totally independent. Readers can talk directly to the dumb storage layer without any centralized resource coordinator to execute queries. That means horizontal scalability in the strict sense.

Re: Cognitect: Relevance merges with Metadata Partners (Datomic)

#25

Post by jawns »

I attended a tech conference earlier this year where Rich Hickey was speaking. He was trumpeting the fact that data never really gets deleted in Datomic, and someone brought up the question: What happens if you are legally required to delete something from your database? I seem to remember him saying that Datomic wasn't really designed for that scenario, which sounds like a major problem.

Re: Cognitect: Relevance merges with Metadata Partners (Datomic)

#26

Post by andrewvc »

jawns wrote: post #25

I attended a tech conference earlier this year where Rich Hickey was speaking. He was trumpeting the fact that data never really gets deleted in Datomic, and someone brought up the question: What happens if you are legally required to delete something from your database? I seem to remember him saying that Datomic wasn't really designed for that scenario, which sounds like a major problem.

That actually isn't the case, you can delete data in datomic. http://docs.datomic.com/excision.html

Re: Cognitect: Relevance merges with Metadata Partners (Datomic)

#27

Post by dustingetz »

zerr wrote: post #10
grayrest wrote: post #8

Earlier quoted context omitted.

Datomic is interesting because it's a different take on what a database should look like. The TLDR version by someone who's looked into it a bit but not actually used it: * Storage, Transactions, and Querying are separated as in different processes/machines separated. * Data is immutable. Storage is pluggable and has implementations on top of Dynamo/Riak. * Transaction semantics and ordering are controlled by a singl…

You mean the whole data is fetched to the client and only queried afterwards? Why did they choose this way?

indexes and chunks of data that are used often remain cached in each application instance, and new changes are streamed to the application cache.

It means that reads from a hot cache do not touch network. Reads are very fast and scale "out". You can write code that does a lot of reads without caring much about performance. (SQL reads only scale "up" and you care very much about their performance.)

Datomic is like Git (distributed reads, central writes); Postgres is like CVS/SVN (centralized reads and writes). This is made possible by immutable history.

Post reply on HN