Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Datomic Free Edition

blog.datomic.com

Post reply on HN
Print view View original

61–70 of 81 posts

Re: Datomic Free Edition

#61

Post by hardwear »

realize wrote: post #58

Great that this now free for small projects. I've been pondering two questions about Datomic: (1) immutable data is great, but what if we have to delete some past data, for privacy or regulatory reasons? Does this screw everything up? (2) can we modify values as-at past points in time? Example, an electricity company records usage monthly for customers, but sometimes past usage is incorrect and needs to be edited. Wh…

For (1), Use a "mark as deleted" option, with extremely strict permissions on viewing deleted data, to protect privacy.

Re: Datomic Free Edition

#62

Post by kleiba »

moe wrote: post #60
arohner wrote: post #59

Earlier quoted context omitted.

I'm pretty sure SQL will be added to datomic over Rich Hickey's dead body, and that's a good thing IMO. There are already enough SQL databases in the world. rhickey isn't looking to make a 'me too' database, he's interested in improving the state of software development. SQL is a terrible language and needs to die. It's fine that we disagree, but don't tell the people who are convinced it's a bad idea that they need…

It's not about making a "SQL-Database". It's about providing an interface that resembles something people are familiar with and that can be reasonably fluently typed in a REPL without breaking fingers. Just like Cassandra did with CQL and hbase does with HQL. SQL-syntax just happens to have stood the test of time for this kind of application. SQL is a terrible language No disagreement here. Just out of curiosity, wha…

Just out of curiosity, what would you call this language then: [:find ?entity :where [?entity :db/doc \"hello world\"]]?

What exactly do you see as the problem with this example? Or with SQL for that matter?

Honestly, I'm interested in what your criticism is. Is it the syntax?

Re: Datomic Free Edition

#63

Post by lukev »

moe wrote: post #60
arohner wrote: post #59

Earlier quoted context omitted.

I'm pretty sure SQL will be added to datomic over Rich Hickey's dead body, and that's a good thing IMO. There are already enough SQL databases in the world. rhickey isn't looking to make a 'me too' database, he's interested in improving the state of software development. SQL is a terrible language and needs to die. It's fine that we disagree, but don't tell the people who are convinced it's a bad idea that they need…

It's not about making a "SQL-Database". It's about providing an interface that resembles something people are familiar with and that can be reasonably fluently typed in a REPL without breaking fingers. Just like Cassandra did with CQL and hbase does with HQL. SQL-syntax just happens to have stood the test of time for this kind of application. SQL is a terrible language No disagreement here. Just out of curiosity, wha…

This isn't obvious if you don't know Clojure, but that's actually a data structure. In Datomic, queries are data you pass to the database, not a special syntax at all.

Re: Datomic Free Edition

#64

Post by moe »

kleiba wrote: post #62
moe wrote: post #60

Earlier quoted context omitted.

It's not about making a "SQL-Database". It's about providing an interface that resembles something people are familiar with and that can be reasonably fluently typed in a REPL without breaking fingers. Just like Cassandra did with CQL and hbase does with HQL. SQL-syntax just happens to have stood the test of time for this kind of application. SQL is a terrible language No disagreement here. Just out of curiosity, wha…

Just out of curiosity, what would you call this language then: [:find ?entity :where [?entity :db/doc \"hello world\"]]? What exactly do you see as the problem with this example? Or with SQL for that matter? Honestly, I'm interested in what your criticism is. Is it the syntax?

Yes, the syntax (or lack thereof, whichever you prefer).

Obviously I'm not entitled to tell anyone how to design their databases. I'm just saying there's good reasons why so many DBs stick to a SQL-like syntax, and that is because the alternatives are usually worse (think: familiarity, tooling, scripting, REPL, etc.).

For example, using the MongoDB REPL, which is probably close to what a datalog-REPL would look like, is rather painful.

This may all be a non-issue when a LISP-language is exclusively used on the client-side. That query-style probably just snaps in naturally there (I don't know lisp).

But if the database is supposed to be general purpose, accessed with lesser languages, scripted, quickly fixed by half-drunk humans at 4am in a REPL... then it certainly matters to have a sane intermediate lingo. Otherwise, at the least, every client-platform is going to invent their own.

I think the example of Cassandra is really a good one to study. They went through an interesting learning-process that seems very applicable here.

Edit: Please see my other comment above. It seems I have misunderstood the project goals and this is in fact not meant to become a general purpose DB. Under that premise my concerns obviously don't apply...

Re: Datomic Free Edition

#65

Post by arohner »

moe wrote: post #60
arohner wrote: post #59

Earlier quoted context omitted.

I'm pretty sure SQL will be added to datomic over Rich Hickey's dead body, and that's a good thing IMO. There are already enough SQL databases in the world. rhickey isn't looking to make a 'me too' database, he's interested in improving the state of software development. SQL is a terrible language and needs to die. It's fine that we disagree, but don't tell the people who are convinced it's a bad idea that they need…

It's not about making a "SQL-Database". It's about providing an interface that resembles something people are familiar with and that can be reasonably fluently typed in a REPL without breaking fingers. Just like Cassandra did with CQL and hbase does with HQL. SQL-syntax just happens to have stood the test of time for this kind of application. SQL is a terrible language No disagreement here. Just out of curiosity, wha…

Clojure exists because rhickey didn't like the current state of programming languages. Dataomic exists because rhickey didn't like the current state of DB design. SQL is an anti-goal.

"Familiarity" is a terrible metric for new things. New things are not always familiar. As rhickey said in one of his talks, there's a difference between what's simple, and what's familiar. "I don't know German, does that make it unreadable?". German may or may not be the ideal thing, but the fact that it isn't english is not an argument against it.

> Just out of curiosity, what would you call this language then: [:find ?entity :where [?entity :db/doc \"hello world\"]]?

Datalog (or a dialect of it). Datalog is a variation on prolog, designed for querying databases. Just as there are many lisps, there are many datalogs. This one is interesting because datomic querys are valid clojure data structures. It's a great example of code as data. It's trivial to write clojure functions that return data that can be passed as a query to datomic. If you wanted, you could probably even write datomic queries that return other queries. Try doing that with SQL ;-)

Re: Datomic Free Edition

#66

Post by augustl »

realize wrote: post #58

Great that this now free for small projects. I've been pondering two questions about Datomic: (1) immutable data is great, but what if we have to delete some past data, for privacy or regulatory reasons? Does this screw everything up? (2) can we modify values as-at past points in time? Example, an electricity company records usage monthly for customers, but sometimes past usage is incorrect and needs to be edited. Wh…

>> (1) immutable data is great, but what if we have to delete some past data, for privacy or regulatory reasons? Does this screw everything up?

I recall reading about the ability to clear out old transactions, to account for fast-growing databases. For example, "delete all transactions and facts older than 2 months". 2 minutes of googling didn't yield any results though, sorry.

>> (2) can we modify values as-at past points in time? Example, an electricity company records usage monthly for customers, but sometimes past usage is incorrect and needs to be edited. What would be the best way to structure this to get the "updated" view as-of six months ago? And also the "original" view as-of six months ago?

If I understand you correctly, you do this by adding a new fact with the correct usage. Your queries will now return the latest number, and you can use `as-of` and so on to get the old value.

Re: Datomic Free Edition

#67

Post by moe »

arohner wrote: post #65
moe wrote: post #60

Earlier quoted context omitted.

It's not about making a "SQL-Database". It's about providing an interface that resembles something people are familiar with and that can be reasonably fluently typed in a REPL without breaking fingers. Just like Cassandra did with CQL and hbase does with HQL. SQL-syntax just happens to have stood the test of time for this kind of application. SQL is a terrible language No disagreement here. Just out of curiosity, wha…

Clojure exists because rhickey didn't like the current state of programming languages. Dataomic exists because rhickey didn't like the current state of DB design. SQL is an anti-goal. "Familiarity" is a terrible metric for new things. New things are not always familiar. As rhickey said in one of his talks, there's a difference between what's simple, and what's familiar. "I don't know German, does that make it unreada…

Thanks, I understand (now).

I made my first comments under the assumption that this aims to be a general purpose database, but multiple people have made clear now that this is not the case.

Obviously it makes no sense to argue for an intermediate QL (and one as half-baked as SQL) when the project is ultimately aimed at Lisp-purists[1].

[1] This is not meant derogatory, it's just a critical distinction from a DB that, say, my junior-admin who knows his SQL and Python and not much else could be expected to get along with.

Re: Datomic Free Edition

#68

Post by stuarthalloway »

Scotchy wrote: post #53

So basically, it is a triple-store with keeps track of time and no mutation of past. They have their own query language but it looks to me that they did a great job at making it as close to SPARQL as possible, which personally I am familiar with. Pretty neat stuff.

It is important to understand that datalog is not a new query language. It is an old one, with a strong pedigree. It has power equivalent to the relational model plus recursion.

SPARQL is cool, and they did a good job making it look a little like datalog.

Re: Datomic Free Edition

#69

Post by dhugiaskmak »

cynicalkane wrote: post #44
jskopek wrote: post #25

Earlier quoted context omitted.

I've seen a number of comments over the last few weeks to the effect of "if you don't understand this product, watch the intro video". In my opinion, if a product can't be described in a couple of concise sentences, it's going to run into some major roadblocks in getting traction. Watching a video incurs a number of costs - the time taken to watch the video, the effort of plugging in headphones, the pain of buffering…

I think the guy who'll be interested in Datomic isn't going to be looking for a data store describable in a few sentences, like "NoSQL with feature X". It looks to me like a NoSQL-ish Datalog with Clojure-style concurrency, reified by its distribution model. You would only go with that sort of thing if you know how it works, or are interested in experimenting and learning about it. It's not something the guy looking…

> It's not something the guy looking for "data store, alternative to SQL, meeting requirements X Y and Z" would choose.

Actually it seems like it's something that this hypothetical guy would be very interested in, given that Datomic is being pitched as an alternative to SQL.

Re: Datomic Free Edition

#70

Post by politician »

nirvana wrote: post #11

I saw this page, and was not surprised that it doesn't say what Datomic is. A lot of people make that mistake. So, I clicked the link at the end of the page to go to your website. I followed some more links. I found three links that talked about feature, benefits, and the architecture, and clicked all three opening them in tabs. I read the first part of every page and then skimmed the rest. I have no damn clue what t…

I realize this probably doesn't address the premise of your post, but Datomic is, by way of my personal understanding/analogy, a giant, multi-indexed SQL table which has 4 columns - a subject column, an attribute column, a value column for the attribute, and a transaction identifier column. The table is streamed to consumers into an LRU cache.

The transaction identifier column represents time which enables you to query the state of the database as of a point in time (e.g. yesterday, last month, etc.) for reporting. The subject-relation-object structure supports adhoc NoSQL-style changes to the schema. Transactions themselves are first-class concepts, so you can add relation-object pairs to them as well.

Their hypothesis seems to be that querying client-side data is faster than server-side data (obvious), and that updating data can be effectively synchronized by a dedicated transaction coordinator (potentially). Although, this is a single point of failure, the claim is that this code can be audited and perfected.

Post reply on HN