Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DoltLab v0.2.0

dolthub.com

Post reply on HN
Print view View original

11–17 of 17 posts

Re: DoltLab v0.2.0

#11

Post by cgio »

sverhagen wrote: post #2

Inspired by... but admittedly off-topic... I had recently been wondering how people keep a full auditable history of their data. I've used Hibernate Envers in the past or the now seemingly-defunct Temporal Tables extension for PostgreSQL. What are people using these days? Is DoltLab it, or are there more common solutions?

Can’t say what people are using but keen to hear from others. I am looking into this atm, no implementation yet, and some of the things I am reading about are (I’ll add dolt to this list)

Xtdb: https://xtdb.com/

Terminus: https://github.com/terminusdb/terminusdb

Nessie: https://projectnessie.org/

Dvc : https://dvc.org

Liquid base: https://liquibase.org/

Datasette: https://datasette.io/

Still framing in my mind how schema evolution, x-temporal and e.g. scd in data modelling, version control etc. tie in together in an approach.

Re: DoltLab v0.2.0

#14

Post by reltuk »

parentheses wrote: post #7

So many questions about the git-like semantics. - how are shas created? - assuming you hash the entire diff, can columns be ignored? e.g. timestamps or other "unimportant" data - do any two insertions into a table "conflict"?

To quickly answer these...

- it's content addresses / merkle DAG all the way down. The commit's hash is something like meta (author, description, timestamp) + parents hash + root value hash. The root value is composed of the schema, and pointers to table and index maps. Tables and indexes are merkle DAGs of the table data organized in a structure a bit like a B-tree, but with cut points chosen by a rolling value hash in order to probabilistically re-synchronize on incremental changes. Some details: https://www.dolthub.com/blog/2020-04-01-how-dolt-stores-tabl... , https://www.dolthub.com/blog/2020-05-13-dolt-commit-graph-an...

- currently table data is stored row major for full rows of the table and so diffing cannot efficiently ignore individual columns.

- direct conflicts are computed on a row-by-row basis, using the primary key of the row. And then constraints and foreign key references are maintained and validated across merges and edits.

HTH, happy to answer any further questions :).

Re: DoltLab v0.2.0

#15

Post by zachmu »

caffeine wrote: post #10

I can’t figure out if this is a real product or a joke site? The name is confusing. If it’s a real product it’s cool, I’ve wanted something like this for a while (currently I just use git repos full of JSON files but this would be better I think).

Yup, it's a real product :)

If you want experiment quickly and aren't squeamish about putting your data on the internet, DoltHub is easier to get started with. DoltLab is just a (limited) self-hosted version of DoltHub.

Re: DoltLab v0.2.0

#16

Post by sverhagen »

cgio wrote: post #11
sverhagen wrote: post #2

Inspired by... but admittedly off-topic... I had recently been wondering how people keep a full auditable history of their data. I've used Hibernate Envers in the past or the now seemingly-defunct Temporal Tables extension for PostgreSQL. What are people using these days? Is DoltLab it, or are there more common solutions?

Can’t say what people are using but keen to hear from others. I am looking into this atm, no implementation yet, and some of the things I am reading about are (I’ll add dolt to this list) Xtdb: https://xtdb.com/ Terminus: https://github.com/terminusdb/terminusdb Nessie: https://projectnessie.org/ Dvc : https://dvc.org Liquid base: https://liquibase.org/ Datasette: https://datasette.io/ Still framing in my mind how sc…

Not what I was looking for, but a nice list. Just want to call out, what I've also seen in the article that a sibling comment pointed to: if you're gonna mention Liquibase, why would you not mention Flyway: https://flywaydb.org/

Good luck with your search.

Re: DoltLab v0.2.0

#17

Post by cgio »

sverhagen wrote: post #16
cgio wrote: post #11

Earlier quoted context omitted.

Can’t say what people are using but keen to hear from others. I am looking into this atm, no implementation yet, and some of the things I am reading about are (I’ll add dolt to this list) Xtdb: https://xtdb.com/ Terminus: https://github.com/terminusdb/terminusdb Nessie: https://projectnessie.org/ Dvc : https://dvc.org Liquid base: https://liquibase.org/ Datasette: https://datasette.io/ Still framing in my mind how sc…

Not what I was looking for, but a nice list. Just want to call out, what I've also seen in the article that a sibling comment pointed to: if you're gonna mention Liquibase, why would you not mention Flyway: https://flywaydb.org/ Good luck with your search.

No specific reason other than not having come across it. Will add it on my list.
Post reply on HN