I never understood why we don’t have SQL databases that track all changes in a “third dimension” (column being one dimension, row being the second dimension). It might be a bit slower to write, but hook the logic in to write/delete, and suddenly you can see exactly when a field was changed to break everything. The right middleware and you could see the user, IP, and query that changed it (along with any other queries…
Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
41–50 of 185 posts
Re: Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
#42I’m a bit sad it doesn’t seem to have git style syncing though.
Re: Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
#43You can also use Git for data! It’s a bit slower, but smart use of partial/shallow clones can address performance degradation on large repositories over time. You just need to take care of the transformation between “physical” trees/blobs and “logical” objects in your dataset (which may not have 1:1 mapping, as having physical layer more granular reduces likelihood of merge conflicts). I’m also following Pijul, which…
Git is too complicated. It's barely usable for daily tasks. Look at how many people have to Google for basic things like uncommitting a commit, or cleaning your local repo to mirror a remote one. Complexity is a liability. Mercurial has a nicer interface. And now I see the real simplicity of non-distributed source control systems. I have never actually needed to work in a distributed manner, just client-server. I hav…
Re: Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
#44Re: Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
#45What people usually miss about these things is normal version control benefits hugely from content addressing and normal forms. The salient aspect of relational data is that it's cyclic, this makes content addressing unable to provide normal forms on it's own (unless someone figures out how to Merkle cylic graphs!), but the normal form can still made other ways. The first part is easier enough, store rows in some ord…
A list of all current entities, by hash, is stored higher up the tree. Whether it's better that objects themselves store their entity ID or if that's a separate data structure mapping entity to hash IDs depends on the situation.
On second reading I guess your comment was actually about how to come up with content-based IDs for objects. I guess my point was that in the real world you don't usually need to do that, because if object identity besides its content is important you can just give it an arbitrary ID. How often does the problem of differentiating between a graph containing identical Alices vs one with a single self-friending Alice actually come up? Is there any way around it other than numbering the Alices?
Re: Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
#46Earlier quoted context omitted.
Git is too complicated. It's barely usable for daily tasks. Look at how many people have to Google for basic things like uncommitting a commit, or cleaning your local repo to mirror a remote one. Complexity is a liability. Mercurial has a nicer interface. And now I see the real simplicity of non-distributed source control systems. I have never actually needed to work in a distributed manner, just client-server. I hav…
Git is used prolifically in the tech industry. What on earth are you talking about?
Re: Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
#47I never understood why we don’t have SQL databases that track all changes in a “third dimension” (column being one dimension, row being the second dimension). It might be a bit slower to write, but hook the logic in to write/delete, and suddenly you can see exactly when a field was changed to break everything. The right middleware and you could see the user, IP, and query that changed it (along with any other queries…
Re: Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
#48I never understood why we don’t have SQL databases that track all changes in a “third dimension” (column being one dimension, row being the second dimension). It might be a bit slower to write, but hook the logic in to write/delete, and suddenly you can see exactly when a field was changed to break everything. The right middleware and you could see the user, IP, and query that changed it (along with any other queries…
Last time I checked, it was supported in SQL server and MariaDB, and Postgres via an extension.
Re: Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
#49It's amazing this isn't a standard feature. The database world seems to have focused on large, high volume, globally distributed databases. Presumably you would't version clickstream or IoT sensor data. Features like this that are only feasible below a certain scale are underdeveloped and I think there's opportunity there.
Every DB engine used at scale has a concept of snapshots and backups. This just looks like someone making a git-like porcelain for the same kind of DB management constructs.
Dolt actually stores the tables, rows, and commits in a Merkle DAG, like Git. So you get branch and merge. You can't do branch and merge with snapshots.
(You also get the full git / github toolbox: push and pull, fork and clone, rebase, and most other git commands)
Re: Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
#50I would rather have a "GitHub for data" - an SQL database I could have hosted for free, give everybody read-only access to and give some people I choose R/W access to. That's a thing I miss really.
Dolt also have a paid product called DoltHub, check it out.