Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
51–60 of 185 posts
Re: Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
#52Wordpress would have benefited from this. What a lot of webmasters want is, test the site locally, then merge it back. A lot of people turned to Jekyll or Hugo for the very reason that it can be checked into git, and git is reliable. A static website can’t get hacked, whereas anyone who has been burnt with Wordpress security fail knows they’d prefer a static site. And even more: People would like to pass the new webs…
[deleted]
Re: Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
#53This is pretty cool. I wish `dolt diff` would use + and - though (isn't that standard?) rather than > and < which is harder to distinguish.
Both are "standard" from the UNIX/GNU diff(1) tool. The default behavior gives you the '>' and '<' format and using -u gives you the "unified" '+' and '-' format.
Re: Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
#54[deleted]
Re: Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
#55Dolt might be good but never underestimate the power of Type 2 Slowly Changing Dimension tables [1]. For example, if you had an SSIS package that took CSV and imported them into a database, and one day you noticed it accidently rounded the value incorrectly, you could fix the data and retain traceability of the data which was there originally.
E.g., SSIS package writes row of data: https://imgur.com/DClXAi5
Then a few months later (on 2020-08-15) we identify that trans_value was imported incorrectly so we update it: https://imgur.com/wdQJWm4
Then whenever we SELECT from the table we always ensure we are extracting "today's" version of the data:
select * from table where TODAY between effective_from and effective_to
Re: Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
#56Well, maybe this is the ONE. But I've heard "git for data" way too many times to jump on board of the latest tool. I'll wait till "dust settles" and there's a clear winner. Till then, it's parquet, or even pickle.
Re: Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
#57Earlier quoted context omitted.
Git is too complicated. It's barely usable for daily tasks. Look at how many people have to Google for basic things like uncommitting a commit, or cleaning your local repo to mirror a remote one. Complexity is a liability. Mercurial has a nicer interface. And now I see the real simplicity of non-distributed source control systems. I have never actually needed to work in a distributed manner, just client-server. I hav…
Git is used prolifically in the tech industry. What on earth are you talking about?
To me there’s some irony in that all insta-criticism of Git in responses to my comment presumably applies to a project that describes itself as “Git for data” and promises exact reproduction of all Git command behaviour—therefore suffering from the same shortcomings.
Re: Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
#58Wordpress would have benefited from this. What a lot of webmasters want is, test the site locally, then merge it back. A lot of people turned to Jekyll or Hugo for the very reason that it can be checked into git, and git is reliable. A static website can’t get hacked, whereas anyone who has been burnt with Wordpress security fail knows they’d prefer a static site. And even more: People would like to pass the new webs…
they have a theme preview now. :)
Re: Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
#59Problematic name, could become a millstone on the neck of the developer far into the future.
Agreed. I couldn't immediately see if it was "DOLT" or "do it", as in "just do it". It's the former.
Reminds me of this user testing story from the early Macintosh days: https://www.folklore.org/StoryView.py?story=Do_It.txt
Re: Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
#60This is absolutely fascinating, conceptually.
However, I'm struggling to figure out a real-world use case for this. I'd love if anyone here can enlighten me.
I don't see how it can be for production databases involving lots of users, because while it seems appealing as a way to upgrade and then roll back, you'd lose all the new data inserted in the meantime. When you roll back, you generally want to roll back changes to the schema (e.g. delete the added column) but not remove all the rows that were inserted/deleted/updated in the meantime.
So does it handle use cases that are more like SQLite? E.g. where application preferences, or even a saved file, winds up containing its entire history, so you can rewind? Although that's really more of a temporal database -- you don't need git operations like branching. And you really just need to track row-level changes, not table schema modifications etc. The git model seems like way overkill.
Git is built for the use case of lots of different people working on different parts of a codebase and then integrating their changes, and saving the history of it. But I'm not sure I've ever come across a use case for lots of different people working on the data and schema in different parts of a database and then integrating their data and schema changes. In any kind of shared-dataset scenario I've seen, the schema is tightly locked down, and there's strict business logic around who can update what and how -- otherwise it would be chaos.
So I feel like I'm missing something. What is this actually intended for?
I wish the site explained why they built it -- if it was just "because we can" or if projects or teams actually had the need for git for data?