Their logo is a squirrel giving an invisible blowjob
Cannot unsee.
Noms – A versioned, forkable, syncable database
31–40 of 107 posts
Re: Noms – A versioned, forkable, syncable database
#32I hope there is a prune option to delete very old commits.
Re: Noms – A versioned, forkable, syncable database
#33Earlier quoted context omitted.
If by "Git for data" you mean accumulate-only (or append-only), immutable data stores... There are already many existing solutions. It's always good to see alternatives, though!
Can you point me at some? Because I've tried a few immutable data stores and been disappointed every time. Given about 10 GB of JSON structures, I keep finding things that can't outperform the boring combo of: * Convert the versioned data to tab-separated values * COPY it into Postgres every time * Hope Postgres can act immutable enough even though it wasn't designed to be The closest I've come to improving this situ…
(Disclaimer - I work at pachyderm)
Pachyderm is git for data. We work hard to make sure we can store data of different types (binary, text, json) efficiently. We also work hard to give you good mechanisms to read the data in a distributed way. I'd be curious how this suits your purposes.
Re: Noms – A versioned, forkable, syncable database
#34How is it that you have 2 reference implementations, written in 2 different cross platform environments, yet there is no support for Windows? Why would I use this if I can't use it everywhere?
Most devs dont use windows these days.
Re: Noms – A versioned, forkable, syncable database
#35Hi Hacker News. I'm one of the founders of the Noms project and Attic Labs, the company behind it. Happy to answer any questions.
In the meantime, as long as I've got your attention, here's a few new stuffs we've been working on since last time Noms was discussed here in August:
- A prototype query language, and a demo of how to create indexes in Noms: https://www.youtube.com/watch?v=fv6_T5yaWns
- Support for merging concurrent (and potentially conflicting) changes: https://www.youtube.com/watch?v=--7dgoJBdjU
Re: Noms – A versioned, forkable, syncable database
#36I hope there is a prune option to delete very old commits.
There isn't yet, but there could be (ala shallow clone in Git).
Re: Noms – A versioned, forkable, syncable database
#37Very interesting, I think we need a git for data. What is the performance of diffs and merges? What data size does it become too slow?
So.. CRDTs?
Re: Noms – A versioned, forkable, syncable database
#38Adam Leventhal (DTrace, OpenZFS) took a look at building a FUSE filesystem on Noms using Go. http://dtrace.org/blogs/ahl/2016/08/09/nomsfs/ https://news.ycombinator.com/item?id=12255450
Sounds like what you'd want for a self-hosted Dropbox clone. I wonder what Syncthing, for instance, uses for reconciling differences on different clients.
Re: Noms – A versioned, forkable, syncable database
#39I think there is a very similar library in Haskell called project m36. Here's its github page on transactions: https://github.com/agentm/project-m36/blob/master/docs/trans...
Re: Noms – A versioned, forkable, syncable database
#40Noms is a great example of the power of decentralized database technology, the interesting research that goes into such systems, and wonderful documentation to browse. I do want to note some tradeoffs with Content-Addressed and Append-Only systems, as my work on a similar project ( an Open Source Firebase, https://github.com/amark/gun ) made me move away from those ideas (even though they are great ideas). - Content-…
1. We have prototyped basic query functioanlity already (https://www.youtube.com/watch?v=fv6_T5yaWns) and Noms was designed from the beginning to support efficient indexes and range scans. So I'm not sure why it would be harder for us to support a query language than any other database.
2. It's true that content addressing can exacerbate data locality which can hurt read performance. However, there are thing you can do to get a lot of that back.