Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Noms – A new decentralized database based on ideas from Git

medium.com

Post reply on HN
Print view View original

21–30 of 180 posts

Re: Show HN: Noms – A new decentralized database based on ideas from Git

#21

Post by aboodman »

pinko wrote: post #17
aboodman wrote: post #13

Earlier quoted context omitted.

Yes, we credited bup in various places, such as the design overview: https://github.com/attic-labs/noms/blob/master/doc/intro.md We were also heavily influenced by camlistore (which I hacked on for awhile), irmin, ipfs, and others who have done a lot of interesting work in this space. We do use rolling checksums, but I think we have done some novel work here: https://github.com/attic-labs/noms/blob/master/doc/intro.m…

FYI, the "HTTP protocol" link[1] in the intro page above is broken. PS: I love the name "Prolly Tree". [1] " rel="nofollow">https://github.com/attic-labs/noms/blob/master/datas/databas...

:) Thanks, we like it too.

Re: Show HN: Noms – A new decentralized database based on ideas from Git

#22

Post by daveloyall »

bra-ket wrote: post #11
aboodman wrote: post #2

Hi all. I'm one of the creators of Noms. Happy to answer any questions!

does noms understand sql and can it do joins?

It doesn't support queries. It's a datastore, not a database. This is from their FAQ. https://github.com/attic-labs/noms/blob/master/doc/faq.md

Re: Show HN: Noms – A new decentralized database based on ideas from Git

#23

Post by lachenmayer »

lachenmayer wrote: post #9

This looks really exciting, congrats to the team for launching! Could you tell us a bit about how this compares to dat? http://dat-data.com/

Some of the node hacker community's work on scuttlebutt, hyperlog etc also seems highly relevant, as it's all based on merkle DAGs.

https://github.com/ssbc/secure-scuttlebutt https://github.com/mafintosh/hyperlog

These are all super modular so fishing around in their dependency graphs should lead you to a whole bunch of really interesting projects :)

Re: Show HN: Noms – A new decentralized database based on ideas from Git

#24

Post by popmystack »

Wouldn't this be better not as an actual data store, but rather a way to snap shot relational databases and then use it as a back up mechanism or perhaps doing historical analytics on that database?

A coworker and I have often talked about doing a git-based database but for far different reasons, and we've run into problems largely with querying data and having large amounts of reads happening concurrently. I'm curious to see how this will (hopefully) be eventually handled.

Re: Show HN: Noms – A new decentralized database based on ideas from Git

#25

Post by abhishivsaxena »

pinko wrote: post #12

Here's a relevant (albeit 4-year-old) StackExchange thread, "Is there a Git for data?": http://opendata.stackexchange.com/questions/748/is-there-a-g...

I once wrote super simple `git in js` for data objects. Was less than a couple of hundred of lines.

But there's also - https://github.com/mirage/irmin

Re: Show HN: Noms – A new decentralized database based on ideas from Git

#27

Post by cdbattags »

I had an idea for this with a buddy in college after doing case study research into Git. I've always considered this the next step into a decentralized world outside of code and non-typed "text". I know .csv's where mentioned a few times; are you looking to narrow into a few specific file types for proof of concept?

Re: Show HN: Noms – A new decentralized database based on ideas from Git

#28

Post by ripa »

aboodman wrote: post #13
the_duke wrote: post #6

Earlier quoted context omitted.

Are you aware of https://github.com/bup/bup ? It's a git file format based backup incremental backup tool. Do you also use rolling checksums (like bup) to prevent re-storing data when only a few bytes have changed? --> What are the major use cases you imagine for noms? I've read the section on Github, but can you give some specific examples that you see as good use cases?

Yes, we credited bup in various places, such as the design overview: https://github.com/attic-labs/noms/blob/master/doc/intro.md We were also heavily influenced by camlistore (which I hacked on for awhile), irmin, ipfs, and others who have done a lot of interesting work in this space. We do use rolling checksums, but I think we have done some novel work here: https://github.com/attic-labs/noms/blob/master/doc/intro.m…

bup is focused on backups. Would you say that noms is useful as a backup utility?

Do you have any mechanisms for securing integrity, specifically repairing the store in case of inconsistencies?

Is there any plans to support any data retention policy/functionality?

Re: Show HN: Noms – A new decentralized database based on ideas from Git

#29

Post by ah- »

bra-ket wrote: post #11
aboodman wrote: post #2

Hi all. I'm one of the creators of Noms. Happy to answer any questions!

does noms understand sql and can it do joins?

It'd be fantastic to combine something like noms with https://arrow.apache.org/ and then use Spark/Drill/Impala to query it.
Post reply on HN