Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Noms – A new decentralized database based on ideas from Git

medium.com

Post reply on HN
Print view View original

141–150 of 180 posts

Re: Show HN: Noms – A new decentralized database based on ideas from Git

#141

Post by wingerlang »

nartz wrote: post #102

So, i realize this project is early, but it would be EXTREMELY helpful to walk through someone's use case - like, who is the target here? A business analyst who iterates on cleaning / analyzing small excel csvs? Or someone else? After watching the screencast, all I saw was a bunch of commands explained (could have read the docs for that), instead, I'd like to walk through a use-case where this solves someones problem…

He mentioned at least one. His friend went to a cabin where there were little to no internet connection, then he updated the database on a local device. And later on other database-nodes would just pull the updated data.

Seems like it is for personal use. Maybe something to build apps on top of.

Re: Show HN: Noms – A new decentralized database based on ideas from Git

#142

Post by kragen »

I've been wanting something like Noms for a while. Prolly trees sound really promising.

In intro.md, you suggest, "If you wanted to find all the people of a particular age AND having a particular hair color, you could construct a second map having type Map>, and intersect the two sets." In that case, how should I keep the two maps in sync? Do I need to atomically update the logic of all the instances of the application to modify both maps instead of just one? Or do I keep the second map (the hair color index) in a separate index database and update the index whenever I pull changes from a remote database? (What does the API look like for getting notified of new changes that haven't been indexed yet?)

I see that "noms sync" does both push and pull. Does that mean I can't pull data from a database I can't write to? How does that work over HTTP — do I need to use a special HTTP server that knows how to accept and authenticate write requests, or can I just dump a Noms dataset in a directory and serve it up with Apache?

Forgive me if these questions are obvious — I've read the docs I could find, but I haven't read any of the code beyond the hr sample.

Re: Show HN: Noms – A new decentralized database based on ideas from Git

#143

Post by rdtsc »

espadrine wrote: post #120
im_down_w_otp wrote: post #115

Earlier quoted context omitted.

What is "latest" in a situation where you're experiencing concurrent writes?

Google's Spanner, for instance, relies on their TrueTime design, which requires having a GPS clock and an atomic clock on each datacenter, I believe. Most designs simply rely on NTP or a similar time synchronization system. Another approach is to maintain total order of writes. Assuming some form of consensus protocol to determine write order, the unicity of the order ensures synchronization. That design, however, te…

> which requires having a GPS clock and an atomic clock on each datacenter,

Right. I'll download an open source database, then buy a commercial GPS clock and attach to it.

https://www.amazon.com/Spectracom-1200-033-SECURESYNC-MODULA...

Only $12k for a Spectracom. Granted they are nice.

Well, I guess if it demonstrates anything is one fact -- if even Google needed GPS hardware to provide "latest" in a distributed system, then OP is right. Time in distributed systems is very hard.

> Most designs simply rely on NTP or a similar time synchronization system.

Not sure if "simply" was mean sarcastic or not. If reliability and deletion of user's data is important, and it relies on getting NTP time, I would strongly advice not to use that distributed db system.

Re: Show HN: Noms – A new decentralized database based on ideas from Git

#145

Post by tamana »

wingerlang wrote: post #141
nartz wrote: post #102

So, i realize this project is early, but it would be EXTREMELY helpful to walk through someone's use case - like, who is the target here? A business analyst who iterates on cleaning / analyzing small excel csvs? Or someone else? After watching the screencast, all I saw was a bunch of commands explained (could have read the docs for that), instead, I'd like to walk through a use-case where this solves someones problem…

He mentioned at least one. His friend went to a cabin where there were little to no internet connection, then he updated the database on a local device. And later on other database-nodes would just pull the updated data. Seems like it is for personal use. Maybe something to build apps on top of.

That doesn't differentiate noms from postgres or any other multimaster database.

Does it do clever merging?

Re: Show HN: Noms – A new decentralized database based on ideas from Git

#146

Post by wingerlang »

tamana wrote: post #145
wingerlang wrote: post #141

Earlier quoted context omitted.

He mentioned at least one. His friend went to a cabin where there were little to no internet connection, then he updated the database on a local device. And later on other database-nodes would just pull the updated data. Seems like it is for personal use. Maybe something to build apps on top of.

That doesn't differentiate noms from postgres or any other multimaster database. Does it do clever merging?

I just went by the use case he mentioned in the video. I don't really know about these technical details for databases etc.

Re: Show HN: Noms – A new decentralized database based on ideas from Git

#149

Post by alexatkeplar »

aboodman wrote: post #126
alexatkeplar wrote: post #76

Earlier quoted context omitted.

Why would you exclusively support schema inference, rather than also allowing users to manually specify their schemas? Schema inference is very difficult to do correctly and safely, especially with small initial samples of instances (source: work on https://github.com/snowplow/schema-guru ).

There's a little bit of terminology overloading going on here. In Noms every value has a type. It's an immutable system, so this type just is. The type of `42` is `Number`. The type of `"foobar"` is `String`. The type of `[42,44]` is `List `. And if you add "foo" to that list, the type becomes `List `. We don't try to infer a general database schema from a few instances of data. We just apply this aggregation up the…

Right - the challenge is that with dynamic typing and without schema validation, it's incredibly easy to break any strongly typed client/consuming application. You think you are dealing with a `List`, you have Go/Java/Haskell/whatever apps which are consuming that in a strongly typed fashion using their idiomatic record types, and then suddenly a user accidentally sends in a single value which turns the tree of values into a `List`, and all your consuming apps break.

Given that schema validation ("does this instance match this type?") is simpler to implement than schema inference ("what is the type of this instance?"), it's surprising to me to deliver inference first...

Re: Show HN: Noms – A new decentralized database based on ideas from Git

#150

Post by dnautics »

nartz wrote: post #102

So, i realize this project is early, but it would be EXTREMELY helpful to walk through someone's use case - like, who is the target here? A business analyst who iterates on cleaning / analyzing small excel csvs? Or someone else? After watching the screencast, all I saw was a bunch of commands explained (could have read the docs for that), instead, I'd like to walk through a use-case where this solves someones problem…

Benchling could use this.
Post reply on HN