Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NoSQL Meets Bitcoin and Brings Down Two Exchanges

hackingdistributed.com

Post reply on HN
Print view View original

1–10 of 69 posts

Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges

#3

Post by adambard »

TLDR: Exactly what you might have guessed: the exchanges backed their financial operations with a datastore without transactions, tragedy ensues.

This can't be viewed as an indictment of "NoSQL" though; there are plenty of databases featuring ACID transactions that qualify as "NoSQL" (FoundationDB and OrientDB come to mind). That said, none of the popular NoSQL contenders have them, and the new guys with transactions tend to be commercial, so if you need transactions in an free and open-source datastore, just use PostgreSQL.

Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges

#4

Post by nemothekid »

While I do recognize this is a marketing piece for HyperDex, I'm disappointed HyperDex only chooses to compare itself with MongoDB and Cassandra.

It would be different if HyperDex was fully open source and free, but its not. The "transaction" feature is only available with a license, which in most cases will price you out of any developer who is looking to start a bitcoin exchange. So unfortunately it comes across as "If only you had spent $x/mo, buying our product, then you wouldn't have had this problem!"

Now that isn't bad, and its the definition of marketing, but then it goes on to compare itself against Cassandra (which has a widely different use case in the wild) and Mongo (which is already the whipping boy of the database community). However it doesn't help the use cases are also extremely well served by something like Postgres, which doesn't need a Warp addon, and has a huge developer community.

What I would be more interested in, is how does HyperDex compare with the new "NoSQL" databases (FoundationDB, OrientDB, Spanner-like Databases) or even the traditional commercial SQL like Oracle.

In short, it doesn't help to test and compare unsuccessful and unpopular use cases with these technologies. Most competent people, the same people who might pay for HyperDex, already know Cassandra is bad at transactions. What I would like to see is tell me why someone like Netflix, or Ooyala (tried, tested and popular use cases) would move from Cassandra to HyperDex.

Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges

#6

Post by patio11 »

That's because banks employ systems that guard against this kind of elementary error. It's called transactions with ACID guarantees.

Bank systems are often not, in fact, ACID. Think of how many billions of transactions are processed in end-of-the-day batch runs, for example. Overdrafting is not only not impossible, it is a ludicrously profitable feature of the system.

How is this secure? Velocity limits, sophisticated anti-fraud systems, acceptable losses, and, as a bit of a by-the-way, the fact that people who make a habit of defrauding banks generally have a few weeks prior to being woken up by very serious men with guns.

Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges

#8

Post by rdtsc »

Please don't be MongoDB, please don't be MongoDB

...

Sigh

> a failure of distributed systems academics to educate developers and to equip them with clear-thinking frameworks.

Here is the thing, distributed systems are not standard academic courses. Just like networking they are often optional. That is really sad and it needs to change.

The CAP theorem is not something fringe and exotic anymore it should be standard taught to everyone.

MongoDB has really messed up. They promoted and marketed a shitty product wrapped in nice marketing. It created 2 things -- low barrier to entry and a trap. Both wrapped in one nice shiny package. "You should have read the docs to enable actually writing data disk when the user asks them to" doesn't jive with me. A really shitty default setup for something calling itself a database.

Oh well, they reaped the benefits of their strategy, both the short term (success) and long term (failure) - it became the butt of jokes and ridicule.

Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges

#9

Post by rdtsc »

patio11 wrote: post #6

That's because banks employ systems that guard against this kind of elementary error. It's called transactions with ACID guarantees. Bank systems are often not, in fact, ACID. Think of how many billions of transactions are processed in end-of-the-day batch runs, for example. Overdrafting is not only not impossible, it is a ludicrously profitable feature of the system. How is this secure? Velocity limits, sophisticate…

It is interesting because the transaction log inconsistencies can often be resolved technically and because of the regulatory framework. Overdrafts can be "solved" in business sense.

Now imagine something like an bitcoin exchange and bitcoin transaction log. Transaction errors cannot be fixed easily. So overdrafting an account means never getting the money back. There a single account would need to have ACID properties in way.

It seems ability to deal with and tolerate data conflicts and inconsistencies is different for each business. There was some interest in CRTDs (data types that can automatically converge after a conflict occurs, like imagine a max() function or a set union operation) but still means shortly there could be an inconsistency in the system and sometimes that can be exploited and is just not an option.

Post reply on HN