Live data from Hacker News

NoSQL Meets Bitcoin and Brings Down Two Exchanges

hackingdistributed.com

11–20 of 69 posts

Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges

#11
The first problem to be had when considering distributed systems is the fact that there is no such thing as a general purpose distributed system.

Some problems require doing 50x more reads than writes. Others have more writes than reads. Sometimes losing a few inserts is not a big deal, but a small delay is terrible. Other systems just can't tolerate data loss, and would pay any price to keep it that way. You might have a lot of small pieces of data, searchable in very simple ways, or you might have multi GB documents that require so much indexing than Solr and Elastic seem inadequate.

In the old days, you just bought a bigger box for the RDBMS, and you just were covered by knowing one tech. Now we need to go through a wide variety of tools that will rarely do everything you want, and have to write a bunch of code to compensate for the limitations of the tools. And then your DB of choice decides that they will only support their self hosted product, then they get bought by IBM, and you wonder if you'll be forced to retool your entire backend (Hello Cloudant!)

Until there is some clarity in the market, and we get to see distributed stacks that are general purpose, we'll see issues like this popping up all the time, as there is a lack of people that both have good knowledge of all the available options and the skill to put them together into something that will solve your specific problem.

Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges

#12
post #6

That's because banks employ systems that guard against this kind of elementary error. It's called transactions with ACID guarantees. Bank systems are often not, in fact, ACID. Think of how many billions of transactions are processed in end-of-the-day batch runs, for example. Overdrafting is not only not impossible, it is a ludicrously profitable feature of the system. How is this secure? Velocity limits, sophisticate…

Bank systems are ACID. They're often not immediate, using batch runs as you say, but the databases the batch runs work on are definitely ACID.

The problem is not necessarily with overdrafts, as you suggest, but - as the article says - with race conditions between multiple readers/writers causing two near-simultaneous withdrawls to only make a single debit to the source account, while crediting both destination accounts.

Note that this could lead to unexpected overdrafts being noticed later, if the exchange also wrote a consistent log of all transactions, and attempted to reconcile the log with the real-time balances periodically (e.g. on a daily basis), but it's not clear if they even did that. (Although I might be missing something)

Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges

#14
Sounds like the author of this 'article' just wanted to make a thinly veiled plug for the "HyperDex" product.

Blaming all of this on MongoDB is silly. You can do the kind of idiocy displayed by these exchanges even on generic DBs by doing things like not using transactions and/or not waiting for write confirmation.

The root problem is in the (lack of) design and programming of these systems not their data store.

Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges

#16
post #6

That's because banks employ systems that guard against this kind of elementary error. It's called transactions with ACID guarantees. Bank systems are often not, in fact, ACID. Think of how many billions of transactions are processed in end-of-the-day batch runs, for example. Overdrafting is not only not impossible, it is a ludicrously profitable feature of the system. How is this secure? Velocity limits, sophisticate…

Bank systems are ACID. They're often not immediate , using batch runs as you say, but the databases the batch runs work on are definitely ACID. The problem is not necessarily with overdrafts, as you suggest, but - as the article says - with race conditions between multiple readers/writers causing two near-simultaneous withdrawls to only make a single debit to the source account, while crediting both destination accou…

That's why banks have an available balance and a posted balance. One is the best available estimate of how much money you can spend. The other is the reconciled balance after transactions have been posted.

It would have been possible for the exchanges to have used a NoSQL db with this strategy, but they didn't. They should have been keeping transaction logs and reconciling balances, but they didn't.

While, I'm no fan of MongoDB, it isn't really their fault. These exchanges were designed by people who didn't know any better.

Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges

#17
post #6

That's because banks employ systems that guard against this kind of elementary error. It's called transactions with ACID guarantees. Bank systems are often not, in fact, ACID. Think of how many billions of transactions are processed in end-of-the-day batch runs, for example. Overdrafting is not only not impossible, it is a ludicrously profitable feature of the system. How is this secure? Velocity limits, sophisticate…

Bank systems are ACID. They're often not immediate , using batch runs as you say, but the databases the batch runs work on are definitely ACID. The problem is not necessarily with overdrafts, as you suggest, but - as the article says - with race conditions between multiple readers/writers causing two near-simultaneous withdrawls to only make a single debit to the source account, while crediting both destination accou…

If I'm not mistaken, an eventually consistent database must also guarantee, well, eventual consistency.

Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges

#19

Its unfortunate to use the ATM example because just such an exploit was used by carders [1] to get more money out of accounts than the accounts actually had. [1] http://time.com/48344/hackers-target-atms-for-unlimited-with...

ATMs aren't as smart as the article says either: "the real code will check to see if there are sufficient funds".

I noticed that (some?) ATMs around here tend to implement this flawed algorithm, paraphrasing the article:

  0. database.begin()
  1. mybalance = database.read("account-number")
  2. newbalance = mybalance - amount
  3. database.write("account-number", newbalance)
  4. database.commit()
  5. dispense_cash(amount)
At first sight this might seem ACID, but its not. Consider what happens if the ATM is out of cash: step#5 fails, but the money was already withdrawn from your account!

It doesn't automatically revert the transaction either: you have to go to a branch office and ask for the transaction to be reverted. So much for atomic transactions at ATMs.

Post reply on HN