Not quite understanding the focus on NoSQL for this use case set. I love MongoDB in the right place and Redis in others. But when you have a bunch of requirements that lend themselves to traditional DBs, choosing MongoDB or HyperDex feels like choosing a tech because it's cool rather than because it is the proper engineering choice.
NoSQL Meets Bitcoin and Brings Down Two Exchanges
51–60 of 69 posts
Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges
#52"They did what anyone would do after reading one too many astroturf articles on Hacker News. Sure, their system failed, but in a sense, the overall system failed them."
The technology didn't fail them. On the contrary, the technology did exactly what it was supposed to do. The devs could have (should have) considered these race condition scenarios.
Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges
#53 I don't think that MongoDB or any nosql is right choice for financial application. You can build for yelp but not for banking.
Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges
#54Marklogic supports ACID transactions. being that it is also a horizontally scalable system, it allows those transactions not just to spawn multiple objects, but also across multiple hosts.
http://www.marklogic.com/blog/can-you-pass-the-acid-test/
They also support data tiering into HDFS, and now most recently RDF with SPARQL
They have been doing ACID for a number of years, not sure for how long.
They are not free (unless your data set is small)
Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges
#55Earlier quoted context omitted.
For what I can only guess are strategic reasons, Datastax is moving away from the schema-less design that made Cassandra popular with early adopters. There are workarounds to this deprecation such as using the old Thrift API, but it doesn't really inspire confidence in the product unless the new schema oriented way of doing things appeals to you. PS those aren't transactions.
This is not true. See this for example: http://www.datastax.com/dev/blog/thrift-to-cql3 You can do CREATE TABLE schemaless (id blob, name blob, value blob, PRIMARY KEY (id, name)) WITH COMPACT STORAGE; and have a 'schemaless' table, essenitally, just as with Thrift. This is not a workaround either - this is first-class supported, just as you can read/write from/to 'schemaless' thrift columnfamilies using CQL.
Sure, would you mind showing me how to use CQL to insert data into an old-school style Column Family that was created with the Thrift interface?
Linus Torvalds would be up in arms if he saw this kind of client facing API breakage.
Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges
#56Earlier quoted context omitted.
"Schema-less" is a hack that appeals to people that don't realize that there is always schema. It's just a question of whether you want your data model to be ambiguous and poorly defined, or explicit.
Schema-less also means that you can afford to have deviations in your schemas not hinder your development needs. I agree with you, but I'd venture to say that a schema-less database means what it does, but that a collection of data always has schemas.
No, the schema is always a property of the system as a whole, data alone doesn't have an implicit schema. The schema is the set of type constraints that data has to conform to so that it can be processed by the system. Thus, a DB being "schema-less" simply means that it won't check/enforce those constraints - the schema still is there, of course, simply because a given piece of software that does anything meaningful will not sensibly process any random piece of garbage you feed it.
Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges
#57Earlier quoted context omitted.
If I'm not mistaken, an eventually consistent database must also guarantee, well, eventual consistency.
Sure! All nodes will be consistent in that your balance is what they all think it is, and that the attacker's balance reflects the attack. But unless those actions are coupled in a transaction, each account may not be consistent with the other.
The problem is that while overdraft is very profitable for the bank; you acquire a legal liability to pay back that debt.
Overdraft cannot be offered in a non fractional exchange and the exchange can't find you if you take more than your balance. Money leaving the exchange must be reconciled with holdings or they are insolvent.
What we may see is blockchain denoted exchange holdings; your exchange wallet is unique and publically verifiable (only the exchange holds the keys). You could verify with balance with the address and all trades are performed on blockchain. Bitcoin would be inappropriate for this with its long confirm times but a newer faster scrypt or primecoin variant might do it. Exciting times!
Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges
#58That's because banks employ systems that guard against this kind of elementary error. It's called transactions with ACID guarantees. Bank systems are often not, in fact, ACID. Think of how many billions of transactions are processed in end-of-the-day batch runs, for example. Overdrafting is not only not impossible, it is a ludicrously profitable feature of the system. How is this secure? Velocity limits, sophisticate…
ACID would have solved this problem though, right?
ACID is simpler to reason about. If there is a strong justification for abandoning it, and enough expertise to still make the system work, then fine. But I doubt they had either the justification (I assume their transaction volume could be easily handled with a SQL system) or the expertise.
I'm bothered that MongoDB strives to make people feel comfortable abandoning ACID who clearly don't understand what they are doing.
Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges
#59Earlier quoted context omitted.
The problem is that eventual consistency doesn't mean eventual correctness. The basic issue is that math usually only works if you do it in the correct order, e.g. (X+2) 5 != (X 5)+2. And if you're using a document store to do mathematical transactions then you're not guaranteed that they execute in the correct order, all that you're guaranteed is that all the nodes will eventually be the same in your database.
But debits and credits do not use multiplication nor division, so for those transactions the order doesn't matter.
If eventually consistent databases stored pending operations rather than pending values, this would matter. Unfortunately, they actually tend to be value based. So when you perform an operation on stale data, either the new operation will be lost, or the old one, depending on the implementation.
Re: NoSQL Meets Bitcoin and Brings Down Two Exchanges
#60Earlier quoted context omitted.
For what I can only guess are strategic reasons, Datastax is moving away from the schema-less design that made Cassandra popular with early adopters. There are workarounds to this deprecation such as using the old Thrift API, but it doesn't really inspire confidence in the product unless the new schema oriented way of doing things appeals to you. PS those aren't transactions.
"Schema-less" is a hack that appeals to people that don't realize that there is always schema. It's just a question of whether you want your data model to be ambiguous and poorly defined, or explicit.
We aren't all building cat photo sharing apps.
Sparse columns dbs (AKA schema-less in Cassandra lingo) have all sorts of important use cases in the sciences, unless you feel like paging through 100,000+ empty columns to find or discard the information you need or don't need.
The alternative approach is to alter your table's schema every time you find a new sub characteristic of how a given system is expressing itself.