Live data from Hacker News

NSA built a NoSQL database

wiki.apache.org

31–40 of 57 posts

Re: NSA built a NoSQL database

#31
Awesome NSA...

So NoSQL approach makes all those skiddies SQLi attacks moot.

Still 200k lines of code = ~2000 bugs...

So, opening it to the public will expose (some) of those, and fixes will be created. and Now, when are you going to show off that really kool advanced A.I. you guys are sitting on!

Re: NSA built a NoSQL database

#32
post #2

It seems that the tags for cells seems to be an important feature of this database, and they also mention it is appropriate for places where "privacy is important". Can someone explain the connection between these two? If I'm understanding right, the labeling makes it easy to address individual cells, but I'm not sure how that enhances privacy.

The labeling likely refers to Mandatory Access Control (MAC) where objects (data, cells) are assigned classification labels (e.g. Top Secret, Confidential) and subjects (users, processes) can only access objects that match the subject's assigned classification level.

Re: NSA built a NoSQL database

#33
post #23

Earlier quoted context omitted.

A joke?! A JOKE?! You jest good sir. I merely put on my tinfoil hat and thought, "Hmmm, didn't this happen to OpenBSD, Windows, every crypto system ever, numerous databases, and probably SELinux?" Then extrapolated out to a very valid point. How dare you claim I am not deadly serious about the NSA putting a back door in a database that is intended to be secure for the internet. How. Dare. You!

I still can't tell if you're joking. I've seen a possible back door or two in this or that, but nothing like "every crypto system ever". If you have evidence of a back door in AES, SHA-2, or anything NIST has standardized (other than Dual_EC_DRBG or openly weakened stuff like export SSL) lots of people would like to hear about it.

Didn't the NSA actually make DES stronger?

Re: NSA built a NoSQL database

#34
post #28
post #27

Earlier quoted context omitted.

Bruce Schneier explains why that is not a back door. http://www.schneier.com/crypto-gram-9909.html#NSAKeyinMicros... So other than this, are there any other NSA "backdoors in everything from crypto systems, operating systems, to even backdoors themselves"?

Oh, the great Bruce Schneier says so, so therefore it must be. How do you know he's not a shill for Microsoft and the NSA? Hmm? The great thing about backdoors is, when they get discovered they have perfect plausible deniability. "Oh that key named NSAKEY isn't for the NSA it's for...uh...this other agency. Yeah that's it! It's not even a key. Right Bruce? Right?!"

Are you going to deny any contrary evidence as being a shill for the people you're accusing? Down that path lies madness and conspiracies, Zed.

Re: NSA built a NoSQL database

#35

Earlier quoted context omitted.

I still can't tell if you're joking. I've seen a possible back door or two in this or that, but nothing like "every crypto system ever". If you have evidence of a back door in AES, SHA-2, or anything NIST has standardized (other than Dual_EC_DRBG or openly weakened stuff like export SSL) lots of people would like to hear about it.

Didn't the NSA actually make DES stronger ?

Yes, the story goes that the NSA assisted IBM in its development by tuning the specific values in the S-boxes to be resistant to differential cryptanalysis, which had not yet been publicly discovered.

They also reduced the key length from 64 to 56 bits. I found this suspicious and didn't accept the explanation that those 8 bits were needed for "parity". Yet, respected cryptographers say this actually brings the key size more in line with the effective strength. So those additional 8 bits in the key were not contributing to the security and it improves the "truth in labeling".

Why would they build weaknesses into standard blocks, the biggest consumer of which is the US government itself?

When the NSA had at times insisted on an upper limit for a protocol's security (e.g., export crypto), they usually would require a simple upper limit on the number of secret bits in the key. When they've submitted fixes they tend to be elegant and minimal (e.g. SHA-0 to SHA-1).

Re: NSA built a NoSQL database

#36
post #9
post #7

" The core codebase consists of 200,000 lines of code (mainly Java) and 100s of pages of documentation." 100s of pages of documentation is a promising start for any open source project.

It depends on what the documentation is. If it's 100 pages of "AbstractClassFactoryClassFactoryFactory is a class that builds AbstractClassFactoryClassFactory objects", then that's useless. Also explains why it's 200,000 lines of code, for something that should be an order of magnitude smaller.

Another Java hater? :s

Re: NSA built a NoSQL database

#37
post #30
post #7

" The core codebase consists of 200,000 lines of code (mainly Java) and 100s of pages of documentation." 100s of pages of documentation is a promising start for any open source project.

NSA uses Java. Great to know. Apparently the professionals at the NSA are not swayed by fashion.

They use more "fashionable" languages too. Just not for distributed databases. Most popular open source databases are also written in C/C++/Java.

Re: NSA built a NoSQL database

#38
post #19

Earlier quoted context omitted.

I don't necessarily think there will be one, but I wouldn't be surprised either. Security flaws can be extremely subtle and 200,000 lines of code is a lot to review... Given that there's plausible deniability (we didn't do it intentionally, it was a genuine bug!), if you were them, wouldn't it at least cross your mind to try it? Also, at some point, if it becomes popular, some sysadmin at a large foreign government a…

But if there is a back door, doesn't releasing it as open source open the possibility that China's or Iran's equivalent of the NSA will audit the code and find it too?

In which case the NSA say "Oops, it was a genuine mistake. Sorry." With 200,000 lines of code, there will almost certainly be unintentional security holes that haven't been found.

Re: NSA built a NoSQL database

#39
post #13
post #12

Earlier quoted context omitted.

C'mon Zed, really? a) The code will be open source - the community can verify the code for anything untoward b) Given the nature of the product, most implementations are going to be behind a firewall anyway, with the storage layer talking to business logic. Even if there was a backdoor, and I'm sure there isn't, not sure how NSA could get in. Do you think there's a backdoor in NSA's open-source algorithm for SHA-1 to…

If there are plenty of good uses for the code, I'd still want to improve it, even if I find out it's used by the Kitten Krusher 3000. Unless a "please don't use this code for evil" license is legally binding, that's just the nature of open source.

A "please don't use this code for evil" license would, by definition, not be open-source. (Also, such a license would almost certainly be ignored by evildoers.)

Re: NSA built a NoSQL database

#40
post #16

Earlier quoted context omitted.

I would imagine that this is similar to other ACL products in which the NSA has previously expressed interest, like SELinux. The "labeling" probably means setting permission levels.

"There is a risk that Accumulo will be criticized for not providing adequate security. The access labels in Accumulo do not in themselves provide a complete security solution, but are a mechanism for labeling each piece of data with the authorizations that are necessary to see it."

I'm guessing that the idea is to make it easy to enforce permissions at the application layer. You give permissions, and you get only cells that the current query-er is allowed to see. With HBase, it would be pretty easy to put permissions by the row (add a permission column, or column family if it's complicated enough), but if you want some columns in a row to have some permissions and some to have different ones, it would get unpleasant and inefficient fast.

And regardless, all of the filtering would have to occur at the application layer, meaning you'd have to wrap every get/scan to have it do the filtering for you. The Accumulo way also gets you some efficiency because it never even has to transfer the cells that get filtered by the permissions (or even fully read their content from disk, possibly).

Even though each cell isn't separately encrypted to get you true security at the cell level (which would destroy your performance, I'd guess), this seems like a huge win if you want to have permissions at the cell level.

Post reply on HN