Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NSA built a NoSQL database

wiki.apache.org

Post reply on HN
Print view View original

11–20 of 57 posts

Re: NSA built a NoSQL database

#11

Post by cookiecaper »

nirvana wrote: post #2

It seems that the tags for cells seems to be an important feature of this database, and they also mention it is appropriate for places where "privacy is important". Can someone explain the connection between these two? If I'm understanding right, the labeling makes it easy to address individual cells, but I'm not sure how that enhances privacy.

I would imagine that this is similar to other ACL products in which the NSA has previously expressed interest, like SELinux. The "labeling" probably means setting permission levels.

Re: NSA built a NoSQL database

#12

Post by dotBen »

zedshaw wrote: post #10

Prediction: There will be a backdoor.

C'mon Zed, really?

a) The code will be open source - the community can verify the code for anything untoward

b) Given the nature of the product, most implementations are going to be behind a firewall anyway, with the storage layer talking to business logic. Even if there was a backdoor, and I'm sure there isn't, not sure how NSA could get in.

Do you think there's a backdoor in NSA's open-source algorithm for SHA-1 too?

I applaud the government for putting tax dollars back into open source. My only gripe is the lack of transparency as to what this is primarily used for within the NSA (to be expected I guess). I generally like to know what I'm helping commit code to go do - although granted you have no idea what other open source projects are used for regardless of whether the lead sponsor is government or private company.

Re: NSA built a NoSQL database

#13

Post by seabee »

dotBen wrote: post #12
zedshaw wrote: post #10

Prediction: There will be a backdoor.

C'mon Zed, really? a) The code will be open source - the community can verify the code for anything untoward b) Given the nature of the product, most implementations are going to be behind a firewall anyway, with the storage layer talking to business logic. Even if there was a backdoor, and I'm sure there isn't, not sure how NSA could get in. Do you think there's a backdoor in NSA's open-source algorithm for SHA-1 to…

If there are plenty of good uses for the code, I'd still want to improve it, even if I find out it's used by the Kitten Krusher 3000.

Unless a "please don't use this code for evil" license is legally binding, that's just the nature of open source.

Re: NSA built a NoSQL database

#14

Post by pipeline_tux »

dotBen wrote: post #12
zedshaw wrote: post #10

Prediction: There will be a backdoor.

C'mon Zed, really? a) The code will be open source - the community can verify the code for anything untoward b) Given the nature of the product, most implementations are going to be behind a firewall anyway, with the storage layer talking to business logic. Even if there was a backdoor, and I'm sure there isn't, not sure how NSA could get in. Do you think there's a backdoor in NSA's open-source algorithm for SHA-1 to…

I don't necessarily think there will be one, but I wouldn't be surprised either.

Security flaws can be extremely subtle and 200,000 lines of code is a lot to review... Given that there's plausible deniability (we didn't do it intentionally, it was a genuine bug!), if you were them, wouldn't it at least cross your mind to try it?

Also, at some point, if it becomes popular, some sysadmin at a large foreign government agency or company will forget to firewall off a box running it (ignoring that they could also be connecting back directly - automatic updates anyone?)

Re: NSA built a NoSQL database

#15

Post by adnam »

dotBen wrote: post #12
zedshaw wrote: post #10

Prediction: There will be a backdoor.

C'mon Zed, really? a) The code will be open source - the community can verify the code for anything untoward b) Given the nature of the product, most implementations are going to be behind a firewall anyway, with the storage layer talking to business logic. Even if there was a backdoor, and I'm sure there isn't, not sure how NSA could get in. Do you think there's a backdoor in NSA's open-source algorithm for SHA-1 to…

I think it was just a joke, chill :)

Re: NSA built a NoSQL database

#16

Post by mjs »

cookiecaper wrote: post #11
nirvana wrote: post #2

It seems that the tags for cells seems to be an important feature of this database, and they also mention it is appropriate for places where "privacy is important". Can someone explain the connection between these two? If I'm understanding right, the labeling makes it easy to address individual cells, but I'm not sure how that enhances privacy.

I would imagine that this is similar to other ACL products in which the NSA has previously expressed interest, like SELinux. The "labeling" probably means setting permission levels.

"There is a risk that Accumulo will be criticized for not providing adequate security. The access labels in Accumulo do not in themselves provide a complete security solution, but are a mechanism for labeling each piece of data with the authorizations that are necessary to see it."

Re: NSA built a NoSQL database

#17

Post by contol-m »

jrockway wrote: post #9
jimbokun wrote: post #7

" The core codebase consists of 200,000 lines of code (mainly Java) and 100s of pages of documentation." 100s of pages of documentation is a promising start for any open source project.

It depends on what the documentation is. If it's 100 pages of "AbstractClassFactoryClassFactoryFactory is a class that builds AbstractClassFactoryClassFactory objects", then that's useless. Also explains why it's 200,000 lines of code, for something that should be an order of magnitude smaller.

Why do you say that has to be order of magnitude smaller? Other BigTable clones like HBase are atleast 100K lines of code, if not more.

Re: NSA built a NoSQL database

#18

Post by taftster »

dotBen wrote: post #12
zedshaw wrote: post #10

Prediction: There will be a backdoor.

C'mon Zed, really? a) The code will be open source - the community can verify the code for anything untoward b) Given the nature of the product, most implementations are going to be behind a firewall anyway, with the storage layer talking to business logic. Even if there was a backdoor, and I'm sure there isn't, not sure how NSA could get in. Do you think there's a backdoor in NSA's open-source algorithm for SHA-1 to…

"My only gripe is the lack of transparency as to what this is primarily used for within the NSA (to be expected I guess)."

It's likely just used exactly how you think it would be; to hold massive amounts of key/value data. No doubt, the NSA likely has tons of data to work with. A NoSQL approach would be seemingly beneficial for this use case.

Re: NSA built a NoSQL database

#19

Post by dmoney »

pipeline_tux wrote: post #14
dotBen wrote: post #12

Earlier quoted context omitted.

C'mon Zed, really? a) The code will be open source - the community can verify the code for anything untoward b) Given the nature of the product, most implementations are going to be behind a firewall anyway, with the storage layer talking to business logic. Even if there was a backdoor, and I'm sure there isn't, not sure how NSA could get in. Do you think there's a backdoor in NSA's open-source algorithm for SHA-1 to…

I don't necessarily think there will be one, but I wouldn't be surprised either. Security flaws can be extremely subtle and 200,000 lines of code is a lot to review... Given that there's plausible deniability (we didn't do it intentionally, it was a genuine bug!), if you were them, wouldn't it at least cross your mind to try it? Also, at some point, if it becomes popular, some sysadmin at a large foreign government a…

But if there is a back door, doesn't releasing it as open source open the possibility that China's or Iran's equivalent of the NSA will audit the code and find it too?
Post reply on HN