NSA built a NoSQL database • Hacker News Board

Re: NSA built a NoSQL database

#41

Post by samstave » Sun, Sep 04, 2011, 4:55 AM UTC

Some time ago I met some people from the .gov cyber security, NSA and other offices. The head of the .gov office on cyber security was really nice and invited me to go to dinner with them.

The guy from the NSA was hands down the biggest evil piece of shit I have ever experienced in my life. The way he talked, what he said, and the fact that he was given free reign to commit crimes in his training, which he openly bragged about, made me want to murder the guy right then and there.

I lost any and all respect for what the government and the NSA do.

Re: NSA built a NoSQL database

#42

Post by spullara » Sun, Sep 04, 2011, 6:30 AM UTC

jimbokun wrote: post #7

" The core codebase consists of 200,000 lines of code (mainly Java) and 100s of pages of documentation." 100s of pages of documentation is a promising start for any open source project.

Government & consulting projects generally require the code and the use of the code to be fully documented. I'm actually looking forward to that part of it more than the software if it is any good.

Re: NSA built a NoSQL database

#43

Post by vegai » Sun, Sep 04, 2011, 6:49 AM UTC

mypov wrote: post #30

jimbokun wrote: post #7

" The core codebase consists of 200,000 lines of code (mainly Java) and 100s of pages of documentation." 100s of pages of documentation is a promising start for any open source project.

NSA uses Java. Great to know. Apparently the professionals at the NSA are not swayed by fashion.

That's a ridiculous defense: they were obviously swayed by fashion at some point. It just happened several years ago.

Re: NSA built a NoSQL database

#44

Post by vegai » Sun, Sep 04, 2011, 6:51 AM UTC

jimbokun wrote: post #7

" The core codebase consists of 200,000 lines of code (mainly Java) and 100s of pages of documentation." 100s of pages of documentation is a promising start for any open source project.

200k. Why do Java programs always balloon up?

Re: NSA built a NoSQL database

#45

Post by SamReidHughes » Sun, Sep 04, 2011, 6:57 AM UTC

contol-m wrote: post #17

jrockway wrote: post #9

Earlier quoted context omitted.

It depends on what the documentation is. If it's 100 pages of "AbstractClassFactoryClassFactoryFactory is a class that builds AbstractClassFactoryClassFactory objects", then that's useless. Also explains why it's 200,000 lines of code, for something that should be an order of magnitude smaller.

Why do you say that has to be order of magnitude smaller? Other BigTable clones like HBase are atleast 100K lines of code, if not more.

A project I'm working on with a similar level of intricacy as BigTable has way less than 100K, and that's in a language more verbose than Java.

Re: NSA built a NoSQL database

#46

Post by marchdown » Sun, Sep 04, 2011, 8:17 AM UTC

marshray wrote: post #29

zedshaw wrote: post #23

Earlier quoted context omitted.

A joke?! A JOKE?! You jest good sir. I merely put on my tinfoil hat and thought, "Hmmm, didn't this happen to OpenBSD, Windows, every crypto system ever, numerous databases, and probably SELinux?" Then extrapolated out to a very valid point. How dare you claim I am not deadly serious about the NSA putting a back door in a database that is intended to be secure for the internet. How. Dare. You!

I still can't tell if you're joking. I've seen a possible back door or two in this or that, but nothing like "every crypto system ever". If you have evidence of a back door in AES, SHA-2, or anything NIST has standardized (other than Dual_EC_DRBG or openly weakened stuff like export SSL) lots of people would like to hear about it.

Can you elaborate on the "openly weakened stuff" part?

I don't know much about security, but I am vaguely aware that there were some efforts by various governments to control, regulate, weaponize and even outlaw crypto, but I don't know where these effort have left us. Are there any crypto systems with acknowledged backdoors? Are there any which are not only widely considered to be secure, but are known to have actually prevented three-letter agencies from getting their way?

Re: NSA built a NoSQL database

#47

Post by brainlounge » Sun, Sep 04, 2011, 9:30 AM UTC

jrockway wrote: post #9

jimbokun wrote: post #7

" The core codebase consists of 200,000 lines of code (mainly Java) and 100s of pages of documentation." 100s of pages of documentation is a promising start for any open source project.

It depends on what the documentation is. If it's 100 pages of "AbstractClassFactoryClassFactoryFactory is a class that builds AbstractClassFactoryClassFactory objects", then that's useless. Also explains why it's 200,000 lines of code, for something that should be an order of magnitude smaller.

200k lines ... of which 85k lines are generated from Thrift and 10% from other projects, according to related ML discussions.

Re: NSA built a NoSQL database

#48

Post by PLejeck » Sun, Sep 04, 2011, 9:57 AM UTC

Now we know how they store the data gleaned from wiretaps!

Re: NSA built a NoSQL database

#49

Post by oomkiller » Sun, Sep 04, 2011, 10:03 AM UTC

marchdown wrote: post #46

marshray wrote: post #29

Earlier quoted context omitted.

I still can't tell if you're joking. I've seen a possible back door or two in this or that, but nothing like "every crypto system ever". If you have evidence of a back door in AES, SHA-2, or anything NIST has standardized (other than Dual_EC_DRBG or openly weakened stuff like export SSL) lots of people would like to hear about it.

Can you elaborate on the "openly weakened stuff" part? I don't know much about security, but I am vaguely aware that there were some efforts by various governments to control, regulate, weaponize and even outlaw crypto, but I don't know where these effort have left us. Are there any crypto systems with acknowledged backdoors? Are there any which are not only widely considered to be secure, but are known to have actua…

Back in the 90s the US Government prohibited export of SSL stronger than 40 bits. I believe this is what they're referring to.

Re: NSA built a NoSQL database

#50

Post by gaius » Sun, Sep 04, 2011, 11:41 AM UTC

vegai wrote: post #44

jimbokun wrote: post #7

" The core codebase consists of 200,000 lines of code (mainly Java) and 100s of pages of documentation." 100s of pages of documentation is a promising start for any open source project.

200k. Why do Java programs always balloon up?

Zawinski's Law.