Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PRISM: The Amazingly Low Cost of ­Using Big Data to Know More About You

highscalability.com

Post reply on HN
Print view View original

11–20 of 30 posts

Re: PRISM: The Amazingly Low Cost of ­Using Big Data to Know More About You

#11

Post by diminoten »

mtgx wrote: post #5
skwirl wrote: post #2

According to the PRISM slides, the program costs $20 million per year. This article doesn't mention that, although it should, because it is telling. If the author believes that this program could be implemented at a minimum of $187 million per year, then that $20 million claim is problematic. Either the $20 million claim is wrong, and then all the information on the slides is suspect, or it is correct, and the scope…

PRISM is probably just a small part of the all-encompassing spying program.

One of the fundamental issues with these discussions is that things such as what you're saying get thrown into the mix when we're talking about things we've actually seen evidence for.

We've, for a very long time, said things like, "this is probably happening". That is in no way whatsoever a novel idea. What is novel, and why these discussions are happening so frequently now, is that we have evidence that a 20mil/year program is actually happening.

So when we're talking about things we have evidence for, let's please avoid throwing in conjecture.

Re: PRISM: The Amazingly Low Cost of ­Using Big Data to Know More About You

#13

Post by angersock »

droz wrote: post #12

The developer salary costs seem really outlandish. 500k euro for a "top notch developer" and 250k euro for "supporting developers". Where are these estimates coming from?

This is what's referred to as "blood money".

I wouldn't sell out my fellow man for anything less.

Re: PRISM: The Amazingly Low Cost of ­Using Big Data to Know More About You

#14

Post by rdouble »

droz wrote: post #12

The developer salary costs seem really outlandish. 500k euro for a "top notch developer" and 250k euro for "supporting developers". Where are these estimates coming from?

It's the cost of renting a developer with security clearance from BAH. Snowden would likely be in the same salary band as a "supporting developer." He was taking home $122K, thus his actual cost to the government was likely $250-$300K.

Re: PRISM: The Amazingly Low Cost of ­Using Big Data to Know More About You

#15

Post by miahi »

skwirl wrote: post #2

According to the PRISM slides, the program costs $20 million per year. This article doesn't mention that, although it should, because it is telling. If the author believes that this program could be implemented at a minimum of $187 million per year, then that $20 million claim is problematic. Either the $20 million claim is wrong, and then all the information on the slides is suspect, or it is correct, and the scope…

The author counts the storage on a yearly basis (servers to store a year of data). If you allow an expiration date for the records (let's say 4 years), after that period you can spend less on hardware, as you can free space from the old records. Then you only need to spend money on the traffic difference (as the traffic would increase in 4 years).

As the storage boxes in the article also have a nice CPU, the collected data can be indexed and then compressed, saving a lot of space.

Re: PRISM: The Amazingly Low Cost of ­Using Big Data to Know More About You

#16

Post by dreamfactory »

skwirl wrote: post #2

According to the PRISM slides, the program costs $20 million per year. This article doesn't mention that, although it should, because it is telling. If the author believes that this program could be implemented at a minimum of $187 million per year, then that $20 million claim is problematic. Either the $20 million claim is wrong, and then all the information on the slides is suspect, or it is correct, and the scope…

$20M doesn't necessarily include infra (e.g. which is already covered in Utah), could just be program running costs.

Re: PRISM: The Amazingly Low Cost of ­Using Big Data to Know More About You

#17

Post by jlgaddis »

skwirl wrote: post #2

According to the PRISM slides, the program costs $20 million per year. This article doesn't mention that, although it should, because it is telling. If the author believes that this program could be implemented at a minimum of $187 million per year, then that $20 million claim is problematic. Either the $20 million claim is wrong, and then all the information on the slides is suspect, or it is correct, and the scope…

> This is a worst case scenario that does not include potential discounts due to renting such a high volume of hardware and traffic or acquiring the aforementioned hardware (which incurs a higher initial investment but lower recurring costs) .

"worst case scenario" is emphasized in the article.

Re: PRISM: The Amazingly Low Cost of ­Using Big Data to Know More About You

#18

Post by dreamfactory »

Sven7 wrote: post #4
capkutay wrote: post #3

I would assume the government would require services and large teams to help them make sense of the data along with the low the infrastructure described by the article.

I am pretty sure they can't make sense of the data, nor can they afford the sort of talent that might be able to make sense of the data. So they do whats easy...just keep collecting more. It's much more efficient as an employment/pension guarantee scheme than an efficient intelligence tool. Reminds me of the qoute from Yes Minister - Something must be done. This is something. Therefore we must do it.

According to an ex-Stasi chief on these revelations: “It is the height of naivete to think that once collected this information won’t be used”.

http://news.cnet.com/8301-1009_3-57591551-83/ex-stasi-boss-g...

Re: PRISM: The Amazingly Low Cost of ­Using Big Data to Know More About You

#19

Post by samatman »

miahi wrote: post #15
skwirl wrote: post #2

According to the PRISM slides, the program costs $20 million per year. This article doesn't mention that, although it should, because it is telling. If the author believes that this program could be implemented at a minimum of $187 million per year, then that $20 million claim is problematic. Either the $20 million claim is wrong, and then all the information on the slides is suspect, or it is correct, and the scope…

The author counts the storage on a yearly basis (servers to store a year of data). If you allow an expiration date for the records (let's say 4 years), after that period you can spend less on hardware, as you can free space from the old records. Then you only need to spend money on the traffic difference (as the traffic would increase in 4 years). As the storage boxes in the article also have a nice CPU, the collecte…

Given that the Internet grows exponentially year by year, while the cost to store a bit of information drops in a similar fashion, I doubt there is any money to be saved by deleting old data. The save in system complexity is likely to handily outweigh the additional cost of storage, not to mention it isn't worth one iota of frustration and bad reviews if data an analyst wants is not available.

Re: PRISM: The Amazingly Low Cost of ­Using Big Data to Know More About You

#20

Post by sneak »

From TFA:

> Do you think that PRISM can be built using a different tech stack?

From https://en.wikipedia.org/wiki/Apache_Accumulo :

> Apache Accumulo is a sorted, distributed key/value store based on Google's BigTable design. It is a system built on top of Apache Hadoop, Apache ZooKeeper, and Apache Thrift. Written in Java, Accumulo has cell-level access labels and server-side programming mechanisms.

> Accumulo was created in 2008 by the National Security Agency and contributed to the Apache Foundation as an incubator project in September 2011.

Post reply on HN