This pretty much sums up most Hadoop projects I've seen at startups:
http://wavii.files.wordpress.com/2011/12/hadoop_too_big.jpg
(Originally from this Wavii Engineering blog post http://blog.wavii.com/2011/12/29/your-mileage-may-vary/ )
Let’s Talk Your Data, Not Big Data
11–20 of 23 posts
Re: Let’s Talk Your Data, Not Big Data
#12The wild confusion around "big data" stems in part from the fact that many people use it to mean something unrelated to data size. I was recently in a back-and-forth on twitter about this. Some people argued that "big data" refers to the complexity of the analysis or the value of the insight, rather than the size of the data. Kaggle CEO Anthony Goldbloom advocated for a definition "too big to fit in an excel spreadsh…
And here I thought it had to do with the size of the companies involved. "Big Pharma", "Big Tobacco", "Big Oil", "Big Blue" and so on, and now there's "Big Data".
The presumption is that big companies have a lot of data, but not necessarily. They may just have unique data needs.
Re: Let’s Talk Your Data, Not Big Data
#13I just read this Wired article and now I'm wondering whether or not I just read an advertisement for Chartio?
Re: Let’s Talk Your Data, Not Big Data
#14I just read this Wired article and now I'm wondering whether or not I just read an advertisement for Chartio?
Yes .... Yes you did (and I'm down voting the parent for that very reason).
P.S. If I could down vote twice, I'd give it a second one for turning a sentence into an article.
Re: Let’s Talk Your Data, Not Big Data
#15A former Yahoo big data engineer here. At some point I was bombarded with enquires from different startups who were looking for a "Big Data Guy". The conversation would go as:
- (Me) So, how much data are we talking about?
- (them) 50GB
- Per hour?
- No thats our dataset so far.
- (pause)
- (pause)
(FIN)Re: Let’s Talk Your Data, Not Big Data
#16The wild confusion around "big data" stems in part from the fact that many people use it to mean something unrelated to data size. I was recently in a back-and-forth on twitter about this. Some people argued that "big data" refers to the complexity of the analysis or the value of the insight, rather than the size of the data. Kaggle CEO Anthony Goldbloom advocated for a definition "too big to fit in an excel spreadsh…
I argue for your definition: "Big data is when the size of the data becomes part of the problem." Mike Loukides (O'Reilly) mentioned to me that Roger Magoulas (O'Reilly) was the first he heard using that definition. According to this definition, physicists in the 80's were doing big data.
People in science and commerce know data issues and have been dealing with the practice, analysis and theory for years.
O'Reilly's just pimpin' the term "Big Data" to sell books. They did the same with "Web 2.0", Java and the initial internet boom.
Re: Let’s Talk Your Data, Not Big Data
#17The only definition of big data I've thought was remotely insightful is "too large to process with one machine."
Note (as the example in the Wired article indicates) that the converse isn't true: just because you are using multiple machines to process data doesn't mean it's big.
Re: Let’s Talk Your Data, Not Big Data
#18Earlier quoted context omitted.
I argue for your definition: "Big data is when the size of the data becomes part of the problem." Mike Loukides (O'Reilly) mentioned to me that Roger Magoulas (O'Reilly) was the first he heard using that definition. According to this definition, physicists in the 80's were doing big data.
People in science and commerce know data issues and have been dealing with the practice, analysis and theory for years. O'Reilly's just pimpin' the term "Big Data" to sell books. They did the same with "Web 2.0", Java and the initial internet boom.
O Reilly may be pimping the term to developers, but that's nothing compared to how Cloudera has pimped "Big Data" to enterprise.
Re: Let’s Talk Your Data, Not Big Data
#19we are company, main products Wigs,False Eyelashes,Remy Hair Extensions,Indian Remy Hair Extensions,Stock Hair http://www.allbeautybox.com Piece,Print Hair Extensions,Fashion Jewelry..... they are 100% original and brand new with cheap price.
the our company have different kinds, there have males, females also girls needing, products are suit for different groups of people, maybe there is something you are just waiting for.
To some people love jewelrys, sure this is the good news for you, our company because our company Remy Hair Extensions is a manufactures, so our company can be lace front wigs sure you are the only one to be focus on the world, the most False Eyelashes important, these are the jewelrys and fashion and design.
our company also have the Do you feeling amazing, then go and action, our company will take best services waiting for Fashion Jewelrys you !
Important!!: our company accept safe payment, like Paypal.... do fast shipping , sure we protect our clients privacy, make order to our company, you get the safe business and privacy !!!!!
Re: Let’s Talk Your Data, Not Big Data
#20At Yammer we used to have a phrase that analytics was "domain knowledge + counting". Lately we've moved to "analytics is workflow" which better captures the deep integration of data, analysis and business decisions that we have now. I still think the first thing matters a lot. You don't necessarily need fancy technology. If you can count and know everything about your business and what makes it valuable then you can start building real insights.