Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Surviving Data Science at the Speed of Hype

john-foreman.com

Post reply on HN
Print view View original

21–30 of 48 posts

Re: Surviving Data Science at the Speed of Hype

#21

Post by tmarthal »

LargeWu wrote: post #17

I once worked at a major big box retailer where somebody came up with a visualization that purported to show, for a given product category, purchases made in other categories. One surprising purchase correlation was customers bought TV stands after buying DVD players. So, this nugget was trumpeted at countless meetings about the value of big data analytics. Multiple marketing campaigns were designed around this disco…

Whenever you say "that made no sense", I think that you are using too much bias and not giving enough credit to what the data is telling you.

If you look at the most "controversial" data science paper from 2013 where a study correlated intelligence to Liking the Facebook pages "Curly Fries" and "Thunderstorms" (here is a summary: http://www.wired.com/2013/03/facebook-like-research/), there were a lot of proponents saying that there was no causation, and the correlation was not founded, etc.

Of course, you would say the study "makes no sense". Intelligence can't be predicted by Facebook Likes. There is no correlation there, etc. But why not? If you read the paper (http://www.pnas.org/content/110/15/5802.full.pdf) their logic is sound. Is the marketing campaigns that the company bought based on the TV StandDVD Player connection any different than other marketing campaigns? Facebook does all of their ad display based on similar data analysis as above, and it seems to be working for them.

Note: There is the not-so-hidden machine learning feedback loop now (explained better here: http://www.john-foreman.com/blog/the-perilous-world-of-machi...), where people Like the 'Curly Fries' and 'Thunderstorms' pages because of the research.

Re: Surviving Data Science at the Speed of Hype

#22

Post by blumkvist »

LargeWu wrote: post #17

I once worked at a major big box retailer where somebody came up with a visualization that purported to show, for a given product category, purchases made in other categories. One surprising purchase correlation was customers bought TV stands after buying DVD players. So, this nugget was trumpeted at countless meetings about the value of big data analytics. Multiple marketing campaigns were designed around this disco…

How are association rules "big data analytics"?

The article is very refreshing and I bookmarked the site. What I am more frustrated with is that a lot of people use this stupid term "big daata" for things which do not fit the description. If it's structured, it's not big data. If it comes at 2MB/s it's not big data. If it fucking fits in your RAM, it most certainly is not big data.

Re: Surviving Data Science at the Speed of Hype

#23

Post by dagw »

mathattack wrote: post #19
mwetzler wrote: post #11

I'm a data scientist that works with companies on their analytics problems every day. This article is spot on. By far the biggest factor influencing the success of an analytics project is that the company has a human who has the time and inclination to think and reason about the business. They figure out what questions are important to ask and then go look at the data to see what they find. Collecting the data is the…

Exactly - the human with domain knowledge is vital. I get scared when I see people trump up black boxes. Black boxes don't help with "Which questions should we be asking?" and "What are the missing variables?"

Domain knowledge is also really useful for spotting bugs. I recently worked on a project where I had very little domain knowledge. So anyway I wrote my code, ran my tests, crunched the data, double checked that all the results seemed reasonable, produced the pretty pictures and everything looked spot on. However once I started showing the results to a domain expert it took him 30 seconds to point to one of the outputs and go "that's impossible, you have a bug in your code". Sure enough I did. As a generalist the results looked fine to me (right size, seemingly reasonable relationship to surrounding values etc.), but to a domain expert the error stuck out like sore thumb.

Re: Surviving Data Science at the Speed of Hype

#24

Post by threeseed »

blumkvist wrote: post #22
LargeWu wrote: post #17

I once worked at a major big box retailer where somebody came up with a visualization that purported to show, for a given product category, purchases made in other categories. One surprising purchase correlation was customers bought TV stands after buying DVD players. So, this nugget was trumpeted at countless meetings about the value of big data analytics. Multiple marketing campaigns were designed around this disco…

How are association rules "big data analytics"? The article is very refreshing and I bookmarked the site. What I am more frustrated with is that a lot of people use this stupid term "big daata" for things which do not fit the description. If it's structured, it's not big data. If it comes at 2MB/s it's not big data. If it fucking fits in your RAM, it most certainly is not big data.

What on earth are you talking about ?

(a) Association rules are big data when you are doing them on large data sets with many variables. I work at a company that sells tens of thousands of different products and tens of millions of customers. Definitely takes us a while to compute those rules.

(b) The majority of big data is structured. For most big data projects it is typically stored in old school Oracle/Teradata/etc data warehouses and shipped into a Hadoop cluster. It may not be consolidated but it is definitely structured.

(c) The total RAM of our Hadoop cluster is 4TB and ours is small. I would consider that to be big data in the sense that it overwhelms any applications that directly try to access the raw data.

Re: Surviving Data Science at the Speed of Hype

#25

Post by edraferi »

threeseed wrote: post #18

Firstly, someone needs to explain to me why smart people get worked up over vendor marketing. Since the beginning of time it has always been about exaggerated claims, bold, specific numbers e.g. 80% better and always targets those who make purchasing decisions. Do people really expect them to say, "Hey our product is great but you know you probably don't need it. But maybe buy it anyway ?". Secondly, the author seems…

You are 100% correct that data availability is always the first problem to solve. However, I think this is addressed indirectly in his thesis that advanced analytics are brittle in a rapidly changing business. Any change that breaks your data by definition breaks your models.

Re: Surviving Data Science at the Speed of Hype

#26

Post by [deleted] »

trhway wrote: post #16

>And that is not primarily a tool problem. >A lot of vendors want to cast the problem as a technological one. That if only you had the right tools then your analytics could stay ahead of the changing business in time for your data to inform the change rather than lag behind it. many people like the author just don't get it and it is fine. The same way like people didn't get the search before Google. >But how do I fee…

[deleted]

Re: Surviving Data Science at the Speed of Hype

#27

Post by kiyoto »

First of all, John Foreman is great. Read his book "Data Smart" and http://analyticsmadeskeezy.com/blog/

(disclaimer: I am in no way tied to John Foreman. Also, I work at a company that provides a data processing/collaboration SaaS...for big data! http://www.treasuredata.com)

A quote from the OP:

>If your business is currently too chaotic to support a complex model, don't build one. Focus on providing solid, simple analysis until an opportunity arises that is revenue-important enough and stable enough to merit the type of investment a full-fledged data science modeling effort requires.

This is consistent with what we see in our customers. The use cases we see most with processing big data boils down to generating reports.

Generating reports may sound really prosaic, but as I learned from our customers, most organizations are very, very far from providing access to their data in a cogent, accessible manner. Just to generate reports/summaries/basic descriptive statistics, incredibly complex enterprise architectures have been proposed, built by a cadre of enterprise architects and deployed with obscenely high maintenance subscription fees billed by various vendors. That's the reality at many companies.

As bad and confusing the buzzword "big data" is, one good byproduct is that it has forced slow-moving enterprises to rethink their data collection/storage/management/reporting systems.

Finally, I am starting to see folks do meaningful predictive modelling on top of large-ish data (in the order of terabytes). Some of them are our customers at Treasure Data, some aren't, but they are definitely not "build[ing] a clustering algorithm that leverages storm and the Twitter API" but actually doing the hard work of thinking through how (or if) the data they collect is meaningful and useful.

And that's a good thing.

Re: Surviving Data Science at the Speed of Hype

#28

Post by joe_the_user »

tmarthal wrote: post #21
LargeWu wrote: post #17

I once worked at a major big box retailer where somebody came up with a visualization that purported to show, for a given product category, purchases made in other categories. One surprising purchase correlation was customers bought TV stands after buying DVD players. So, this nugget was trumpeted at countless meetings about the value of big data analytics. Multiple marketing campaigns were designed around this disco…

Whenever you say "that made no sense", I think that you are using too much bias and not giving enough credit to what the data is telling you. If you look at the most "controversial" data science paper from 2013 where a study correlated intelligence to Liking the Facebook pages "Curly Fries" and "Thunderstorms" (here is a summary: http://www.wired.com/2013/03/facebook-like-research/ ), there were a lot of proponents s…

Whenever you say "that made no sense", I think that you are using too much bias and not giving enough credit to what the data is telling you.

What? If a data scientist sees something seems illogical, there is no reason not to investigate it and see if he/she can find a more logical explanation. Sure, if the effect seems real but unexplained, you can accept and use it but advocating a kind of big data mysticism, "don't investigate, accept" seems to be buying into the senseless hype. And if you read the post, you'll notice the parent actually discovered the association was just an artifact of an easily explained association.

And, no, there's no much reason for companies to advertise just a TV stand and DVD player. Common sense tells one what the data actually data, that those two items, by themselves aren't and weren't what many people were just dreaming about.

Re: Surviving Data Science at the Speed of Hype

#29

Post by IndianAstronaut »

mfdupuis wrote: post #20

Very good post. Refreshing. I think that the hype and buzzwords around Big Data and data science cause more than just bad business decisions. I believe they are also damaging the industry and creating a larger sense of disillusionment (I'm mostly thinking of "deep learning"). Not sure what this means for data science in the long term though, just thinking out loud. I'll also add that I frequently see sledge hammers b…

I had a conversation with a 'big data consultant' some time back. He mentioned one of his clients needed to set up a Hadoop cluster and wanted me to work with him. I said, 'why do they need a cluster, they probably don't have that much data'. His response was 'if a client wants to jump off a building, you don't say don't do it, you ask them what floor'.

Re: Surviving Data Science at the Speed of Hype

#30

Post by igrekel »

mwetzler wrote: post #11

I'm a data scientist that works with companies on their analytics problems every day. This article is spot on. By far the biggest factor influencing the success of an analytics project is that the company has a human who has the time and inclination to think and reason about the business. They figure out what questions are important to ask and then go look at the data to see what they find. Collecting the data is the…

Not doing data analytics but selling software that has forecasting with a model that we build and calibrate. We have fairly good performance, recalibrating the same model tales a few seconds but building the model or changing it is never quick.

The effect of these marketing campaigns on would be clients is terrible. They start going after crazy crackpot solutions to gain revenue while they haven't addressed the simplest easy to reach low risk revenue gains. In a a lot of cases integrating complex side effect data costs a lot and provides only marginal revenue gains.

Post reply on HN