Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Surviving Data Science at the Speed of Hype

john-foreman.com

Post reply on HN
Print view View original

11–20 of 48 posts

Re: Surviving Data Science at the Speed of Hype

#11

Post by mwetzler »

I'm a data scientist that works with companies on their analytics problems every day. This article is spot on.

By far the biggest factor influencing the success of an analytics project is that the company has a human who has the time and inclination to think and reason about the business. They figure out what questions are important to ask and then go look at the data to see what they find. Collecting the data is the easy part. There is no analytics product that asks & answers your most important business questions for you.

I enjoyed the jab at predictive modeling; it's almost comical how many companies dream about predictive when they haven't yet got basic tracking in place for what's _already_ happening in their business.

Love the post, thanks for sharing.

Re: Surviving Data Science at the Speed of Hype

#12

Post by cwyers »

dchuk wrote: post #5

I just love the way this guy writes. His book, Data Smart, is hands down the most approachable intro to data science you could ever possibly read if you don't have the sufficient math background to dive into full on textbooks. And it's hilarious too.

It looks like an interesting book. Anyone have experience with the ebook version? Does it hold up with the illustrations, or should I hold out for a hardcopy?

Re: Surviving Data Science at the Speed of Hype

#13

Post by dchuk »

cwyers wrote: post #12
dchuk wrote: post #5

I just love the way this guy writes. His book, Data Smart, is hands down the most approachable intro to data science you could ever possibly read if you don't have the sufficient math background to dive into full on textbooks. And it's hilarious too.

It looks like an interesting book. Anyone have experience with the ebook version? Does it hold up with the illustrations, or should I hold out for a hardcopy?

I have the ebook version, it's fine if you're ok with ebooks for this sort of stuff.

Re: Surviving Data Science at the Speed of Hype

#14

Post by tbjohns »

An important distinction is that the author's experience is mostly with the businessy side of data science, and his jab is at people who use buzzword tools that add complexity rather than simple solutions.

In defense of the hype, many tools like storm are worth their hype many times over when used for the right application.

The author makes this distinction, but it can easily be lost in the post.

Re: Surviving Data Science at the Speed of Hype

#15

Post by strictnein »

dchuk wrote: post #10
strictnein wrote: post #8

Earlier quoted context omitted.

I have mixed feelings about that book. I enjoyed his writing style and humor, but the amount of beating on Excel he has to do to manipulate all that data hurts my head. I kept thinking of how much easier it would be to do with code. Maybe it's just not a good book for developers? shrug I would love to have a copy of that book that doesn't use Excel.

I share your same yearning for a code equivalent of the book. However, I think writing the book using only Excel was a smart move on his part, simply because: 1) Excel is "visual" in the sense that you can watch the data change as you tweak things. There is no command line or program to execute, it's all happening live 2) For programmers, there's no "well I'm a python guy and this book is written in Java so it's not…

I agree with both your points. For #2, the only decent option may to make the book more focused on R, instead of just chapter 10.

Re: Surviving Data Science at the Speed of Hype

#16

Post by trhway »

>And that is not primarily a tool problem.

>A lot of vendors want to cast the problem as a technological one. That if only you had the right tools then your analytics could stay ahead of the changing business in time for your data to inform the change rather than lag behind it.

many people like the author just don't get it and it is fine. The same way like people didn't get the search before Google.

>But how do I feel good about my graduate degree if all I'm doing is pulling a median?

the graduate degree is what allows to receive $Nx10e5/year (for a respectable value of N) for that pulling of a median

>If your goal is to positively impact the business, not to build a clustering algorithm that leverages storm and the Twitter API, you'll be OK.

on the other hand if your goal is power(OK, OK) instead of just OK then the clustering algorithm/storm/twitter is the way to go.

Re: Surviving Data Science at the Speed of Hype

#17

Post by LargeWu »

I once worked at a major big box retailer where somebody came up with a visualization that purported to show, for a given product category, purchases made in other categories. One surprising purchase correlation was customers bought TV stands after buying DVD players. So, this nugget was trumpeted at countless meetings about the value of big data analytics. Multiple marketing campaigns were designed around this discovery.

Of course, that made no sense, so I checked a little deeper. You know what else people also buy when they buy DVD players? TV's. The DVD/furniture relationship was an artifact of the high degree of correlation between TV's and DVD players, which the visualization tool failed to account for.

I brought this up immediately, but received tepid response. Of course, months later, I was still hearing about DVD players and furniture. It had become part of the institutional lore, and no facts were going to replace that.

Re: Surviving Data Science at the Speed of Hype

#18

Post by threeseed »

Firstly, someone needs to explain to me why smart people get worked up over vendor marketing. Since the beginning of time it has always been about exaggerated claims, bold, specific numbers e.g. 80% better and always targets those who make purchasing decisions. Do people really expect them to say, "Hey our product is great but you know you probably don't need it. But maybe buy it anyway ?".

Secondly, the author seems to have conflated two different parts of the data science picture. Yes great analysts who do amazing work is important. But it relies on (a) having data available and (b) in the right format. For those of us doing significant volume ingestions it is not trivial to do this. Hadoop is painfully slow and overall data science end to end tooling is slow, fragmented and incomplete. Some of us do need vendors to be bold and coming up with new technologies/approaches.

And the point about IBM is just stupid. Did you ever think that maybe Watson DID help them slow their sales losses ? Weird that a data scientist would make predictions based on inadequate data.

Re: Surviving Data Science at the Speed of Hype

#19

Post by mathattack »

mwetzler wrote: post #11

I'm a data scientist that works with companies on their analytics problems every day. This article is spot on. By far the biggest factor influencing the success of an analytics project is that the company has a human who has the time and inclination to think and reason about the business. They figure out what questions are important to ask and then go look at the data to see what they find. Collecting the data is the…

Exactly - the human with domain knowledge is vital. I get scared when I see people trump up black boxes. Black boxes don't help with "Which questions should we be asking?" and "What are the missing variables?"

Re: Surviving Data Science at the Speed of Hype

#20

Post by mfdupuis »

Very good post. Refreshing.

I think that the hype and buzzwords around Big Data and data science cause more than just bad business decisions. I believe they are also damaging the industry and creating a larger sense of disillusionment (I'm mostly thinking of "deep learning"). Not sure what this means for data science in the long term though, just thinking out loud.

I'll also add that I frequently see sledge hammers being used to hang a picture frame. By that I mean using huge clusters to run algos that would actually run in Tableau, Excel etc.

Post reply on HN