Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ways Data Projects Fail

martingoodson.com

Post reply on HN
Print view View original

21–30 of 79 posts

Re: Ways Data Projects Fail

#21

Post by kpmcc »

Declanomous wrote: post #12

>Your Data Scientists are about to quit. This is me. I work for a non-profit that is stuck in the stone age--not for lack of money, mind you, but because the IT Director is an incompetent megalomaniac who views "security" as a reasonable justification to refuse any and all requests, and treats everyone like an enemy. I haven't been allowed to use Python or R. In fact, the only programming language I have access to is…

Is the nonprofit you work for affiliated with a particular University on the south side?

I can sympathize with having to deal with VBA. I'm working in a lab that deals with lots of questionnaire data and uses Access as the main tool for gathering said data because that's the way things have been done in the past, despite the fact that nobody in the lab really knows how it works.

I can also sympathize with having your network drives go down and render everything inoperable. Everything we use in the lab is stored on an offsite network drive, probably because of HIPAA compliance, and said network drive has dropped out twice in the past couple weeks. Once for almost an entire day, and once for an hour or two.

Best of luck with the job search.

Re: Ways Data Projects Fail

#22

Post by martingoodson »

toxikitty_ wrote: post #17
martingoodson wrote: post #10

Author here - in case of any criticisms or comments.

A lot of data scientists these days (me included) are former academics with backgrounds in numerical simulation in fields like chemistry, physics, mechanical engineering etc. They live and breath numerical linear algebra and are comfortable reading advanced theoretical books or papers. It's easy for them to pick up the basics needed to pass interviews and find a data science job. How would they go about adding some r…

I personally love this for statistical inference: http://www.springer.com/cn/book/9780387402727

and this for statistical ML: http://statweb.stanford.edu/~tibs/ElemStatLearn/

Re: Ways Data Projects Fail

#23

Post by danso »

Absolutely agree that data cleaning should be at the top -- how someone prioritizes data cleaning is for me, the main litmus test to how effective they are at real-world data problems. I also agree with how the author summarizes the issue, but he also runs into the same issue I have: data cleaning is such a broad term that it obscures how difficult and important of a problem it is.

For example, some people think data cleaning is "Convert 12-FEB-2012 to 2016-02-12" type problems, and can't believe that such a task would be 80 to 90% of the difficulty in data work (compared to say, learning enough ggplot2 to make a nice chart).

On the other side of the equation, you have people who want to do a JOIN-GROUP-BY aggregate so they can calculate how much "evil" Wall Street money goes to each political candidate, a la OpenSecrets's calculation [0], only to find that the FEC does not classify campaign contributions by industry type or company, nor is the "employer" field filled with normalized entries such as "Evil Wall Street Company" that would lend itself to easy GROUP BY calls. For fucks sake, I've found that executive-level/professor folks can't even spell "Goldman Sachs" and "Berkeley" correctly (even on a typed form)

And that doesn't even scratch the surface of how little this person knows about the data question the purport to answer, or about how the FEC, the American political system, and real life works. Among the data cleaning problems they will have to mitigate are also the 2 hardest problems in computer science (how things are named/classified, and how up-to-date the data is).

I don't have any better ideas at the moment for how to break apart the category of "data cleaning" that reveals the many facets of the problem but also still preserves the interelatedness of the facets. But it's possible to be very good at some of the parts of data cleaning without knowing the rest.

[0] https://www.opensecrets.org/industries/indus.php?Ind=F

[1] http://www.fec.gov/finance/disclosure/metadata/DataDictionar...

Re: Ways Data Projects Fail

#24

Post by Declanomous »

kpmcc wrote: post #21
Declanomous wrote: post #12

>Your Data Scientists are about to quit. This is me. I work for a non-profit that is stuck in the stone age--not for lack of money, mind you, but because the IT Director is an incompetent megalomaniac who views "security" as a reasonable justification to refuse any and all requests, and treats everyone like an enemy. I haven't been allowed to use Python or R. In fact, the only programming language I have access to is…

Is the nonprofit you work for affiliated with a particular University on the south side? I can sympathize with having to deal with VBA. I'm working in a lab that deals with lots of questionnaire data and uses Access as the main tool for gathering said data because that's the way things have been done in the past, despite the fact that nobody in the lab really knows how it works. I can also sympathize with having your…

It is not. I've heard working there is hit or miss depending on who your boss is.

I don't want to hate too much on Access. It's really an amazing program. You should see the processes some of my coworkers managed to create despite not knowing an iota of SQL.

Thanks for the well-wishes. I'm being very methodical because I really want to make the move count. I'm okay with waiting for the right opportunity because I really enjoy my job outside of the horrible IT situation: my boss is awesome, the people I work with are awesome, there is a lot of variety in the role, and I get to make a lot of decisions. Regardless, the IT situation is limiting my growth, so I'm on the lookout for the next thing.

Re: Ways Data Projects Fail

#25

Post by throw_away_777 »

danso wrote: post #23

Absolutely agree that data cleaning should be at the top -- how someone prioritizes data cleaning is for me, the main litmus test to how effective they are at real-world data problems. I also agree with how the author summarizes the issue, but he also runs into the same issue I have: data cleaning is such a broad term that it obscures how difficult and important of a problem it is. For example, some people think data…

I've always disliked the term data cleaning for the reasons you mention - it doesn't tell me anything about what is meant by "cleaning".

Re: Ways Data Projects Fail

#26

Post by eanzenberg »

_delirium wrote: post #4
jackschultz wrote: post #2

Good to see that data cleaning was #1 on that list. Whenever I do work on a side project, it takes way way more time to get and structure the data than it does running the algorithms. Granted, that's because I have to go out and get the data in the first place, and then make sure it's useable and in the correct format. Like the recent project I'm doing trying to classify country music songs based on their topic on th…

I agree, and would add: data cleaning's importance to the quality of the result is also often underemphasized compared to the much bigger focus on the quality of the algorithms. A single bad decision on data cleaning can have a large effect on the end result (in many cases, more than choosing between algorithms, assuming you pick some vaguely reasonable algorithm). Especially any choice that ends up producing non-ran…

Absolutely, having data that improves signal-noise will always trump a better algorithm.

Re: Ways Data Projects Fail

#27

Post by eanzenberg »

jackschultz wrote: post #2

Good to see that data cleaning was #1 on that list. Whenever I do work on a side project, it takes way way more time to get and structure the data than it does running the algorithms. Granted, that's because I have to go out and get the data in the first place, and then make sure it's useable and in the correct format. Like the recent project I'm doing trying to classify country music songs based on their topic on th…

>> I've been looking for jobs recently, and I've seen only one job posting that mentions data cleaning as a necessity, whereas the rest only talk about data science and algorithm knowledge, or overall ETL design on the data engineering side. Seems like data set knowledge should be emphasized more.

Actual data cleaning, usually in an automated sense, is more 'data engineering' than 'data science' or applied statistics. Feature engineering and 'massaging' training data is more related to DS but it's understood that this data being consumed by the DS is already in decent shape.

Re: Ways Data Projects Fail

#28

Post by throw_away_777 »

martingoodson wrote: post #10

Author here - in case of any criticisms or comments.

I think point 7) needs work. Often times people use words like interpretable to avoid having to think about the data - usually in the context of linear or logistic regression. The model seems "interpretable" because the coefficients are "meaningful" - but often times the model is just as much a black box as other models, for instance regression coefficients depend on what other features you include and the scale of those features. Similarly regression p-values are very easy to misinterpret. I think you should use the data to determine what the model is doing regardless of the model you use.

In summary, it is important to iterate quickly and to validate your results. Using complex models, like gradient boosted decision trees, can often iterate much more quickly than simple models because you don't have to do extensive data preparation. Many analysts are stuck in the mode of using linear or logistic regression for every problem, when there are better tools out there.

Re: Ways Data Projects Fail

#29

Post by eanzenberg »

d--b wrote: post #5

The big one that's missing: There is nothing you can conclude from your data. It's clean, it makes its way properly to the analyst, and yet, there's just nothing there...

If done properly that's still a useful result. "None of the variables were predictive of X."

Re: Ways Data Projects Fail

#30

Post by searine »

TeMPOraL wrote: post #19
searine wrote: post #6

Earlier quoted context omitted.

Hypothesis driven data science is over the head of too many people.

That's why it's called "data mining" - you keep digging until you find something. And with lots of data, you can always find something if you look hard enough - which leads to things like http://tylervigen.com/discover .

We always called it a fishing expedition.

Though, that's what has always appealed to me about truly large data sets, low risk of turing up jack-squat.

Post reply on HN