Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Data Science: Reality Doesn't Meet Expectations

dfrieds.com

Post reply on HN
Print view View original

161–168 of 168 posts

Re: Data Science: Reality Doesn't Meet Expectations

#161

Post by Barrin92 »

itsmefaz wrote: post #129
Barrin92 wrote: post #110

I'm generally confused by the hype around ML and 'data science'. it seems like CS has somehow regressed to the behavourism era of psychology or economics before the Lucas critique. The problem with all this data talk isn't just about implementation or bad structure, the limitations of putting all your bets on inductive reasoning are systemic. The insights that economists had in the 70s and 80s was that reasoning from…

> CS has somehow regressed to the behavourism era of psychology or economics before the Lucas critique? Can you please elaborate on this please?

in psychology there was a time period between ca 1900 to the mid century where behaviourism rose in prominence, which was the paradigm, simplified, that internal processes of the mind are not really interesting, and what matters is rather only the relationship between input and output, treating the mind as a black box of sorts (roughly analog to ML models).

This came under heavy attack during what is called the cognitive revolution, which put focus on understanding mental processes at a structural level (for the reasons outlined in the post above).

Economics went through a similar process. Up until the 70s Keynesianism was very dominant, which mostly focusses on using aggregate economic quantified data, i.e output, unemployment, capital and so on to make policy suggestions. This began to be attacked and supplemented with what's called 'micro-foundations', which aimed to not just look at quantified data, but to model, from the individual up, not just top-down, fundamental behaviour and interaction, i.e the actual entities that generate the aggregate data.

There was also a similar movement to this in linguistics starting (mostly) with Chomsky at about the same time applying the same criticism to how we model language.

Re: Data Science: Reality Doesn't Meet Expectations

#162

Post by datenhorst »

danmostudco wrote: post #43

I stood up a data science operation at my company over the last few years, and have noticed a key difference in data-science projects that have been successful and those that have failed. It hits on a number of points brought up in the article, namely where does data science "fit" in an organization delivering software and how is the value realized by the business. The worst cases I have seen is when executives take…

> The worst cases I have seen is when executives take a problem and ask data scientists to "do some of that data science" on the problem, looking for trends, patterns, automating workflows, making recommendations, etc.

While I agree on the point, there's a case that's arguably worse: When those executives hire Data Scientists and then ask them: "So what can we do with Data Science?"

Re: Data Science: Reality Doesn't Meet Expectations

#163

Post by antonvs »

itsmefaz wrote: post #129
Barrin92 wrote: post #110

I'm generally confused by the hype around ML and 'data science'. it seems like CS has somehow regressed to the behavourism era of psychology or economics before the Lucas critique. The problem with all this data talk isn't just about implementation or bad structure, the limitations of putting all your bets on inductive reasoning are systemic. The insights that economists had in the 70s and 80s was that reasoning from…

> CS has somehow regressed to the behavourism era of psychology or economics before the Lucas critique? Can you please elaborate on this please?

See: https://en.m.wikipedia.org/wiki/Lucas_critique

At a high level, it argued that basing predictions on historical data is problematic. The details of the argument are somewhat specific to economics, but the principle is more general. That's also why people recommending stocks say "past performance is no guarantee of future results."

One of the key issues is that circumstances change, and information about such changes will often be external to a data set.

In the Lucas critique, policy changes are an example of this. You can't predict future economic performance based on past economic performance if relevant policies have changed. But any complex situation has such factors that are external to the data that one can easily collect about it.

Re: Data Science: Reality Doesn't Meet Expectations

#164

Post by avs733 »

plants wrote: post #148
avs733 wrote: post #95

Earlier quoted context omitted.

>This. Universities and online challenges provide clean labeled data, and score on model performance. First homework assignment in the stats class I teach is to clean data that the class generated with directions they all perceived as clear. It's near about the most hated assignment I have ever given. Amazing how many ways there are to encode gender of a experimental participant. Male, M, m, male, Man, ...

gender.lower().startswith('m')... done! :)

I would pay student who figured that out $20

Re: Data Science: Reality Doesn't Meet Expectations

#165

Post by MR4D »

starpilot wrote: post #90
kqr wrote: post #33

Earlier quoted context omitted.

I took a data visualisation class in uni that handled this really cleverly. The second assignment sounded very easy. The teacher provided links to the sources where we could find data. Most people figured that with such a simple assignment (not significantly harder than the first one, which was also easy-ish) they could put off doing it until the last moment. Most people failed. This real world data needed hours upon…

This is universal to STEM degrees I think. In mechanical engineering classes you analyze a beam, in real life you analyze an assembly with 50 components that have undergone 100 revisions with 20 different materials and loading from 4 directions that vary with time. Oh, and you have 4 sensors to give you information to analyze critical stresses. But one of them is broken, and Bob who can fix it is on PTO until next Mo…

> get them exposed to real, messy data

This times 1,000.

The biggest surprise to me when I got out of school was how messy things were - data, systems, management, priorities...everything.

When I went back to grad school, we had arguments about the assumptions. It was a total 180 from undergrad, and much more useful. So when I came out of grad school, I was able to deal with the ambiguities - maybe even thrived because I understood them.

Re: Data Science: Reality Doesn't Meet Expectations

#167

Post by Breza »

starpilot wrote: post #90
kqr wrote: post #33

Earlier quoted context omitted.

I took a data visualisation class in uni that handled this really cleverly. The second assignment sounded very easy. The teacher provided links to the sources where we could find data. Most people figured that with such a simple assignment (not significantly harder than the first one, which was also easy-ish) they could put off doing it until the last moment. Most people failed. This real world data needed hours upon…

This is universal to STEM degrees I think. In mechanical engineering classes you analyze a beam, in real life you analyze an assembly with 50 components that have undergone 100 revisions with 20 different materials and loading from 4 directions that vary with time. Oh, and you have 4 sensors to give you information to analyze critical stresses. But one of them is broken, and Bob who can fix it is on PTO until next Mo…

I majored in nonprofit management and every class had a required field work component with an area charity. I learned so much from the combination of intense coursework and real world experience. Now that I'm the head of data science at a corporation, I wish such integration existed in this field.

Re: Data Science: Reality Doesn't Meet Expectations

#168

Post by Breza »

throwaway713 wrote: post #54

As a research-oriented data scientist at one of the larger tech companies, I can confirm that even here, a lot of people are unsure about what exactly data scientists are supposed to do. My most frequent request is "tell us why metric X dropped", to which the answer is often a subtle combination of many different factors (often random fluctuation) that doesn't lead to a pleasing actionable result in the sense of "her…

This is very accurate. I've found that the simplest model with good enough results is often the best in the business world. On the one hand, that means I spend less time pushing the boundaries of what we're capable of doing as an organization. On the other hand, most business questions don't need massively complex answers so a quick regression may suffice.
Post reply on HN