Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Data Science: Reality Doesn't Meet Expectations

dfrieds.com

Post reply on HN
Print view View original

101–110 of 168 posts

Re: Data Science: Reality Doesn't Meet Expectations

#101

Post by tomrod »

natalyarostova wrote: post #93
mattmanser wrote: post #86

Earlier quoted context omitted.

Sounds more like it simply doesn't work very well, rather than any of the reasons you listed. It's often the case, I remember when that stupid Amazon infographic was going around about decreased load times meaning big upswings in conversions. A client paid for a significant project to reduce load times, which we succeeded in to a huge degree with most of the pages going from 1.5-3 seconds secs down to 250-500 ms. Abs…

Load times might not effect conversion linearly. People deal with 3 second loads until one day a competitor does .3 second loads and gives a better experience, then in a matter of months you lose your customer base.

I think this is where causal inference and experimental design are important.

Re: Data Science: Reality Doesn't Meet Expectations

#102

Post by vasili111 »

sgt101 wrote: post #27

Disclaimer: I use the term Data Scientist throughout this post; however, popular titles such as Data Analyst, Data Engineers and BI analyst are randomly applied by people who know nothing, and these people share none of the responsibilities of a Data Scientist. I have never had hopes about the potential impact of being a Data Scientist. I felt every company should be a “data company”, but everything I knew told me th…

How important is Ph.D for data science?

Re: Data Science: Reality Doesn't Meet Expectations

#103

Post by bcrosby95 »

mattmanser wrote: post #86
apohn wrote: post #79

Earlier quoted context omitted.

>But in terms of career progression and job safety, the risk is just way too high, at least for me personally. I save the highly mathematical stuff for a hobby. I think the sad truth is that this is the reality of work no matter if you are a Data Scientist or not. What you thought you would be doing to show your worth and climb the ladder gets blurred in with KPIs you didn't set, politics you didn't create, goals and…

Sounds more like it simply doesn't work very well, rather than any of the reasons you listed. It's often the case, I remember when that stupid Amazon infographic was going around about decreased load times meaning big upswings in conversions. A client paid for a significant project to reduce load times, which we succeeded in to a huge degree with most of the pages going from 1.5-3 seconds secs down to 250-500 ms. Abs…

We had this effect one of our gaming websites, but in reverse: we accidentally added around 900ms to every page load. Gameplays dropped by around 15%. We removed what was causing this and they instantly went back up.

People played it mostly during breaks: lunch breaks (our peak load was during lunch hours in the US), "smoke breaks", etc. So they didn't have a goal, they just had time to spend doing something. Each gameplay took anywhere from 1-5 minutes. Users averaged to 5 plays per day. Our guess was the extra load time caused people to hit some exactly poor threshhold where they were able to play 1 less game during their time allotment.

Edit: we were curious and A/B tested it and saw the effect too. We didn't run it for too long, but a 15% difference is quick to verify when you're measuring something that happens 35 million times per day.

Re: Data Science: Reality Doesn't Meet Expectations

#104

Post by arborism »

wenc wrote: post #51

Data science is correctly valued when you realize how relatively unimportant it is. It is a small cog in a larger machinery (or at least it ought to be). You see, decision-making involves (1) getting data, (2) summarizing and predicting, and (3) taking action. Continuous decision-making -- the kind that leads to impact -- involves doing this repeatedly in a principled fashion, which means creating a system around the…

Had to make an account to upvote this. Absolutely dead-on. I think you can generalize this comment to almost any specialist skill. "No Silver Bullet" should be a business doctrine as well as a technical one. You need to do a lot of things well to succeed in business. Specialists just provide you a capability. You have to implement and use those capabilities as part of a larger system if you want to create a machine that generates profit.

Re: Data Science: Reality Doesn't Meet Expectations

#105

Post by diehunde »

_jal wrote: post #21
worik wrote: post #11

I do not understand. Have never understood. "Data Science" is, surly, newspeak. The appropriate term, surly, is "statistics".

A new title means a new opportunity to ask for more money and influence. See also, "devops".

Or microservices developer/architect

Re: Data Science: Reality Doesn't Meet Expectations

#106

Post by cwyers »

proverbialbunny wrote: post #97
ims wrote: post #73

Earlier quoted context omitted.

The standard whatever.fit(X, y) isn't very appealing but there are much more bespoke models that require creative engagement with stats/CS knowledge, e.g. Bayesian hierarchical models or deep learning models that are more complicated than what can be copy/pasted from Medium.

I've done a lot of ensemble and stacked ensemble learning. I've also used BERT and a couple of other advanced ML, but usually I resort to advanced feature engineering if I can first, so I get what you mean, but it's still not as fun to me as figuring out patterns in data.

It's sort of two-sided, I think. It can be fun to figure out _meaningful_ patterns in data. I don't really find it fun to figure out that "so and so didn't use software that understood NA values back in nineteen tickety two, so some NA values are NA because they're newer, and some NA values are 0 because 0 is just like NULL in somebody's head, and some NA values are -999 because that was a thing they did in the Before Times."

Re: Data Science: Reality Doesn't Meet Expectations

#107

Post by wenc »

Der_Einzige wrote: post #77
Icathian wrote: post #3

This rings very true to me. I'm working on moving over to an SWE role in the next few years for many of these reasons. I'll just add one: the business absolutely doesn't care how you get your answer, only if they're reliable enough (hand grenade close is better than most companies have today). While this seems obvious enough to anyone with a few years under their belt, to the new DS grad who has their time series ana…

Nothing reliably consistently beats ARIMA models in time series forecasting to this day That's pretty sad when you think about it, but it's painfully true.

> Nothing reliably consistently beats ARIMA models in time series forecasting to this day

Not sure this is true in practice. In some situations, Holt-Winters (ie. algorithms in the ETS family) may do better, and it's often a good idea to try both.

There's a claim that Holt-Winters is a special case of ARIMA (the claim is ARIMA is more general), but this is actually not the case. There is equivalence in only a subset of cases. [1]

I've fitted Holt-Winters models that beat ARIMA models. ARIMA models can have trouble generalizing from training data with long horizons because they tend to overfit to the distant past. Holt-Winters on the other hand has a natural "forgetting factor" built-in which moderates this.

As well, my experience is that stacked models with well-chosen exogeneous variables (if you have causal variables) tend to outperform pure time-series methods because they are anchored on more independent variables than just t. Pure time-series models bank on the assumption that patterns have a repeatable time-dependence, and most of the time this is just not true, so have to be augmented with other variables.

[1] https://otexts.com/fpp2/arima-ets.html

Re: Data Science: Reality Doesn't Meet Expectations

#108

Post by wenc »

worik wrote: post #11

I do not understand. Have never understood. "Data Science" is, surly, newspeak. The appropriate term, surly, is "statistics".

Data science is an overloaded term, but even so there are some salient differences between it and statistics.

Data science more closely related to "statistical learning" and the knowledge required overlaps with but looks quite different with that of conventional statistics.

An easy way to get a sense of the difference is to compare the table of contents of a book like ISL (PDF free) [1] to the undergraduate curriculum of a statistics program. You'll find that that the focus and indeed culture of data science is really quite different from that of statistics.

Leo Breiman wrote about this in his paper "Statistical Modeling: the Two Cultures" [2]. Conventional statistics belongs to one culture, and statistical learning/data science sort of veers toward to the other (though not completely).

Much has been made about how "data science" is just statistics dressed up to look new, but I'm not convinced this is true. I'm also not convinced that pure statisticians have the right training to be data scientists -- additional training and mindset changes are needed. The reverse is also true: most data scientists lack the rigor and epistemological training to be statisticians.

[1] http://faculty.marshall.usc.edu/gareth-james/ISL/

[2] https://projecteuclid.org/euclid.ss/1009213726

Re: Data Science: Reality Doesn't Meet Expectations

#109

Post by thaumasiotes »

natalyarostova wrote: post #93
mattmanser wrote: post #86

Earlier quoted context omitted.

Sounds more like it simply doesn't work very well, rather than any of the reasons you listed. It's often the case, I remember when that stupid Amazon infographic was going around about decreased load times meaning big upswings in conversions. A client paid for a significant project to reduce load times, which we succeeded in to a huge degree with most of the pages going from 1.5-3 seconds secs down to 250-500 ms. Abs…

Load times might not effect conversion linearly. People deal with 3 second loads until one day a competitor does .3 second loads and gives a better experience, then in a matter of months you lose your customer base.

> People deal with 3 second loads until one day a competitor does .3 second loads and gives a better experience, then in a matter of months you lose your customer base.

I tend to suspect that the effect of pricing will make a difference of 2.7 seconds in load time negligible. A 3 second load just isn't a large cost, even if you run into it repeatedly.

Re: Data Science: Reality Doesn't Meet Expectations

#110

Post by Barrin92 »

I'm generally confused by the hype around ML and 'data science'. it seems like CS has somehow regressed to the behavourism era of psychology or economics before the Lucas critique.

The problem with all this data talk isn't just about implementation or bad structure, the limitations of putting all your bets on inductive reasoning are systemic.

The insights that economists had in the 70s and 80s was that reasoning from aggregated quantities is extremely limited. Without understanding at a structural level the generators of your data, trying to create policy based on outputs is like trying to reason about inhabitants of a city by looking at light pollution from the sky.

My guess why data science so rarely delivers what it promises is because you can't get any value from historical data if your circumstances change to the point where past data is irrelevant. Which in the world of business happens pretty quickly. To have a competitive advantage, one needs to figure out what has not been seen yet.

And trying to exploit signals suffers from the issue laid out above. There was a funny case of an AI hiring startup trying to predict good applicants, and the result was people putting "Oxford" in their application in a font matching the background color

Post reply on HN