Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Data Science: Reality Doesn't Meet Expectations

dfrieds.com

Post reply on HN
Print view View original

151–160 of 168 posts

Re: Data Science: Reality Doesn't Meet Expectations

#151

Post by bart_spoon »

mattmanser wrote: post #86
apohn wrote: post #79

Earlier quoted context omitted.

>But in terms of career progression and job safety, the risk is just way too high, at least for me personally. I save the highly mathematical stuff for a hobby. I think the sad truth is that this is the reality of work no matter if you are a Data Scientist or not. What you thought you would be doing to show your worth and climb the ladder gets blurred in with KPIs you didn't set, politics you didn't create, goals and…

Sounds more like it simply doesn't work very well, rather than any of the reasons you listed. It's often the case, I remember when that stupid Amazon infographic was going around about decreased load times meaning big upswings in conversions. A client paid for a significant project to reduce load times, which we succeeded in to a huge degree with most of the pages going from 1.5-3 seconds secs down to 250-500 ms. Abs…

> I've always suspected since that it was someone massaging figures in Amazon to justify their job.

Well the first rule should be looking skeptically at someone whose "analysis" involves something their core business provides/sells. Facebook and Google have been pushing data driven narratives about how effective their advertising is, and yet as a data scientist working at a large Fortune 500 company, we never were able to show meaningful impact anywhere close to what was claimed. This was met with pushback, as before my team was created the company relied on external analytics vendors who always came back with results that were magically what everyone was expecting/hoping for. But when my team tried to recreate what they had done, they would withhold information claiming it they were "trade secrets", or what they did provide was riddled with egregious errors.

I actually think that is the biggest argument as to why every company should have some kind of data science team. There is certainly important predictive models and analytics to be done, but the most consistent ROI would be to keep the company grounded and not dropping huge sums of money on the trendiest snake-oil analytics/AI solutions being hawked by vendors.

Re: Data Science: Reality Doesn't Meet Expectations

#152

Post by apohn »

mattmanser wrote: post #86
apohn wrote: post #79

Earlier quoted context omitted.

>But in terms of career progression and job safety, the risk is just way too high, at least for me personally. I save the highly mathematical stuff for a hobby. I think the sad truth is that this is the reality of work no matter if you are a Data Scientist or not. What you thought you would be doing to show your worth and climb the ladder gets blurred in with KPIs you didn't set, politics you didn't create, goals and…

Sounds more like it simply doesn't work very well, rather than any of the reasons you listed. It's often the case, I remember when that stupid Amazon infographic was going around about decreased load times meaning big upswings in conversions. A client paid for a significant project to reduce load times, which we succeeded in to a huge degree with most of the pages going from 1.5-3 seconds secs down to 250-500 ms. Abs…

>Sounds more like it simply doesn't work very well, rather than any of the reasons you listed.

The use case you've described has a defined problem and a measurable metric. Problem: We think load times influence conversions: Metrics: Measure load times and see if they are correlated with conversions. Maybe in your case somebody decided to skip the research part and just pay to reduce load times.

Imagine a totally difference scenario. You work for a established (30+ years old) company that sells consumer goods. Executives approve a $25 Million budget to "improve the customer experience" over the next 3 years.

This directive goes to all the various organizations: Sales and Marketing, Product Development, Technology, Customer and Market Research, Customer Support, etc. The various orgs have 3 months to come back to executive management to justify how much budget they need and their execution strategy. Each org thinks thinks they are the key mover in improving customer experiences and wants as much of that budget as possible. Every org works at a difference speed and with different philosophies (e.g. all work is done in-house versus some or a lot of work done by external agencies).

Let's add some more reality into this. Even if the CXO of an org thinks they don't need to be in this process, it looks bad if they don't say they have a strategy and need budget. There's also a significant chance that 9 months into this project somebody will get restless and the whole initiative will get restructured with different timelines and goals.

I could do on, but armies of Analysts and Data Scientists will get pulled into this to drive "data driven decision-making." A lot of the expectations will that the "smart people" will show that everyone's particular biases will be the most important one and needs.

It's hardly an environment for anybody to do rigorous analysis or for anybody in an Analytical role to shine. Think the scenario sounds insane and made up? It's not. Welcome to big-co.

Re: Data Science: Reality Doesn't Meet Expectations

#153

Post by alixedi »

To point #5 in the article, in my experience, ascending order of potential to generate value for business:

An astonishingly large fraction of Data Science output goes to die in pretty presentations.

From what's left, a large fraction ends up in Spreadsheets.

A disappointingly small fraction ends up in live services.

Re: Data Science: Reality Doesn't Meet Expectations

#154

Post by apohn »

bart_spoon wrote: post #151
mattmanser wrote: post #86

Earlier quoted context omitted.

Sounds more like it simply doesn't work very well, rather than any of the reasons you listed. It's often the case, I remember when that stupid Amazon infographic was going around about decreased load times meaning big upswings in conversions. A client paid for a significant project to reduce load times, which we succeeded in to a huge degree with most of the pages going from 1.5-3 seconds secs down to 250-500 ms. Abs…

> I've always suspected since that it was someone massaging figures in Amazon to justify their job. Well the first rule should be looking skeptically at someone whose "analysis" involves something their core business provides/sells. Facebook and Google have been pushing data driven narratives about how effective their advertising is, and yet as a data scientist working at a large Fortune 500 company, we never were ab…

>...we never were able to show meaningful impact anywhere close to what was claimed. This was met with pushback, as before my team was created the company relied on external analytics vendors who always came back with results that were magically what everyone was expecting/hoping for...

This was why I left my last job managing a Data Science team at a large company. It's nearly impossible to complete with a slidedeck from an external vendor that shows exactly what people want to see. Especially when decision-makers and check-signers move on to different jobs in 2 years, so there is nobody to answer why that was done in the first place. Arguing against those vendors brings out the worst in the interested parties and you become the bad guy.

Re: Data Science: Reality Doesn't Meet Expectations

#155

Post by sgt101 »

vasili111 wrote: post #144
sgt101 wrote: post #127

Earlier quoted context omitted.

I think it's quite important - or an equivalent. From about 2012 to 2018 I went round a lot of universities, conferences and companies doing presentations and I used to often ask the audience for a definition of data science (in the hope of getting a good one). The best one I heard came at the University of Bath where someone (I know who, but he didn't say it to back it with his reputation so it's not fair to name hi…

Will you employ data scientist that have articles and years of expirience as data analyst in University but not Ph.D? How much role will be lack of Ph.D in this case?

As I said - if the person is capable of independent scientific investigation then I think they'd be good. I think that a Ph.D is formal training for that - but not the only way to learn.

Re: Data Science: Reality Doesn't Meet Expectations

#156

Post by starpilot »

mcrad wrote: post #123
starpilot wrote: post #90

Earlier quoted context omitted.

This is universal to STEM degrees I think. In mechanical engineering classes you analyze a beam, in real life you analyze an assembly with 50 components that have undergone 100 revisions with 20 different materials and loading from 4 directions that vary with time. Oh, and you have 4 sensors to give you information to analyze critical stresses. But one of them is broken, and Bob who can fix it is on PTO until next Mo…

> This is universal to STEM degrees I think. In mechanical engineering classes you analyze a beam, in real life you ... Hard to believe this. Don't these degrees require rigorous laboratory assignments where the student learns to differentiate best case scenario with real world uncertainties? STEM is not just some IT certification

Not really. MechE courses are really theoretical, and the labs are focused on just being enough to demo the theories. Most of my professors had never worked in industry, they had been in academia their entire lives. Even they wouldn't know how to bridge the gap.

In an ideal world, we'd have separate tracks for people entering industry versus academia/research, but that's a long way off.

Re: Data Science: Reality Doesn't Meet Expectations

#157

Post by mirimir »

kevin_thibedeau wrote: post #118
mirimir wrote: post #113

I did "data science" for about a decade, consulting with plaintiffs firms and state AGs on antitrust and fraud cases. For each case, the work flow was roughly this: -- write discovery requests -- review production, and check out data and documentation -- write supplementary discovery requests -- review production, and check out data and documentation [repeat as needed] -- analyze data, and write deposition questions…

I would expect "data science" is doing some form of numerical analysis. Otherwise it's just record keeping... with computers.

The hardest part of what I did was getting enough documentation to understand the data. Sometimes we got fixed width text files, with no in formation about column definitions. Or column names. Or what values in descriptive columns meant. Stuff like "class of trade".

But generally you're right. It was just simple calculations using sales records. But lots of records, at least several gigabytes, and sometimes several hundred gigabytes.

Re: Data Science: Reality Doesn't Meet Expectations

#158

Post by mirimir »

beckingz wrote: post #145
kevin_thibedeau wrote: post #118

Earlier quoted context omitted.

I would expect "data science" is doing some form of numerical analysis. Otherwise it's just record keeping... with computers.

Record keeping is 90% of data projects. The second 90% is basic math at high speeds.

Right, record keeping. But when it's not your data, things get complicated. Imagine trying to understand how another firm's data systems work. You can talk with managers, who know how the business uses data. But they have no clue how the data are stored or managed. And you can talk with IT people, who know how data are stored or managed. But they have no clue how the data are used.

And yes, speed. Aggregating hundreds of gigabytes was nontrivial to do quickly. I started with Access, and then learned to manage and use SQL Server. And eventually a multi-Xeon server with lots of RAM and SAS-attached storage.

Re: Data Science: Reality Doesn't Meet Expectations

#159

Post by mcrad »

starpilot wrote: post #156
mcrad wrote: post #123

Earlier quoted context omitted.

> This is universal to STEM degrees I think. In mechanical engineering classes you analyze a beam, in real life you ... Hard to believe this. Don't these degrees require rigorous laboratory assignments where the student learns to differentiate best case scenario with real world uncertainties? STEM is not just some IT certification

Not really. MechE courses are really theoretical, and the labs are focused on just being enough to demo the theories. Most of my professors had never worked in industry, they had been in academia their entire lives. Even they wouldn't know how to bridge the gap. In an ideal world, we'd have separate tracks for people entering industry versus academia/research, but that's a long way off.

That's insane. ME degrees that I know seem to be defined by industry (ie. application of theory). Nobody pursues that degree to stay in academia/research. Anyway you can always pursue an advanced degree if you want to stay in academia. Don't get it twisted though - STEM is not a vocation as per your suggestion that "people entering industry" deserve a special path.

Re: Data Science: Reality Doesn't Meet Expectations

#160

Post by mcrad »

Underqualified wrote: post #128
mcrad wrote: post #123

Earlier quoted context omitted.

> This is universal to STEM degrees I think. In mechanical engineering classes you analyze a beam, in real life you ... Hard to believe this. Don't these degrees require rigorous laboratory assignments where the student learns to differentiate best case scenario with real world uncertainties? STEM is not just some IT certification

As a mechanical engineer : No, my education didn't. The problem is that most real world problems take too much time to really solve to fit in any modern ciriculum.

Hmmm. We had a whole course on measurement systems that get to the heart of understanding that source of your data and inevitable bias/error is more important than just crunching the data as given. For example, from a typical four year degree.
Post reply on HN