Data Science: Reality Doesn't Meet Expectations

Post by ehnto » Wed, Apr 08, 2020, 1:04 PM UTC

thablackbull wrote: post #134

gcatalfamo wrote: post #132

Earlier quoted context omitted.

You are being downvoted but you are exactly on point. If some fail they may be bad students, but if the majority of my students fail they're not bad students, it is me who is a bad teacher.

> Of course, the teacher knew this, gave bonus points to the ones who did start in time, and then extended the deadline as he had expected to from the start. I think what he meant is they 'failed' to get it completed on time and it was meant as a teaching lesson.

Post by beckingz » Wed, Apr 08, 2020, 1:41 PM UTC

FridgeSeal wrote: post #83

mr_toad wrote: post #50

Earlier quoted context omitted.

> the scientific method to do research on data Exploratory data analysis is often overlooked and underrated.

Ppphhh we don’t need to do exploratory data analysis or prepare the days, don’t you know that neural networks will do all that themselves! Doesn’t yield the right results? Clearly not enough data. Still doesn’t work? Change to whatever the latest model google or fb is using and try again. /s

Post by beckingz » Wed, Apr 08, 2020, 1:48 PM UTC

kevin_thibedeau wrote: post #118

mirimir wrote: post #113

I did "data science" for about a decade, consulting with plaintiffs firms and state AGs on antitrust and fraud cases. For each case, the work flow was roughly this: -- write discovery requests -- review production, and check out data and documentation -- write supplementary discovery requests -- review production, and check out data and documentation [repeat as needed] -- analyze data, and write deposition questions…

I would expect "data science" is doing some form of numerical analysis. Otherwise it's just record keeping... with computers.

Post by rafiki6 » Wed, Apr 08, 2020, 2:25 PM UTC

bcrosby95 wrote: post #103

mattmanser wrote: post #86

Earlier quoted context omitted.

Sounds more like it simply doesn't work very well, rather than any of the reasons you listed. It's often the case, I remember when that stupid Amazon infographic was going around about decreased load times meaning big upswings in conversions. A client paid for a significant project to reduce load times, which we succeeded in to a huge degree with most of the pages going from 1.5-3 seconds secs down to 250-500 ms. Abs…

We had this effect one of our gaming websites, but in reverse: we accidentally added around 900ms to every page load. Gameplays dropped by around 15%. We removed what was causing this and they instantly went back up. People played it mostly during breaks: lunch breaks (our peak load was during lunch hours in the US), "smoke breaks", etc. So they didn't have a goal, they just had time to spend doing something. Each ga…

Post by Eridrus » Wed, Apr 08, 2020, 2:58 PM UTC

shoguning wrote: post #26

Eridrus wrote: post #14

Earlier quoted context omitted.

As somebody in an ML Engineering role, i.e. somebody who could be asked to either fix the logging infrastructure or build some models, I would have agreed with this. But even in this day and age with ML being the new hotness, you will find people who are quite happy to work on infrastructure and don't have a huge amount of interest in training models themselves, and it is probably a lot easier to hire them than peopl…

I wrestle with this too, there's a lot of context to determine what skillset is better. I suspect, if there are lots of relatively simple ML problems, then a generalist with integration chops will be more effective in getting them out quickly and "good enough". The specialist may take too long on models that are too heavy and impractical. If there's one big ML problem (Google search, Netflix recommender, Amazon searc…