Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Goodbye, data science

ryxcommar.com

Post reply on HN
Print view View original

61–70 of 415 posts

Re: Goodbye, data science

#61

Post by strikelaserclaw »

jmhammond wrote: post #13

> But there’s also a part of me that’s just like, how can you not be curious? How can you write Python for 5 years of your life and never look at a bit of source code and try to understand how it works, why it was designed a certain way, and why a particular file in the repo is there? How can you fit a dozen regressions and not try to understand where those coefficients come from and the linear algebra behind it? I d…

curiosity is good but there is so much stuff to learn out there that for many fields learning things deeply is much less important than learning a lot at 25-35% depth.

Re: Goodbye, data science

#62

Post by lajosbacs »

I've been a 'data scientist' for years, and I probably will be again at some point as it is the biggest item in my CV. It was in a company, where data science was not the bread and butter, but just something extra to show to the clients.

For me therefore, data science is the epitome of Graber's 'bullshit job' -- if the position didn't exist, the company would go on just the same.

Re: Goodbye, data science

#64

Post by gillesjacobs »

> Managers will say they want to make data-driven decisions, but they really want decision-driven data.

Has been my experience as ML engineer too. Decision making being intuition- and not data-driven was one of the largest shocks to me when I went from academia into industry.

How upper management and the board determine the course of the company was based more on emotion than anything else.

Re: Goodbye, data science

#65

Post by adamsmith143 »

CapmCrackaWaka wrote: post #30
samvher wrote: post #6

> Nobody knew or even cared what the difference was between good and bad data science work. Meaning you could absolutely suck at your job or be incredible at it and you’d get nearly the same regards in either case. In my experience it's even a little bit worse than that. Approaches that are wrong from a statistics point of view are more likely to generate impressive seeming results. But the flaws are often subtle. A…

I've seen this a LOT in my professional group. Many people (who often have PhDs!!) I interview for data science positions seem to know absolutely nothing about the algorithms they use professionally, or how to optimize them, or why they are a good fit for their use case, etc etc etc. I usually see through LinkedIn that these same people are now in impressive-sounding positions at other companies. I had one candidate…

Where are these jobs where you can interview this badly and still get hired because in my experience DS interviews are extremely hard and often expect people to have very high Stats skills as well as Data Structures/Algo skills at FAANG level.

Re: Goodbye, data science

#66

Post by sebastiansm »

Icathian wrote: post #24

This hit all the same high notes I was feeling when I quit Data Science to become a software engineer. It's an infinitely better gig and I encourage all my colleagues with enough chops to make the same switch.

How did you do the transition from DS to SWE?

Re: Goodbye, data science

#67

Post by adamsmith143 »

drgiggles wrote: post #58

Unfortunately it seemed pretty clear from the start that this is what data science would turn into. Data science effectively rebranded statistics but removed the requirement of deep statistical knowledge to allow people to get by with a cursory understanding of how to get some python library to spit out a result. For research and analysis data scientists must have a strong understanding of underlying statistical theo…

On the flip side you used to have statisticians writing code that is frankly unusable in a Production environment. You would weep at the R code I've seen and had to turn into something to actually produce business value.

Re: Goodbye, data science

#68

Post by TrackerFF »

itissid wrote: post #59

I have never understood the what a good ML engineer couldn't do and a Data scientist could in _majority_ situations. When you need a decision to be made based on data its just common sense risk analysis added together with basic statistics. I feel some good field training in statistics(Look up Andrew Gelman) a couple of good courses on Linear, Bayesian Regression is all you need, rest is just engineering skill. The d…

IMO a data scientist should also be a domain expert, in the same way analysts are.

But of course, too many view DS as some abstract skill where domain knowledge is not needed, and where the methodology will solve all problems / provide insight.

Re: Goodbye, data science

#69

Post by wittycardio »

oneoff786 wrote: post #19
jmhammond wrote: post #13

> But there’s also a part of me that’s just like, how can you not be curious? How can you write Python for 5 years of your life and never look at a bit of source code and try to understand how it works, why it was designed a certain way, and why a particular file in the repo is there? How can you fit a dozen regressions and not try to understand where those coefficients come from and the linear algebra behind it? I d…

Seems pretentious to me. I’ve never bothered to look through many things I use. I look extensively at how to use them and what the API offers. I have a good intuition for how most models work. I don’t really care about the specifics of the implementations. I have more important things to do. The hacker mentality, imo, is about identifying what’s useful for you to explore to accomplish whatever you need. Often that’s…

"it's pretentious to know what you're doing" just be a haxxor xD

Re: Goodbye, data science

#70

Post by wittycardio »

cmmeur01 wrote: post #35
xyzelement wrote: post #11

// it was often personally unfulfilling (e.g. tuning a parameter to make the business extra money). He lost me here. Something I've always loved about being an engineer (and now in product) is that something small we do/tweak can have big impact. If you tuned a parameter and that actually had tangible impact on the business, that's like the best case scenario and should be celebrated (vs doing some cool rocket scienc…

And all that extra profit is hovered up by the people above you that had nothing to do with it. Validated engineering cost savings should be treated like sales, the engineer gets a percentage.

Engineers get paid much better than sales on average. You probably don't want such a variable compensation structure
Post reply on HN