Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Goodbye, data science

ryxcommar.com

Post reply on HN
Print view View original

131–140 of 415 posts

Re: Goodbye, data science

#131

Post by fnands »

adamsmith143 wrote: post #67
drgiggles wrote: post #58

Unfortunately it seemed pretty clear from the start that this is what data science would turn into. Data science effectively rebranded statistics but removed the requirement of deep statistical knowledge to allow people to get by with a cursory understanding of how to get some python library to spit out a result. For research and analysis data scientists must have a strong understanding of underlying statistical theo…

On the flip side you used to have statisticians writing code that is frankly unusable in a Production environment. You would weep at the R code I've seen and had to turn into something to actually produce business value.

There is a bit of a joke that a data scientist is someone who can do better stats then the average SWE and can write better code than the average statistician. Both of those are relatively low bars to clear though

Re: Goodbye, data science

#132

Post by d23 »

> Nobody knew or even cared what the difference was between good and bad data science work. Meaning you could absolutely suck at your job or be incredible at it and you’d get nearly the same regards in either case.

This seems to be a problem with the industry as a whole. I'm speaking as a SWE, but I've observed similar things with PMs. I don't think it's impossible or even very hard to appreciate the right things, it just requires a bit of thought and the correct value system. Both of those seem to be a bit too far of a reach though.

Re: Goodbye, data science

#133

Post by oneoff786 »

6gvONxR4sf7o wrote: post #33
oneoff786 wrote: post #19

Earlier quoted context omitted.

Seems pretentious to me. I’ve never bothered to look through many things I use. I look extensively at how to use them and what the API offers. I have a good intuition for how most models work. I don’t really care about the specifics of the implementations. I have more important things to do. The hacker mentality, imo, is about identifying what’s useful for you to explore to accomplish whatever you need. Often that’s…

If you think it’s about implementation details, you’re misunderstanding. It’s about understanding the principles behind it. As an example, it’s more about understanding the statistics and linear algebra around estimating uncertainty in GLM regression estimates, than about reading the code for how the statsmodels library implements that.

If you want to understand the intuition behind a model you should read a paper, book, or even a blog before the repo imo.

Re: Goodbye, data science

#134

Post by throwaway2037 »

mynameisash wrote: post #116
jstx1 wrote: post #3

As someone who also wants to move away from data science, data engineering is the last thing I would want to do. I think DE comes with many of the same problems and it's also a very ill-defined career track; I wouldn't recommend it to anyone. ML engineer or backend developer seem like much more appealing job profiles.

My title is still software engineer, but I effectively do data engineering, and I work closely with data scientists. I love a lot of it, but there's still plenty of bullshit to deal with. Just in the technical side, dealing with Python is a perpetual gong show, and most of my team's work seems to revolve around configuration of secrets and K8s. I'm fortunate to be the guy that nerds out about performant code, so when…

You wrote: > What does this mean?

Re: Goodbye, data science

#135

Post by fnands »

angry_moose wrote: post #72

In a recent past life, I was a HPC (high performance computing) administrator for a mid size company (just barely S&P400) who was in the transportation industry, so I had a lot of interactions with the "data science" team and it was just a fascinating delusion to watch. Our CTO did the "Quick, this is the future! I'll be fired if I don't hop on this trend" panic thing and picked up a handful of recent grads and gave…

Honestly, I can't tell you how many jobs ads I saw where I was wondering: "What would they expect me to bring to the table here?"

Some companies just don't have the data, or heck even the need, for data scientist yet try and hire them anyway.

Give smart people a fundamentally ill-posed problem and they won't get anywhere anyway.

Re: Goodbye, data science

#136

Post by nindalf »

lotsofpulp wrote: post #117
etempleton wrote: post #84

Earlier quoted context omitted.

I become wary any time someone utters the phrase, "show me the data" or any variation there of. There is a specific type of leader who thinks that within the data lurks a magical solution just waiting to be discovered. There is also the leader who uses data as a trump card to win arguments and these folks are perhaps even worse. This is not new. The origination of the phrase, "lies, damned lies, and statistics," can…

I have only heard “show me the data” when someone wants someone else to support a claim. I do not see why this would necessarily be a bad thing.

GP was asked to support their claims with data and didn’t like it.

Re: Goodbye, data science

#137

Post by scottLobster »

etempleton wrote: post #84
samvher wrote: post #6

> Nobody knew or even cared what the difference was between good and bad data science work. Meaning you could absolutely suck at your job or be incredible at it and you’d get nearly the same regards in either case. In my experience it's even a little bit worse than that. Approaches that are wrong from a statistics point of view are more likely to generate impressive seeming results. But the flaws are often subtle. A…

I become wary any time someone utters the phrase, "show me the data" or any variation there of. There is a specific type of leader who thinks that within the data lurks a magical solution just waiting to be discovered. There is also the leader who uses data as a trump card to win arguments and these folks are perhaps even worse. This is not new. The origination of the phrase, "lies, damned lies, and statistics," can…

Yep, these are the same people who backtest their portfolio and go "see, if you'd held this exact portfolio I put together through trial and error, you'd have turned one dollar into a million without any additional contributions!"

Not a data scientist, but it seems like a lot of people in business refuse to accept the fact that reality is generally boring, best practices are often "best" for a reason, and meaningful progress is hard. Of course it is possible to be too conservative, but 95% of ideas to improve a business or product are ego-stroking bullshit. Everyone wants the V10 engine to go down the highway at 65 mph, while towing a trailer, and there's only budget for an oil change every 15000 miles; don't look at the transmission fluid, just don't look.

Re: Goodbye, data science

#138

Post by oneoff786 »

r-zip wrote: post #27
oneoff786 wrote: post #19

Earlier quoted context omitted.

Seems pretentious to me. I’ve never bothered to look through many things I use. I look extensively at how to use them and what the API offers. I have a good intuition for how most models work. I don’t really care about the specifics of the implementations. I have more important things to do. The hacker mentality, imo, is about identifying what’s useful for you to explore to accomplish whatever you need. Often that’s…

That's all fine, as long as you still understand the underlying assumptions and pitfalls. Many people who skim documentation and throw things together haphazardly do not.

Skim documentation != going through repos

Re: Goodbye, data science

#139

Post by chaosbutters314 »

physics-based modeling with some supplemental data science/ML will always be superior. Every data scientist I've worked with has so little domain knowledge anything they do is already useless. They also have little to no presentation skills especially in a business setting.

obviously there are some examples like computer vision that require ML.

Re: Goodbye, data science

#140

Post by yamtaddle »

chris_wot wrote: post #43
Davertron wrote: post #36

Earlier quoted context omitted.

Well good luck then, in my experience the most free time I've ever had in my life was during college. I squandered massive amounts of that time doing things completely unrelated to education, and I definitely don't regret doing that. College isn't just about book learning after all. But still, BY FAR, college is the time of my life when I had the most free time to do whatever I wanted.

We’re you working whilst you were in college?

I worked 10-15 hours a week (and somewhat more in the Summer) for about three years of college and can confirm, still the most free time and lowest stress ever. Worst, by a mile, was high school, and I even had a pretty good experience there. Far worse than working a full time job while having multiple young kids, even. Worse than before we had kids but when we made very little money and struggled to pay the bills every month. High school is terrible.
Post reply on HN