Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Goodbye, data science

ryxcommar.com

Post reply on HN
Print view View original

261–270 of 415 posts

Re: Goodbye, data science

#261

Post by srajabi »

starwind wrote: post #71

I made this same transition from data science to data engineering about 18 months ago and I've never looked back. I hated working with bad code and dealing with arrogant phds who don't value good code. I've seen so many terrible Jupyter Notebooks just copied and pasted into VS Code and the data scientist just washed their hands of it calling it "production ready." Here's a conversation I've had multiple times: Me: ha…

This resonates with me so much, I stumbled into data science out of University a decade ago. Left it to do SWE and came back to it in the last 3 years.

So many data scientists are full of themselves thinking they are magicians and software developers are blacksmiths who are beneath them.

Incrementally at my company the SWE's have automated so much of the data scientists workflow that they end up just as you describe, using the tooling and being relegated to becoming analysts.

After 3 years coming back to this field, I see the writing on the wall: In the 90's most models were created by software developers, in the 2030's most models will be created by software developers.

Re: Goodbye, data science

#262

Post by remram »

_jx7j wrote: post #95
samvher wrote: post #6

> Nobody knew or even cared what the difference was between good and bad data science work. Meaning you could absolutely suck at your job or be incredible at it and you’d get nearly the same regards in either case. In my experience it's even a little bit worse than that. Approaches that are wrong from a statistics point of view are more likely to generate impressive seeming results. But the flaws are often subtle. A…

>Meaning you could absolutely suck at your job or be incredible at it and you’d get nearly the same regards in either case. One of the things I don't like about statements like this said in a Data Science context, is that they are true outside of Data Science as well. Executives make big decisions, managers make smaller decisions, nobody can evaluate how good/bad they really were for months or years. Engineers build…

This reminds me of the seminal article by The Correspondent about online advertising: https://thecorrespondent.com/100/the-new-dot-com-bubble-is-h...

Relying on your data science or marketing department to tell you how good your data science or marketing department is doing, with their own metrics and their own evaluation methods that you don't understand, can only really lead to one outcome.

Re: Goodbye, data science

#263

Post by aschearer »

scottLobster wrote: post #156
_jx7j wrote: post #95

Earlier quoted context omitted.

>Meaning you could absolutely suck at your job or be incredible at it and you’d get nearly the same regards in either case. One of the things I don't like about statements like this said in a Data Science context, is that they are true outside of Data Science as well. Executives make big decisions, managers make smaller decisions, nobody can evaluate how good/bad they really were for months or years. Engineers build…

Not to get too off topic, but as a 35 year old engineer it seems the world in general has far fewer consequences than I was raised to expect. Everything from businesses with bullshit ideas flourishing at a loss, to January 6 even being possible (politics aside I expected the Capitol Police to crack a lot more skulls than they did once people started smashing windows), to the whole FTX situation and the tepid response…

While what you're saying appeals to my biases I think it's a somewhat ahistorical. Not long ago we had Nixon. We had JFK's, MLK's, and RFK's assinations. Plus Reagan's attempted assination. We had the Vietnam war. And so forth. If I were an adult during that era, I imagine it would have felt like consequences were slow to come.

Re: Goodbye, data science

#264

Post by f1shy »

justsocrateasin wrote: post #102
Davertron wrote: post #36

Earlier quoted context omitted.

Well good luck then, in my experience the most free time I've ever had in my life was during college. I squandered massive amounts of that time doing things completely unrelated to education, and I definitely don't regret doing that. College isn't just about book learning after all. But still, BY FAR, college is the time of my life when I had the most free time to do whatever I wanted.

Yeah I hardcore disagree with this. Partly my fault for saying yes too much, partly my work schedule, partly being in a weed out program that really worked you to the bone. Some semesters I was doing like 70-80 hours a week on average, split between managing clubs, homework, attending class, working part time jobs, studying. One week I remember being busy from 7am to 2am for 6 days straight. a few semesters I had a l…

This was also my case... but tbh. I was the only one of the many thousands I met in the university...

Re: Goodbye, data science

#265

Post by f1shy »

boringg wrote: post #115
Davertron wrote: post #36

Earlier quoted context omitted.

Well good luck then, in my experience the most free time I've ever had in my life was during college. I squandered massive amounts of that time doing things completely unrelated to education, and I definitely don't regret doing that. College isn't just about book learning after all. But still, BY FAR, college is the time of my life when I had the most free time to do whatever I wanted.

My sense is that your program at school had a light work load - so a difference in experience. My peak workload so far in my life was at college - I had over 40 hours of class time a week which you then have to add on homework, projects and exams. It was a grind. Since then workload has been intense of course but never comparable. I've had much more time to be able to explore personal interests since college.

Where did you study? I was working full time while doing the university... It was HARD... but I was nowhere nearly 40Hs of class a week... unless you do the whole university in 2 years?!

Re: Goodbye, data science

#266

Post by DenisM »

CapmCrackaWaka wrote: post #30
samvher wrote: post #6

> Nobody knew or even cared what the difference was between good and bad data science work. Meaning you could absolutely suck at your job or be incredible at it and you’d get nearly the same regards in either case. In my experience it's even a little bit worse than that. Approaches that are wrong from a statistics point of view are more likely to generate impressive seeming results. But the flaws are often subtle. A…

I've seen this a LOT in my professional group. Many people (who often have PhDs!!) I interview for data science positions seem to know absolutely nothing about the algorithms they use professionally, or how to optimize them, or why they are a good fit for their use case, etc etc etc. I usually see through LinkedIn that these same people are now in impressive-sounding positions at other companies. I had one candidate…

> XYZ says they can get better performance

Can you do your analysis both ways? Give your customers both, then tell them you method is more modern, but if they want outdated methods you have those too.

Re: Goodbye, data science

#268

Post by deckard1 »

onlyrealcuzzo wrote: post #200
mumblemumble wrote: post #177

Earlier quoted context omitted.

The problem is that nobody actually wants data science. They want data pseudoscience. And for the same reason that people tend to want pseudoscience instead of science in any other domain, too. Science is slow, tentative, and messy, and usually responds to questions with even more questions rather than with answers. Pseudoscience tends to be much more concerned with exuding confidence and providing clean-cut answers.…

> The problem is that nobody actually wants data science. They want data pseudoscience. Technically, I think investors & owners would want the company to use real data science to improve products & maximize profits. Everybody in the middle just wants to use data to lie to get promoted faster - because you don't get promoted for actually doing a good job - you get promoted for convincing people you did a good job, and…

Welcome to the world of OKRs/KPIs/Scrum, where everything's made up and the points don't matter.

Re: Goodbye, data science

#269

Post by asdff »

jrochkind1 wrote: post #253
asdff wrote: post #245

Earlier quoted context omitted.

Former body worker. 100% of us worked, because if you had nothing to do the boss will invent something for you to do or send you home.

Are you saying that meant you did a lot of useless work?

It was more like backlogged maintenance. Like "Ok its too wet to mow today, lets get the backhoe and finally redo the storm drain so it doesn't clog"

Re: Goodbye, data science

#270

Post by f1shy »

throwaway2037 wrote: post #122
mschuster91 wrote: post #47

Earlier quoted context omitted.

Not everyone is wired that way. Personally, I have taken apart and reassembled most of the tech stuff I have at home simply because it interests me how things work (and broke and repaired a non-negligible amount of them in the process, to add), I've dabbled in repairing cars, gas boilers, do my own electricity work... but in my social circle, I'm pretty much the only one. And as I grew older, managed to land myself a…

You wrote: > Is this true in Germany?

Having relatives and friends across Europe (UK, DE, IT, ES, FR, PT) I cannot understand the parent comment (not having enough time for curiosity). I see only "PARTYYY!" in the universities... plenty of free time -- and typically more than enough money. Specially with the "Orgasmus" ehhh sorry! "Erasmus" program. Do I have all friends with 160 IQ? I really doubt it.
Post reply on HN