Live data from Hacker News

Goodbye, data science

ryxcommar.com

191–200 of 415 posts

Re: Goodbye, data science

#191

This blog post is directed at me, personally. Thanks W.D. This isn't just Data Science, I'd say that the gripes of the author are valid for about 60% of activity in tech companies. Not saying we can just eliminate 60% of it, but a lot of it supplies non-quantitative value that is driven by fashion (subset of politics) and direct politics. There are a lot of naked emperors walking around with lots of folks standing as…

Not just tech companies either. Go bigger. It's possibly 60% of activity in US white collar economy. (I can't speak for those who work with their bodies, could be or not).

Re: Goodbye, data science

#192
> Shitty management & insane projects > Shitty code & shitty data science

Shouldn't the author conclude that "Goodbye, shitty companies"? How were the two problems unique to data science?

Re: Goodbye, data science

#193

Earlier quoted context omitted.

I become wary any time someone utters the phrase, "show me the data" or any variation there of. There is a specific type of leader who thinks that within the data lurks a magical solution just waiting to be discovered. There is also the leader who uses data as a trump card to win arguments and these folks are perhaps even worse. This is not new. The origination of the phrase, "lies, damned lies, and statistics," can…

I have only heard “show me the data” when someone wants someone else to support a claim. I do not see why this would necessarily be a bad thing.

> I have only heard “show me the data” when someone wants someone else to support a claim.

I've heard it a lot in situations where somebody is demanding a level of rigor that they themselves do not live up to. This is usually soon after they have framed the conversation around a solution that they want to pursue that also lacks any supporting data. That is to say, being data driven is on net good but it can also just be a thinly veiled appeal to status quo bias (which is itself not a terrible heuristic) or "highest-paid-person-in-the-room" bias.

Re: Goodbye, data science

#194
I realise the article was written for a specific audience for which this may be obvious, but what is the difference between data scientist and data engineer (in terms of what their job is)?

Re: Goodbye, data science

#195
For years I was low key obsessed with the idea that I should ditch regular programming, and become a data science. My thought process was very crude. Data science was mathsy and paid more, therefore it was a great career move!

After a few abortive attempts to learn statistics and linear algebra in isolation, I decided to sign up to that famous Andrew Ng online course.

I was bored out of my mind. I just did not find it at all interesting.

Not sure what my point is - maybe that a really easy way to learn if a seemingly lucrative and interesting thing is for you or not is to go ahead and learn the very basics of it directly, and see if you are at all motivated. I was not. And after that it was out of my mind.

Re: Goodbye, data science

#196
post #95

Earlier quoted context omitted.

>Meaning you could absolutely suck at your job or be incredible at it and you’d get nearly the same regards in either case. One of the things I don't like about statements like this said in a Data Science context, is that they are true outside of Data Science as well. Executives make big decisions, managers make smaller decisions, nobody can evaluate how good/bad they really were for months or years. Engineers build…

Not to get too off topic, but as a 35 year old engineer it seems the world in general has far fewer consequences than I was raised to expect. Everything from businesses with bullshit ideas flourishing at a loss, to January 6 even being possible (politics aside I expected the Capitol Police to crack a lot more skulls than they did once people started smashing windows), to the whole FTX situation and the tepid response…

Consequences often catch up slowly. It took years for Elizabeth Holmes to be sentenced because it takes time to collect evidence, build an airtight case, and give people their due process.

As I get older, I'm actually noticing more and more consequences catching up with people, albeit slowly. The people I knew who drank heavily through their 20s and 30s are in much worse shape than basically anyone who made an effort to stay healthy. People with poor diets and low physical activity are visibly worse off than others who paid attention to their inputs. I knew several people who got into recreational drugs in their 20s thinking they were safe because they educated themselves before hand, yet who ended up losing jobs, relationships, wealth, and a few who even lost their lives.

I've also noticed more peoples' career reputations catching up with them. It's not uncommon to interview someone only to later discover that they left a very negative reputation at a previous company where I happen to know someone.

I was very jealous of one of my peers who job-hopped his way up the salary ladder, joining companies and then immediately focusing on nothing other than interviewing at his next salary increase. He rotated through several of the big companies here until his reputation for demanding high salaries and then delivering nothing at all finally locked him out of any company with well-networked people who knew about him. He literally had to leave the state and go somewhere new to escape his past network and get new jobs after 10 years of this.

Consequences do catch up to people most times, but it's not immediately obvious. If you expect immediate justice or for people like SBF to go straight to jail the moment the headlines break, you're only seeing the beginning of the story.

Re: Goodbye, data science

#197

Earlier quoted context omitted.

Where are these jobs where you can interview this badly and still get hired because in my experience DS interviews are extremely hard and often expect people to have very high Stats skills as well as Data Structures/Algo skills at FAANG level.

These days if you have a company selling cat food or rivets for aerospace or providing taxi swrvice to a random city, or whatever, they might have a few data scientists helping them make "optimized" business choices. Obviously they won't have a very adcanced recruiting process for that.

Like the market for lemons: https://en.m.wikipedia.org/wiki/The_Market_for_Lemons

Re: Goodbye, data science

#198
> Like bro, you want to do stuff with “diffusion models”? You don’t even know how to add two normal distributions together! You ain’t diffusing shit!

Made me LOL. First time I did that while reading a tech post/blog in years. Also neatly describes half the HN audience fawning over the latest AI thing.

Re: Goodbye, data science

#199

I realise the article was written for a specific audience for which this may be obvious, but what is the difference between data scientist and data engineer (in terms of what their job is)?

Data scientist actually cook up and run the statistical/ML models on data and write reports about their "findings".

However, the data that data scientists want to use is often messy and comes from varied sources. Hence, data engineers do supporting infra work like cleaning/loading data from different databases, etc.

Re: Goodbye, data science

#200
post #6

> Nobody knew or even cared what the difference was between good and bad data science work. Meaning you could absolutely suck at your job or be incredible at it and you’d get nearly the same regards in either case. In my experience it's even a little bit worse than that. Approaches that are wrong from a statistics point of view are more likely to generate impressive seeming results. But the flaws are often subtle. A…

The problem is that nobody actually wants data science. They want data pseudoscience. And for the same reason that people tend to want pseudoscience instead of science in any other domain, too. Science is slow, tentative, and messy, and usually responds to questions with even more questions rather than with answers. Pseudoscience tends to be much more concerned with exuding confidence and providing clean-cut answers.…

> The problem is that nobody actually wants data science. They want data pseudoscience.

Technically, I think investors & owners would want the company to use real data science to improve products & maximize profits.

Everybody in the middle just wants to use data to lie to get promoted faster - because you don't get promoted for actually doing a good job - you get promoted for convincing people you did a good job, and lying is a VERY useful / effective tool.

Post reply on HN