Earlier quoted context omitted.
To counter your professor opinion. The amount of extra time available as a student that I had to pursue things of interest was in the negative. All academic time was spent getting course content accomplished. I am a naturally curious individual but time limitations prevent further exploration in most circumstances. Additionally there is a relevancy factor weighed on top of it. If something looks curious I have to pre…
Well good luck then, in my experience the most free time I've ever had in my life was during college. I squandered massive amounts of that time doing things completely unrelated to education, and I definitely don't regret doing that. College isn't just about book learning after all. But still, BY FAR, college is the time of my life when I had the most free time to do whatever I wanted.
Goodbye, data science
41–50 of 415 posts
Re: Goodbye, data science
#42> Nobody knew or even cared what the difference was between good and bad data science work. Meaning you could absolutely suck at your job or be incredible at it and you’d get nearly the same regards in either case. In my experience it's even a little bit worse than that. Approaches that are wrong from a statistics point of view are more likely to generate impressive seeming results. But the flaws are often subtle. A…
I see this pointing to any of the following:
a) DS teams overpromising the accuracy of their approaches b) marketing driving the narrative and DS getting pulled along c) incompetence from the DS team
Re: Goodbye, data science
#43Earlier quoted context omitted.
To counter your professor opinion. The amount of extra time available as a student that I had to pursue things of interest was in the negative. All academic time was spent getting course content accomplished. I am a naturally curious individual but time limitations prevent further exploration in most circumstances. Additionally there is a relevancy factor weighed on top of it. If something looks curious I have to pre…
Well good luck then, in my experience the most free time I've ever had in my life was during college. I squandered massive amounts of that time doing things completely unrelated to education, and I definitely don't regret doing that. College isn't just about book learning after all. But still, BY FAR, college is the time of my life when I had the most free time to do whatever I wanted.
Re: Goodbye, data science
#44"Managers will say they want to make data-driven decisions, but they really want decision-driven data" Ooofff. This is too true. How often is the case that data is collected to test hypotheses vs confirming priors?
Rather than wanting to confirm priors, I believe this usually is a problem with neither the PM nor the data scientist ensuring that the problem formulation is good enough before diving in. I.e., what data would be needed to actually test the hypothesis? Do we have that data or not? Is the hypothesis even formulated in a way to be falsified in theory? I've seen so many analysis tasks where data scientists without ques…
Re: Goodbye, data science
#45Earlier quoted context omitted.
Data engineers don't work with machine learning at all. In fact one of the reasons why it developed as a job title over time waas specifically to differentiate the people who work with data but don't do any statistics or ML. If a DE who is doing "datasets for ML" decides to call themselves an ML engineer, they're just getting a bit too creative with the job titles (maybe they want a career change, more money, they th…
> Data engineers don't work with machine learning at all. This very much depend on the company. From experience DE is used as a catch-all title.
Re: Goodbye, data science
#46Earlier quoted context omitted.
Seems pretentious to me. I’ve never bothered to look through many things I use. I look extensively at how to use them and what the API offers. I have a good intuition for how most models work. I don’t really care about the specifics of the implementations. I have more important things to do. The hacker mentality, imo, is about identifying what’s useful for you to explore to accomplish whatever you need. Often that’s…
I agree with you. Super powers are seeing value in doing something, and then finding the easiest and most efficient path to get there. That said, sometimes I do like to read the code in libraries I use but often this is more for enjoyment with occasionally learning something interesting.
Re: Goodbye, data science
#47> But there’s also a part of me that’s just like, how can you not be curious? How can you write Python for 5 years of your life and never look at a bit of source code and try to understand how it works, why it was designed a certain way, and why a particular file in the repo is there? How can you fit a dozen regressions and not try to understand where those coefficients come from and the linear algebra behind it? I d…
The first issue is a lot of acquiring broad-spectrum knowledge involves risking quite an amount of money. A good DSLR cam can easily rack up a few thousand euros, a fully spec'd Mac Pro or larger drones cross the five digits without blinking. Messing around with gas and electricity can kill you, messing with water pipes can cause immense water damage. It takes a lot of ... let's say recklessness to even think about dealing with this if you're not a professional, and you have to have the resources in the first place.
But the real issue is time. Students, at least here in Europe, don't have the luxury of taking six or seven years for their basic diploma - "thanks" to the Bologna reforms, you're fucked if you can't make it in the designed timeframe as you won't be eligible for most kinds of financial aid. That means you simply cannot afford "wasting" a week to get that deep level of knowledge, you simply are happy enough if it runs well enough to get a passing grade. And once you've entered the workforce, it becomes even harder to have actual hobbies. It's one thing if you live alone, no one will bat an eye if you pull in an all-nighter on a weekend with just yourself, a crate of beer and a laptop and that's assuming you're not completely drained from your average 40 hours work week, 10 hours of getting to the workplace, and another 10 hours on domestic chores. When you live together with another person, the game completely changes: they also want time and attention from you - bonus points if your s/o has roughly the same interests that you have (which is why I suspect so many people meet their s/o at work). And with children... forget about hobbies of any kind if you don't have enough resources for either yourself or your s/o to be a stay-at-home parent.
This is why I so strongly advocate for a four-day and six-hour work week, a proper minimum wage and government-subsidized affordable housing for everyone. Just imagine what useful things people could run as side projects if they actually had the time to pull them off, not to mention the obvious physical and mental health benefits of not having to struggle with survival every single day. Add to that the elimination of "bullshit jobs" and an end of wasting the best minds of the world on financial bullshit (i.e. HFT, "quant investment funds") or advertising... or getting rid of racism and other discrimination. We as humanity could make so much more progress if we were not so hell-bent on exploiting each other.
Re: Goodbye, data science
#48Earlier quoted context omitted.
To counter your professor opinion. The amount of extra time available as a student that I had to pursue things of interest was in the negative. All academic time was spent getting course content accomplished. I am a naturally curious individual but time limitations prevent further exploration in most circumstances. Additionally there is a relevancy factor weighed on top of it. If something looks curious I have to pre…
I think this inclination to be curious can still be apparent even when someone doesn't have the time to pursue that inclination. It will be more subtle, but I think it's something rather fundamental that applies in broad ways across our lives.
Re: Goodbye, data science
#49Coles notes: Data Engineer - more money, more clout, less analysis / interesting projects, more job security, more infra style work Data Science - less money, a lot of random projects (Sometimes totally overqualified for), more analytical, don't have as much clout / confusion & lots of people don't actually understand capabilities.
50k+ lines of R, 10k+ lines of Julia, 5k+ in Python, C, and who knows what else. Most of it for what is, essentially, data engineering work.
Where do researchers with social science degrees fall on this scale? Less money, less clout. The projects are certainly interesting though (which is why I do what I do).
Re: Goodbye, data science
#50"Managers will say they want to make data-driven decisions, but they really want decision-driven data" Ooofff. This is too true. How often is the case that data is collected to test hypotheses vs confirming priors?
The preceding sentence is a hilariously cynical zinger:
“Those who have seen my Twitter posts know that I believe the role of the data scientist in a scenario of insane management is not to provide real, honest consultation, but to launder these insane ideas as having some sort of basis in objective reality even if they don’t.”