Goodbye, data science
ryxcommar.com
Goodbye, data science
1–10 of 415 posts
Re: Goodbye, data science
#2This sounds accurate from my experience.
I was a data scientist who moved to engineering.
In a lot of orgs data science is there to help decision making, but human nature often makes it such that the decisions are already made.
Re: Goodbye, data science
#3As someone who also wants to move away from data science, data engineering is the last thing I would want to do. I think DE comes with many of the same problems and it's also a very ill-defined career track; I wouldn't recommend it to anyone. ML engineer or backend developer seem like much more appealing job profiles.
Re: Goodbye, data science
#4No post body was provided.
Re: Goodbye, data science
#5As someone who also wants to move away from data science, data engineering is the last thing I would want to do. I think DE comes with many of the same problems and it's also a very ill-defined career track; I wouldn't recommend it to anyone. ML engineer or backend developer seem like much more appealing job profiles.
would you highlight some of the biggest differences between ML engineering and data engineering? I believe they're sometimes used interchangebly especially if "data" is "datasets" for ML.
Re: Goodbye, data science
#6> Nobody knew or even cared what the difference was between good and bad data science work. Meaning you could absolutely suck at your job or be incredible at it and you’d get nearly the same regards in either case.
In my experience it's even a little bit worse than that. Approaches that are wrong from a statistics point of view are more likely to generate impressive seeming results. But the flaws are often subtle.
A common one I've seen quite many times is people using a flawed validation strategy (e.g. one which rewards the model for using data "leaked" from the future), or to rely on in-sample results too much in other ways.
Because these issues are subtle, management will often not pick up on them or not be aware that this kind of thing can go wrong. With a short-term focus they also won't really care, because they can still put these results in marketing materials and impress most outsiders as well.
Re: Goodbye, data science
#7Read this yesterday and absolutely loved it. Especially feel the pain regarding working with management. I think there's an accountability that comes with evaluating management decisions with data that nobody really wants. I still have a lot of half-formed thoughts/opinions about this but it really feels like data-driven requires strict discipline but the data people who would be accountable for that discipline both 1. aren't empowered to wield it and 2. probably don't want to wield it anyways.
Also agree about the simple tools but it's really hard from a career perspective. If I deploy XGBoost in production and put it on my resume, I'm making double my salary next year. If I can find a simple ruleset or linear regression that performs 90%+ as well as the XGBoost and put it in production then nobody cares even though it feels like distilling the complex down to the simple is really where the value is.
Re: Goodbye, data science
#8The upstream problems are real.
Yes, there are a few very meaningful dashboards that are high value to the business, and then there is analysis meant to justify a project.
After core dashboards have been built, a lot of data analysis is a political weapon and the data science people are designers of those weapons.
Re: Goodbye, data science
#9As someone who also wants to move away from data science, data engineering is the last thing I would want to do. I think DE comes with many of the same problems and it's also a very ill-defined career track; I wouldn't recommend it to anyone. ML engineer or backend developer seem like much more appealing job profiles.
would you highlight some of the biggest differences between ML engineering and data engineering? I believe they're sometimes used interchangebly especially if "data" is "datasets" for ML.
Data engineers don't work with machine learning at all. In fact one of the reasons why it developed as a job title over time waas specifically to differentiate the people who work with data but don't do any statistics or ML. If a DE who is doing "datasets for ML" decides to call themselves an ML engineer, they're just getting a bit too creative with the job titles (maybe they want a career change, more money, they think it sounds better, all sorts of reasons).
As an ML engineer you might need to do some data engineering work as part of your job but not the other way around.
Re: Goodbye, data science
#10"Managers will say they want to make data-driven decisions, but they really want decision-driven data"
Ooofff. This is too true. How often is the case that data is collected to test hypotheses vs confirming priors?