Ask HN: Which new skills for a data science career?

I suspect a lot of python jobs would hire a good 'R person' knowing they'll be up to speed in a month or two, and vice versa. Similar to other CS domains, where strong fundamentals are much more important than whatever language you learned them in.

The thing about data sciencey stuff is that the data you're working with will often be extraordinarily messy. It's not uncommon to realize two weeks into a project that you made a terrible cleanup assumption on day one... then have to run hours and hours of mostly-hand-executed ETL work again. And then again when you realize you forgot that you ran a unix one-liner on a random input file, but didn't write it down anywhere.

So, one of the most important things to learn off the bat is learning how to clean data programmatically, specifically with the goal of making sure that your cleanup is repeatable at any step of the way. You want to be able to get to a point where you feel confident that you can mostly trivially recover after deleting all cache/temp files, tables, etc. Makefiles are great for this.

You'll save a lot of time in the long run if you can get good at this.

That aside, I work with many data scientists at clients. Most companies are still windows shops. Some may get linux. I've never seen a data science group using STATA successfully. R is ok, but in my experience is falling out of favor rapidly. While R is great for data science, it's relevance as the glue for other things is not so great. I would softly advise python.

I wouldn't bother learning viz tools like tableau beyond basic familiarity enough to slap it on your resume as something you've touched. It's all company specific. Same for the cloud and pipeline shit.

Think hard about what kinds of problems you want to solve. Most problems, that is, every single problem I have worked on, are poor fits for neural networks. PyTorch has never been relevant. Real world business problems just don't really benefit from that kind of stuff all that much. A real data science value add is picking up the low hanging fruit by being smarter about decisions that used to be made on gut instinct or whatever. Unless you really want to work on computer vision or whatever, it's just not something you need to bother with. I typically end up using a lightGBM model for pretty much everything at the end of the day. Which is basically just a fancier random forest.

Many data scientists shops fail to achieve anything because the data scientists are too complacent. Be a business person. Show a willingness to engage on problems and grill business folks for how they make decisions, and discuss how your model could change that process to add value. Don't make book reports on your findings and expect them to figure out how to use it. It's so, so common to have data scientists who don't feel qualified to take that part of the job on, and so they build nice models and visuals that everyone applauds and then collect dust on a shelf. Every output should be clearly dictating a path to generating value, be it money or some other worthy metric.

Highlighting that you're someone who can use their data science to solve real problems will be much more appealing in interviews than someone who can say they data science things the best. IMO, good question would be "I want to make sure I'm joining a group that has the power and support to really influence how decisions get made. Can you give some examples of the work that the business has adopted from your outputs?". Both from a virtue signaling perspective and a genuine desire to avoid joining a back office data science skunkworks that nobody listens to.

edit: I'm on the east coast, and not interested in working for big tech, for context. All of my examples here are from experience working with "normal" companies from dozens of industries, but no west coast tech.

I laughed a bit when I read this. This matches my experience exactly. Most of the crap I do daily is basically the ML equivalent of CRUD apps. Oh we need to make some data-driven decision here to optimize this metric. Typically any basic ML, especially something like LightGBM or CatBoost, works so well it's pointless trying to squeeze out a little more performance. And yeah, this is also normal-company shit, not MAANG stuff. There is so much low-hanging fruit at normal companies. Another bonus is that you get to do full-stack DS, from understanding the business problem, gathering the data, building the solution, getting it to production and showing the result. I feel like I get to keep up to date with pretty much all the dev tooling, leaving an exit back to software dev if things go to shit (I was full-stack SWE about 7 years ago), or just continuing to specialize in statistics and ML (currently applying mixed-effect models and survival analysis in a domain that no one has probably thought of applying it to).

Self study is not a job.

It's attractive because it's easier than looking for a job.

Or to put it another way, job hunting is the key skill for any career transition.

Good luck.