Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I became a software engineer after 5 years in data science

zachary-thomas.medium.com

Post reply on HN
Print view View original

41–50 of 58 posts

Re: I became a software engineer after 5 years in data science

#41

Post by thomzi12 »

kfk wrote: post #30
thomzi12 wrote: post #20

Hey everyone, OP here -- thanks for reading or discussing. Curious if anyone else has switched from data science to software engineering, or has thoughts on comparing the two as a career choice. I'm all ears!

I run a small data consulting company and whenever I find an open minded data scientist or analyst I tell them to consider data and software engineering or business analytics (learning about real business problems). DA, BA and SE roles are harder to find, manage more complexity, but most importantly are closer to having an impact as they are the ones close to prod. It’s a hard pitch, especially for juniors, AI and DS…

Yep. In any case, building a model is a just part of the puzzle -- serving it in production is a whole other skillset that is very valuable. imho

Re: I became a software engineer after 5 years in data science

#42

Post by eddsh1994 »

nkingsy wrote: post #31
eddsh1994 wrote: post #22

Earlier quoted context omitted.

The UK does, you can get chartered engineer status with a BSc & MSc in Software Engineering or Computer Science if it's applied enough with the IET. When I looked at the US i'm 99% sure I found similar degrees in Software Engineering too.

I’ve never seen a “software engineering” degree on a resume as an interviewer in the us.

Fwiw, I have. They're becoming more common - and probably a good thing too, covering important non-CS skills like requirements engineering that are mostly made up in small tech companies without that experience.

Re: I became a software engineer after 5 years in data science

#43

Post by bobleeswagger »

thomzi12 wrote: post #20

Hey everyone, OP here -- thanks for reading or discussing. Curious if anyone else has switched from data science to software engineering, or has thoughts on comparing the two as a career choice. I'm all ears!

I have always been an engineer, but when I started as an intern in 2015, most of my work centered around helping analyses: cleaning up CSVs, data mining, minor processing and transformations, etc. None of it was connected to production but these were ad-hoc requests. I think this start was invaluable to my general philosophy today (devops or die), and believe there are a lot of missed opportunities identifying the right analysts that should take the next step into complexity and help craft better devops processes for analysts. They're developers too.

One thing I don't think gets talked about enough: How much time analysts spend doing analysis vs. time spent dealing with data quality/architecture/process. So much analyst time is lost in the latter, which I think contributes greatly to burnout. But the growing intersection with engineering has and will continue to address this, albeit slowly.

Re: I became a software engineer after 5 years in data science

#44

Post by bobleeswagger »

melony wrote: post #24
gjvc wrote: post #17

Earlier quoted context omitted.

it's not clear to whom you are referring

People obsessed with protecting the term “engineer” are usually those who want to gatekeep their jobs and salary. In other words, they are worried about competition and want to keep the career “prestigious” and kick the ladder away.

It is a bit facetious to say that all analysts are engineers, or that it is "easy" for any analyst to become an engineer.

There is overlap, but they are not identical skillsets.

Analysts are certainly developers, but engineering involves more of a sense of scalability, analysis can do this, but it is generally more granular.

Re: I became a software engineer after 5 years in data science

#45

Post by lytefm »

thomzi12 wrote: post #20

Hey everyone, OP here -- thanks for reading or discussing. Curious if anyone else has switched from data science to software engineering, or has thoughts on comparing the two as a career choice. I'm all ears!

My educational background is much closer to data science (Math with CompSci Minor B.Sc., Data Science M.Sc.) but I've never worked in a pure data science role.

Training and tweaking models looks like the easy part of developing data driven products. Hiring compenent enough people also seems easier than for software engineers.

Many ML libraries produce good enough results without having to design elobare models myself. I see data science more as yet another tool in the belt of a a software engineer - like server admin, CI/CD, IaC, databases,...

Re: I became a software engineer after 5 years in data science

#46

Post by b33j0r »

I came of age in the industry when there was little distinction (python guy since 2004 or so). That provides many pros and cons.

The more you know broadly, the smarter you’ll sound. And generally, you will tend to be more useful and fulfilled.

If you want a career that is focused, you have to choose a focus. I’ve struggled with that. You need a broad base of knowledge to be good at understanding all things, and how they connect.

But eventually someone needs a lightbulb changed, and I’m testing the grounding of a nearby lightswitch while on the phone with a contractor about shielding noise from the antiquated home stereo system.

Edit: I should probably mention explicitly.

Investing in TensorFlow or CUDA is one thing. Investing in certain ML techniques or modeling domains another—and python yet another—

Systems and general engineering is a completely messier enchilada. You still get paid!

So. I guess. Do better than I. Choose a scale. It’s impossible to focus on everything unless that’s just an impulse you’ve got :)

Re: I became a software engineer after 5 years in data science

#48

Post by darth_avocado »

thomzi12 wrote: post #39
darth_avocado wrote: post #35

Sorry for the tangent, but medium is so bad. Can’t read content without installing the app or logging in. Part of the reason I stopped writing on medium and jumped to Substack.

Oh, I'm sorry about that! Here's a friend link that should allow you to bypass any paywall (if you've already read a couple paywalled posts): https://zachary-thomas.medium.com/how-i-became-a-software-en... Here's an earlier draft of the post on Quip: https://gstudent.quip.com/ILF1ABpvjsSh/Can-a-Data-Scientist-...

Thank you for sharing!

Re: I became a software engineer after 5 years in data science

#50

Post by carabiner »

I was hoping for more advice for those pursuing this transition, but it was more just an account of Google's incredible support for his wish to change careers. He never mentions his employer by name, but I think it'd be helpful to point this out. Those of us on the outside (I and a few other DS's I know are doing this) are spending 6 months or more grinding leetcode and studying system design, doing mock interviews, and applying to hundreds of jobs. So it can be much more painful if you don't have an employer that holds your hand (and pays you) through this entire transition.
Post reply on HN