Ask HN: Machine learning engineers, what do you do at work? • Hacker News Board

Re: Ask HN: Machine learning engineers, what do you do at work?

#51

Post by geoduck14 » Fri, Jun 07, 2024, 10:55 PM UTC

dblohm7 wrote: post #14

tambourineman88 wrote: post #2

The opposite of what you’d think when studying machine learning… 95% of the job is data cleaning, joining datasets together and feature engineering. 5% is fitting and testing models.

As somebody whose machine learning expertise consists of the first cohort of Andrew Ng's MOOC back in 2011, I'm not too surprised. One of the big takeaways I took from that experience was the importance of getting the features right.

>was the importance of getting the features right.

Yeah, but also knowing which features to get right. Right?

Re: Ask HN: Machine learning engineers, what do you do at work?

#52

Post by burnedout_dc4e3 » Fri, Jun 07, 2024, 11:31 PM UTC

I've been doing machine learning since the mid 2000s. About half of my time is spent keeping data pipelines running to get data into shape for training and using in models.

The other half is spent doing tech support for the bunch of recently hired "AI scientists" who can barely code, and who spend their days copy/pasting stuff into various chatbot services. Stuff like telling them how to install python packages and use git. They have no plan for how their work is going to fit into any sort of project we're doing, but assert that transformer models will solve all our data handling problems.

I'm considering quitting with nothing new lined up until this hype cycle blows over.

Re: Ask HN: Machine learning engineers, what do you do at work?

#53

Post by giantg2 » Sat, Jun 08, 2024, 12:07 AM UTC

I've interviewed for a few of the ML positions and turned them down because they were just data jockey positions.

Re: Ask HN: Machine learning engineers, what do you do at work?

#54

Post by layer8 » Sat, Jun 08, 2024, 12:48 AM UTC

KeplerBoy wrote: post #36

tenache wrote: post #29

Although I studied machine learning and was originally hired for that role, the company pivoted and is now working with LLMs, so I spend most of my day working on figuring out how different LLMs work, what parameters work best for them, how to do RAG, how to integrate them with other bots.

Would you not consider LLMs as a part of machine learning?

They are the result of machine learning.

Re: Ask HN: Machine learning engineers, what do you do at work?

#55

Post by daemonologist » Sat, Jun 08, 2024, 12:56 AM UTC

mysteria wrote: post #22

davedx wrote: post #5

pip install pytorch Environment broken Spend 4 hours fixing python environment pip install Pillow Something something incorrect cpu architecture for your Macbook Spend another 4 hours reinstalling everything from scratch after nuking every single mention of python pip install … oh time to go home!

I thought most ML engineers use their laptops as dumb terminals and just remote into a Linux GPU server.

Yeah, the workday there looks pretty similar though, except that installing pytorch and pillow is usually no problem. Today it was flash-attn I spent the afternoon on.

Re: Ask HN: Machine learning engineers, what do you do at work?

#56

Post by daemonologist » Sat, Jun 08, 2024, 12:59 AM UTC

jshbmllr wrote: post #31

davedx wrote: post #5

pip install pytorch Environment broken Spend 4 hours fixing python environment pip install Pillow Something something incorrect cpu architecture for your Macbook Spend another 4 hours reinstalling everything from scratch after nuking every single mention of python pip install … oh time to go home!

I do this... but air-gapped :(

Oof. At our company only CI/CD agents (and laptops) are allowed to access the internet, and that's bad enough.

Re: Ask HN: Machine learning engineers, what do you do at work?

#57

Post by redwood » Sat, Jun 08, 2024, 1:14 AM UTC

Do people feel like they are more or less in demand with the hyper around genai?

Re: Ask HN: Machine learning engineers, what do you do at work?

#58

Post by redwood » Sat, Jun 08, 2024, 1:14 AM UTC

What are the tools people up to use? Feature platforms like Tecton on the list?

Re: Ask HN: Machine learning engineers, what do you do at work?

#59

Post by SushiHippie » Sat, Jun 08, 2024, 1:24 AM UTC

tasuki wrote: post #26

SushiHippie wrote: post #19

Earlier quoted context omitted.

Can recommend using conda, more specifically mambaforge/micromamba (no licensing issues when used at work). This works way better than pip, as it does more checks/dependency checking, so it does not break as easily as pip, though this makes it definitely way slower when installing something. It also supports updating your environment to the newest versions of all packages. It's no silver bullet and mixing it with pip…

I had a bad experience with Conda: - If they're so good at dependency management, why is Conda installed through a magical shell script? - It's slow as molasses. - Choosing between Anaconda/Miniconda... When forced to use Python, I prefer Poetry, or just pip with freezing the dependencies. The Python people probably can't even imagine how great dependency management is in all the other languages...

Yeah, I agree, maybe I should have also mentioned the bad things about it, but after trying many different tools that's the one that I stuck with, as creating/destroying environments is a breeze once you got it working and the only time my environment broke was when I used pip in that environment.

> The Python people probably can't even imagine how great dependency management is in all the other languages...

Yep, I wish I could use another language at work.

> Choosing between Anaconda/Miniconda...

I went straight with mamba/micromamba as anaconda isn't open source.

Re: Ask HN: Machine learning engineers, what do you do at work?

#60

Post by ClimaxGravely » Sat, Jun 08, 2024, 2:07 AM UTC

phaedrus wrote: post #48

davedx wrote: post #5

pip install pytorch Environment broken Spend 4 hours fixing python environment pip install Pillow Something something incorrect cpu architecture for your Macbook Spend another 4 hours reinstalling everything from scratch after nuking every single mention of python pip install … oh time to go home!

As an amateur game engine developer, I morosely reflect my hobby seems to actually consist of endlessly chasing things that were broken by environment updates (OS, libraries, compiler, etc.) That is, most of the time I sit down to code I actually spend nuking and reinstalling things that (I thought) were previously working. Your comment makes me feel a little better that this is not merely some personal failing of fo…

Happens in AAA too but we tend to have teams that shield everyone from that before they get to work. I ran a team like that for a couple years.

For hobby stuff at home though I don't tend to hit those types of issues because my projects are pretty frozen dependency-wise. Do you really have OS updates break stuff for you often? I'm not sure I recall that happening on a home project in quite a while.