not sure if this counts as ML engineering, but I support all the infra around the ML models: caching, scaling, queues, decision trees, rules engines, etc.
Ask HN: Machine learning engineers, what do you do at work?
21–30 of 233 posts
Re: Ask HN: Machine learning engineers, what do you do at work?
#22pip install pytorch Environment broken Spend 4 hours fixing python environment pip install Pillow Something something incorrect cpu architecture for your Macbook Spend another 4 hours reinstalling everything from scratch after nuking every single mention of python pip install … oh time to go home!
I thought most ML engineers use their laptops as dumb terminals and just remote into a Linux GPU server.
Re: Ask HN: Machine learning engineers, what do you do at work?
#23pip install pytorch Environment broken Spend 4 hours fixing python environment pip install Pillow Something something incorrect cpu architecture for your Macbook Spend another 4 hours reinstalling everything from scratch after nuking every single mention of python pip install … oh time to go home!
If you're still doing ML locally in 2024 and also use an ARM macbook, you're asking for trouble.
> ARM macbook
Funnily, the only real competitor for Nvidias' GPUs are Macbooks with 128GB of RAM.
Re: Ask HN: Machine learning engineers, what do you do at work?
#24Earlier quoted context omitted.
Maybe pip should not work by default (but python -m venv then pip install should)
Legends say there were times when you'd have a program.c file and just run cc program.c, and then could just execute the compiled result. Funny that programmer's job is highly automatable, yet we invent ourselves tons of intermediate layers which we absolutely have to deal with manually.
And then you'd have to deal with wrong glibc versions or mysterious segfaults or undefined behavior or the the code assuming the wrong arch or ...
Re: Ask HN: Machine learning engineers, what do you do at work?
#25[deleted]
Re: Ask HN: Machine learning engineers, what do you do at work?
#26pip install pytorch Environment broken Spend 4 hours fixing python environment pip install Pillow Something something incorrect cpu architecture for your Macbook Spend another 4 hours reinstalling everything from scratch after nuking every single mention of python pip install … oh time to go home!
Can recommend using conda, more specifically mambaforge/micromamba (no licensing issues when used at work). This works way better than pip, as it does more checks/dependency checking, so it does not break as easily as pip, though this makes it definitely way slower when installing something. It also supports updating your environment to the newest versions of all packages. It's no silver bullet and mixing it with pip…
I had a bad experience with Conda:
- If they're so good at dependency management, why is Conda installed through a magical shell script?
- It's slow as molasses.
- Choosing between Anaconda/Miniconda...
When forced to use Python, I prefer Poetry, or just pip with freezing the dependencies.
The Python people probably can't even imagine how great dependency management is in all the other languages...
Re: Ask HN: Machine learning engineers, what do you do at work?
#27Earlier quoted context omitted.
Sounds like a Data Scientist job?
This is a large problem in industry: defining away some of the most important parts of a job or role as (should be) someone else's. There is a lot of toil and unnecessary toil in the whole data field, but if you define away all of the "yucky" parts, you might find that all of those "someone elses" will end up eating your lunch.
> There is a lot of toil and unnecessary toil in the whole data field, but if you define away all of the "yucky" parts, you might find that all of those "someone elses" will end up eating your lunch.
See: the use of "devops" to encapsulate "everything besides feature development"
Re: Ask HN: Machine learning engineers, what do you do at work?
#28Pretty much the same as the others, building tool, data cleaning, etc. But something I don't see mentioned: experiment design/ data collection protocols
Re: Ask HN: Machine learning engineers, what do you do at work?
#29Although I studied machine learning and was originally hired for that role, the company pivoted and is now working with LLMs, so I spend most of my day working on figuring out how different LLMs work, what parameters work best for them, how to do RAG, how to integrate them with other bots.
Re: Ask HN: Machine learning engineers, what do you do at work?
#30Earlier quoted context omitted.
Sounds like a Data Scientist job?
This is a large problem in industry: defining away some of the most important parts of a job or role as (should be) someone else's. There is a lot of toil and unnecessary toil in the whole data field, but if you define away all of the "yucky" parts, you might find that all of those "someone elses" will end up eating your lunch.
It's not about "yucky" so much as specialization and only having a limited time in life to learn everything.
Should your reseacher have to manage nvidia drivers and infiniband networking? Should your operations engineer need to understand the math behind transformers? Does your researcher really gain any value from understanding the intricacies of docker layer caching?
I've seen what it looks like when a company hires mostly researchers and ignores other expertise, versus what happens when a company hires diverse talent sets to build a cross domain team. The second option works way better.