Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Machine learning engineers, what do you do at work?

news.ycombinator.com

Post reply on HN
Print view View original

131–140 of 233 posts

Re: Ask HN: Machine learning engineers, what do you do at work?

#131

Post by sigmoid10 »

spmurrayzzz wrote: post #13
sigmoid10 wrote: post #9

Earlier quoted context omitted.

If you're still doing ML locally in 2024 and also use an ARM macbook, you're asking for trouble.

Can you expand on this a bit? My recent experiences with MLX have been really positive, so I'm curious what footguns you're alluding to here. (I don't do most of my work locally, but for smaller models its pretty convenient to work on my mbp).

MPS implementations generally lag behind CUDA kernels, especially for new and cutting edge stuff. Sure, if you're only running CPU inference or only want to use the GPU for simple or well established models, then things have gotten to the point where you can almost get the plug and play experience on Apple silicon. But if you're doing research level stuff and training your own models, the hassle is just not worth it once you see how convenient ML has become in the cloud. Especially since you don't really want to store large training datasets locally anyways.

Re: Ask HN: Machine learning engineers, what do you do at work?

#132

Post by rwalle »

blitzar wrote: post #112
sigmoid10 wrote: post #9

Earlier quoted context omitted.

If you're still doing ML locally in 2024 and also use an ARM macbook, you're asking for trouble.

Nahh im l33t - intel macbook and no troubles.

can't read? parent clearly says "ARM macbook".

Re: Ask HN: Machine learning engineers, what do you do at work?

#133

Post by naveen99 »

bowsamic wrote: post #88
naveen99 wrote: post #63

Earlier quoted context omitted.

You’re living the dream. Why quit ?

Is that really your idea of a dream?

My dreams are usually more disturbing, or fun…

But yes. My work is kind of similar… I do some data curation / coding, and help 2 engineers who report to me. I enjoy it.

Re: Ask HN: Machine learning engineers, what do you do at work?

#134

Post by mountainriver »

loftyal wrote: post #115
davedx wrote: post #5

pip install pytorch Environment broken Spend 4 hours fixing python environment pip install Pillow Something something incorrect cpu architecture for your Macbook Spend another 4 hours reinstalling everything from scratch after nuking every single mention of python pip install … oh time to go home!

...and people criticise node's node_modules. At least you don't spend hours doing this

Not nearly as hard of a problem. Python does work just fine when it’s pure Python. The trouble comes with all the C/Cuda dependencies in machine learning

Re: Ask HN: Machine learning engineers, what do you do at work?

#135

Post by mountainriver »

globular-toast wrote: post #107
davedx wrote: post #5

pip install pytorch Environment broken Spend 4 hours fixing python environment pip install Pillow Something something incorrect cpu architecture for your Macbook Spend another 4 hours reinstalling everything from scratch after nuking every single mention of python pip install … oh time to go home!

You could learn how to use Python. Just spend one of those 4 hours actually learning. Imagine just getting into a car and pressing controls until something happened. This wouldn't be allowed to happen in any other industry.

Oh wow, I’ve been a Python engineer for over a decade and getting dependencies right for machine learning has very little to do with Python and everything to do with c++/cuda

Re: Ask HN: Machine learning engineers, what do you do at work?

#136

Post by aiforecastthway »

jollofricepeas wrote: post #129
saulrh wrote: post #75

Earlier quoted context omitted.

> HIPAA my left foot. That was my experience as well - training documentation for fresh college grads (i.e. me) directed new engineers to just... send SQL queries to production to learn. There was a process for gaining permissions, there were audit logs, but the only sign-off you needed was your manager, permission lasted 12 months, and the managers just rubber-stamped everyone. That was ten years ago. Every time I t…

So… Is it surprising that Engineers in healthcare dont read the actual HIPAA documentation? Use of health data is permitted so long as it’s for payment, treatment or operations. Disclosures and patient consent are not required. There are helpful summaries on the US Department of Health and Humans Services website of the various rules (Security, Privacy & Notification) Source: https://www.hhs.gov/hipaa/for-professiona…

> If it wasn’t then, theoretically the US healthcare industry would grind to a halt considering the number of intermediaries for a single transaction.

It just occurred to me that cleaning up our country's data privacy / data ownership mess might have extraordinarily positive second-order effects on our Kafkaesque and criminally expensive healthcare "system".

Maybe making it functionally impossible for there to be hundreds of middlemen between me and my doctor would be a... good thing?

Re: Ask HN: Machine learning engineers, what do you do at work?

#137

Post by llama_person »

tambourineman88 wrote: post #2

The opposite of what you’d think when studying machine learning… 95% of the job is data cleaning, joining datasets together and feature engineering. 5% is fitting and testing models.

Same here, it's tons of work to collect, clean, validate data, followed by a tiny fun portion where you train models, then you do the whole loop over again.

Re: Ask HN: Machine learning engineers, what do you do at work?

#139

Post by whiplash451 »

tambourineman88 wrote: post #2

The opposite of what you’d think when studying machine learning… 95% of the job is data cleaning, joining datasets together and feature engineering. 5% is fitting and testing models.

In a sense, the data _is_ the model (inductive bias) so splitting « data work » and « model work » like you do is arbitrary.

Re: Ask HN: Machine learning engineers, what do you do at work?

#140

Post by whiplash451 »

burnedout_dc4e3 wrote: post #52

I've been doing machine learning since the mid 2000s. About half of my time is spent keeping data pipelines running to get data into shape for training and using in models. The other half is spent doing tech support for the bunch of recently hired "AI scientists" who can barely code, and who spend their days copy/pasting stuff into various chatbot services. Stuff like telling them how to install python packages and u…

There are companies where applied scientists are required to code well. Just ask how they are hired before joining (that should be a positive feature).
Post reply on HN