91–100 of 233 posts
pip install pytorch Environment broken Spend 4 hours fixing python environment pip install Pillow Something something incorrect cpu architecture for your Macbook Spend another 4 hours reinstalling everything from scratch after nuking every single mention of python pip install … oh time to go home!
If you're still doing ML locally in 2024 and also use an ARM macbook, you're asking for trouble.
Earlier quoted context omitted.
Can recommend using conda, more specifically mambaforge/micromamba (no licensing issues when used at work). This works way better than pip, as it does more checks/dependency checking, so it does not break as easily as pip, though this makes it definitely way slower when installing something. It also supports updating your environment to the newest versions of all packages. It's no silver bullet and mixing it with pip…
I had a bad experience with Conda: - If they're so good at dependency management, why is Conda installed through a magical shell script? - It's slow as molasses. - Choosing between Anaconda/Miniconda... When forced to use Python, I prefer Poetry, or just pip with freezing the dependencies. The Python people probably can't even imagine how great dependency management is in all the other languages...
Earlier quoted context omitted.
Just use conda.
Then you get one of my favorites: NVIDIA- has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.
Earlier quoted context omitted.
Maybe pip should not work by default (but python -m venv then pip install should)
Legends say there were times when you'd have a program.c file and just run cc program.c, and then could just execute the compiled result. Funny that programmer's job is highly automatable, yet we invent ourselves tons of intermediate layers which we absolutely have to deal with manually.
pip install pytorch Environment broken Spend 4 hours fixing python environment pip install Pillow Something something incorrect cpu architecture for your Macbook Spend another 4 hours reinstalling everything from scratch after nuking every single mention of python pip install … oh time to go home!
Do people doing ML/DS not use conda anymore?
The interminable solves were awful. Mamba made it better, but can still be slow.
Plenty of more esoteric packages are on PyPI but not Conda. (Yes, you can install pip packages in a conda env file.)
Many packages have a default version and a conda-forge version; it's not always clear which you should use.
In Github CI, it takes extra time to install.
Upon installation it (by default) wants to mess around with your .bashrc and start every shell in a "base" environment.
It's operated by another company instead of the Python Software Foundation.
idk, none of these are deal-breakers, but I switched to venv and have not considered going back.
not sure if this counts as ML engineering, but I support all the infra around the ML models: caching, scaling, queues, decision trees, rules engines, etc.
Earlier quoted context omitted.
> HIPAA my left foot. That was my experience as well - training documentation for fresh college grads (i.e. me) directed new engineers to just... send SQL queries to production to learn. There was a process for gaining permissions, there were audit logs, but the only sign-off you needed was your manager, permission lasted 12 months, and the managers just rubber-stamped everyone. That was ten years ago. Every time I t…
... you didn't have a UAT environment?
That was also the company where prod pushes happened once a month, over the weekend, and were all hands on deck in case of hiccups. It was an extraordinarily strong lesson in how not to organize software development.
(edit: if what you're really asking is "did every engineer have write access to production", the answer was, I believe, that only managers did, and they were at least not totally careless with it. not, like, actually responsible, no "formal post-mortem for why we had to use break-glass access", but it generally only got brought out to unbreak prod pushes. Still miserable.)
Earlier quoted context omitted.
Sounds like a Data Scientist job?
A good DS can double as an MLE.
Personally I think we calcified the roles around data a little too soon but that's probably because there was such demand and the space is wide.
pip install pytorch Environment broken Spend 4 hours fixing python environment pip install Pillow Something something incorrect cpu architecture for your Macbook Spend another 4 hours reinstalling everything from scratch after nuking every single mention of python pip install … oh time to go home!
Surely work can provide that?