Ask HN: Machine learning engineers, what do you do at work?

Post by 1oooqooq » Sun, Jun 09, 2024, 4:44 PM UTC

jollofricepeas wrote: post #129

saulrh wrote: post #75

Earlier quoted context omitted.

> HIPAA my left foot. That was my experience as well - training documentation for fresh college grads (i.e. me) directed new engineers to just... send SQL queries to production to learn. There was a process for gaining permissions, there were audit logs, but the only sign-off you needed was your manager, permission lasted 12 months, and the managers just rubber-stamped everyone. That was ten years ago. Every time I t…

So… Is it surprising that Engineers in healthcare dont read the actual HIPAA documentation? Use of health data is permitted so long as it’s for payment, treatment or operations. Disclosures and patient consent are not required. There are helpful summaries on the US Department of Health and Humans Services website of the various rules (Security, Privacy & Notification) Source: https://www.hhs.gov/hipaa/for-professiona…

Post by fragmede » Sun, Jun 09, 2024, 6:27 PM UTC

sjducb wrote: post #195

fragmede wrote: post #188

Earlier quoted context omitted.

Disk space is cheap, so where it's possible to have 2 (or more) venvs, that seems easiest. The problem with venv is that they don't automatically activate. I've been using a very simple wrapper around python to automatically activate venvs so I can just cd into the directory and do python foo.py and have it use the local venv. I threw it online at https://github.com/fragmede/python-wool/

You’re already managing a few hundred dependencies and their versions. Each venv roughly doubles the number of dependencies and they all have slightly different versions. Now your 15 venvs deep, and have over 3000 different package version combinations installed. Your job is to upgrade them right now because of a level 8 CVE

    for venv in $(find ~/projects/ -type d -name 'venv'); do
      (
        source ${venv}/bin/activate
        pip install --upgrade pip
        pip install --upgrade package_with_cve
        deactivate
      ) &
    done

Post by specialist » Sun, Jun 09, 2024, 8:08 PM UTC

Spooky23 wrote: post #212

specialist wrote: post #204

Earlier quoted context omitted.

Mid aughts, we devs sat in the meetings discussing how to enable this. With teams from Google Health, Microsoft Health Vault, some pharmas. In attendance were marketing, biz dev, execs. Ours and theres. But no legal. Hmmm. And it would have worked if it weren't for those meddling teenagers. From your comment, I'm inferring they're now allowed to use health records for marketing.

My wife about 9 years ago was admitted to the hospital for a ruptured ectopic pregnancy. The baby would have been about two months along. On the would-be due date, a box arrived via FedEx of Enfamil samples and a chirpy “welcome baby” message. Of course there was no baby. It turns out that using prescription data, admission data and other information that is aggregated as part of subrogation and other processes, you…

Post by tmnvix » Sun, Jun 09, 2024, 8:52 PM UTC

fragmede wrote: post #188

sjducb wrote: post #130

Earlier quoted context omitted.

Two problems intersect: - You can’t have two versions of the same package in the namespace at the same time. - The Python ecosystem is very bad at backwards compatibility This means that you might require one package that requires foo below version 1.2 and another package that requires foo version 2 and above. There is no good solution to the above problem. This problem is amplified when lots of the packages were wri…

Disk space is cheap, so where it's possible to have 2 (or more) venvs, that seems easiest. The problem with venv is that they don't automatically activate. I've been using a very simple wrapper around python to automatically activate venvs so I can just cd into the directory and do python foo.py and have it use the local venv. I threw it online at https://github.com/fragmede/python-wool/

Post by nhuantd » Mon, Jun 10, 2024, 2:42 AM UTC

trybackprop wrote: post #15

In a given week, I usually do the following: * 15% of my time in technical discussion meetings or 1:1's. Usually discussing ideas around a model, planning, or ML product support * 40% ML development. In the early phase of the project, I'm understanding product requirements. I discuss an ML model or algorithm that might be helpful to achieve product/business goals with my team. Then I gather existing datasets from ana…