Ask HN: Machine learning engineers, what do you do at work? • Hacker News Board

Re: Ask HN: Machine learning engineers, what do you do at work?

#201

Post by Spooky23 » Sun, Jun 09, 2024, 2:00 PM UTC

outside1234 wrote: post #146

jollofricepeas wrote: post #129

Earlier quoted context omitted.

So… Is it surprising that Engineers in healthcare dont read the actual HIPAA documentation? Use of health data is permitted so long as it’s for payment, treatment or operations. Disclosures and patient consent are not required. There are helpful summaries on the US Department of Health and Humans Services website of the various rules (Security, Privacy & Notification) Source: https://www.hhs.gov/hipaa/for-professiona…

Define operations, because that sounds like a loophole that basically allows you to use it for anything

HIPPA protects you against gossipy staff. Beyond that, it’s mostly smoke - entire industries are built on loopholes.

Pre-HIPPA, the hospital would tell a news reporter about your status. Now, drug marketing companies know about your prescriptions before your pharmacy does.

Re: Ask HN: Machine learning engineers, what do you do at work?

#202

Post by specialist » Sun, Jun 09, 2024, 2:33 PM UTC

jollofricepeas wrote: post #129

saulrh wrote: post #75

Earlier quoted context omitted.

> HIPAA my left foot. That was my experience as well - training documentation for fresh college grads (i.e. me) directed new engineers to just... send SQL queries to production to learn. There was a process for gaining permissions, there were audit logs, but the only sign-off you needed was your manager, permission lasted 12 months, and the managers just rubber-stamped everyone. That was ten years ago. Every time I t…

So… Is it surprising that Engineers in healthcare dont read the actual HIPAA documentation? Use of health data is permitted so long as it’s for payment, treatment or operations. Disclosures and patient consent are not required. There are helpful summaries on the US Department of Health and Humans Services website of the various rules (Security, Privacy & Notification) Source: https://www.hhs.gov/hipaa/for-professiona…

This is why I advocate all PII data be encrypted at rest at the field level.

Worked on EMRs (during the aughts). Had yearly HIPAA and other security theater training. Not optional for any one in the field.

Of course we had root access. I forget the precise language, but HIPAA exempts "intermediaries" such as ourselves. How else could we build and verify the system(s). And that's why HIPAA is a joke.

Yes, our systems had consent and permissions and audit logs cooked in. So theoretically peeking at PII could be caught. Alas, it was just CYA. None of our customers reviewed their access logs.

--

I worked very hard to figure out how to protect patient (and voter) privacy. Eventually conceded that deanon always beats anon, because of Big Data.

I eventually found the book Translucent Databases. Shows how to design schemas to protect PII for common use cases. Its One Weird Trick is applying proper protection of passwords (salt + hash) to all other PII. Super clever. And obvious once you're clued in.

That's just 1/2 of the solution.

The other 1/2 is policy (non technical). All data about a person is owned by that person. Applying property law to PII transmutes 3rd party retention of PII from an asset to a liability. And once legal, finance, and accounting people get involved, orgs won't hoard PII unless it's absolutely necessary.

(The legal term "privacy" means something like "sovereignty over oneself", not just the folk understanding of "keeping my secrets private.)

Re: Ask HN: Machine learning engineers, what do you do at work?

#203

Post by specialist » Sun, Jun 09, 2024, 2:38 PM UTC

aiforecastthway wrote: post #136

jollofricepeas wrote: post #129

Earlier quoted context omitted.

So… Is it surprising that Engineers in healthcare dont read the actual HIPAA documentation? Use of health data is permitted so long as it’s for payment, treatment or operations. Disclosures and patient consent are not required. There are helpful summaries on the US Department of Health and Humans Services website of the various rules (Security, Privacy & Notification) Source: https://www.hhs.gov/hipaa/for-professiona…

> If it wasn’t then, theoretically the US healthcare industry would grind to a halt considering the number of intermediaries for a single transaction. It just occurred to me that cleaning up our country's data privacy / data ownership mess might have extraordinarily positive second-order effects on our Kafkaesque and criminally expensive healthcare "system". Maybe making it functionally impossible for there to be hun…

Correct. Would also mostly stop a lot of scams like identity theft.

A national identity service, like a normal mature economy, would be a massive public policy and public health win.

But it'd slightly decrease profits. So of course is quite impossible to implement.

Re: Ask HN: Machine learning engineers, what do you do at work?

#204

Post by specialist » Sun, Jun 09, 2024, 2:44 PM UTC

Spooky23 wrote: post #201

outside1234 wrote: post #146

Earlier quoted context omitted.

Define operations, because that sounds like a loophole that basically allows you to use it for anything

HIPPA protects you against gossipy staff. Beyond that, it’s mostly smoke - entire industries are built on loopholes. Pre-HIPPA, the hospital would tell a news reporter about your status. Now, drug marketing companies know about your prescriptions before your pharmacy does.

Mid aughts, we devs sat in the meetings discussing how to enable this. With teams from Google Health, Microsoft Health Vault, some pharmas.

In attendance were marketing, biz dev, execs. Ours and theres. But no legal. Hmmm.

And it would have worked if it weren't for those meddling teenagers.

From your comment, I'm inferring they're now allowed to use health records for marketing.

Re: Ask HN: Machine learning engineers, what do you do at work?

#205

Post by reportgunner » Sun, Jun 09, 2024, 3:19 PM UTC

ninkendo wrote: post #124

davedx wrote: post #5

pip install pytorch Environment broken Spend 4 hours fixing python environment pip install Pillow Something something incorrect cpu architecture for your Macbook Spend another 4 hours reinstalling everything from scratch after nuking every single mention of python pip install … oh time to go home!

I count at least a half dozen “just use X” replies to this comment, for at least a half dozen values of X, where X is some wrapper on top of pip or a replacement for pip or some virtual environment or some alternative to a virtual environment etc etc etc. Why is python dependency management so cancerously bad? Why are there so many “solutions” to this problem that seem to be out of date as soon as they exist? Are pyt…

Just replace python with mac or apple in your comment and I think you will understand.

Re: Ask HN: Machine learning engineers, what do you do at work?

#206

Post by reportgunner » Sun, Jun 09, 2024, 3:21 PM UTC

SushiHippie wrote: post #19

davedx wrote: post #5

pip install pytorch Environment broken Spend 4 hours fixing python environment pip install Pillow Something something incorrect cpu architecture for your Macbook Spend another 4 hours reinstalling everything from scratch after nuking every single mention of python pip install … oh time to go home!

Can recommend using conda, more specifically mambaforge/micromamba (no licensing issues when used at work). This works way better than pip, as it does more checks/dependency checking, so it does not break as easily as pip, though this makes it definitely way slower when installing something. It also supports updating your environment to the newest versions of all packages. It's no silver bullet and mixing it with pip…

Whenever anyone recommends conda I automatically assume they don't know much about python.

Re: Ask HN: Machine learning engineers, what do you do at work?

#207

Post by reportgunner » Sun, Jun 09, 2024, 3:25 PM UTC

avmich wrote: post #10

makapuf wrote: post #8

Earlier quoted context omitted.

Maybe pip should not work by default (but python -m venv then pip install should)

Legends say there were times when you'd have a program.c file and just run cc program.c, and then could just execute the compiled result. Funny that programmer's job is highly automatable, yet we invent ourselves tons of intermediate layers which we absolutely have to deal with manually.

You can do the same in python if you're not new to programming.

Re: Ask HN: Machine learning engineers, what do you do at work?

#208

Post by reportgunner » Sun, Jun 09, 2024, 3:32 PM UTC

eb0la wrote: post #175

shoggouth wrote: post #171

Earlier quoted context omitted.

Could you talk more about “BTW: perl is much better than python on this.”?

I haven't touched Perl in more than 20 years... ... but I (routinely) miss something like: $variable = something() if sanity_check() And do_something() unless $dont_do_that

Both work in python:

        variable = something() if sanity_check() else None


        do_something() if not dont_do_that else None

Re: Ask HN: Machine learning engineers, what do you do at work?

#209

Post by reportgunner » Sun, Jun 09, 2024, 3:40 PM UTC

rldjbpin wrote: post #196

junior level role, but currently it is a mix of working like a proxy product owner and half software engg. the users are researchers and have deep technical knowledge of their use case. it is still a challenge to map their needs into design decisions of what they want in the end. thanks to open-source efforts, the model creation is rather straightforward. but everything around making that happen and shaping it like a…

Junior level role product owner ?

Re: Ask HN: Machine learning engineers, what do you do at work?

#210

Post by apwheele » Sun, Jun 09, 2024, 4:01 PM UTC

ProjectArcturis wrote: post #162

schmookeeg wrote: post #66

Getting my models dunked on by people who can't open MS Outlook more than 3 tries out of 5, however, have a remarkable depth and insight into their chosen domain of expertise. It's rather humbling. Collaborating with nontechnical people is oddly my favorite part of doing MLE work right now. It wasn't the case when I did basic web/db stuff. They see me as a magician. I see them as voodoo priests and priestesses. When…

What business are you in that predicting health data can make you millions?

The scale of health insurance claims is incredible, my company has a process that simply identifies when car insurance should pay the bill instead of medicaid/medicare post traffic accident (subrogation). Seems a minor thing right? We process right at 1 billion a year in related claims (and I don't know our US market share, maybe like 10-20%).

I am guessing every datascientist that works for a BlueCross BlueShield at individual states deals with processes that touch multiple-million dollars of claims.

We even now have various dueling systems -- one company has a model to tack on more diagnoses to push up the bill, another has a process to identify that upcoding. One company has models to auto accept/deny claims, another has an automatic prior authorization process to try to usurp that later denial stage, etc.