I meant - "crack" is near "damage" but also near "drug" which is near both "abuse and "doctor" and AI does love synonyms/homonyms, but if we are talking about racism - we know AI tools also favours medical research by native English speakers? https://www.bmj.com/content/392/bmj-2025-087581/rr-8 - so now a non-white written "accent" can silence non-white doctors.
As a doctor navigating AI scribes (which I don't use) it feels like we are being distracted with toys while tech companies figure out how to become what pharma was to the last generation. Interested in non-medical perspectives
Health NZ staff told to stop using ChatGPT to write clinical notes
61–63 of 63 posts
Re: Health NZ staff told to stop using ChatGPT to write clinical notes
#62Earlier quoted context omitted.
> I'm all for AI and it's great for things like copywriting, brainstorming and code generation It's funny how the assumption is always that LLMs are very useful in an industry other than your own.
I mean they are not wrong. For all the whinging about bugs and errors around here the software industry in general (some niche sub-fields excepted) long ago decided 80% is good enough to ship and we will figure the rest out later. This entire site is based on startup culture which largely prided itself on MVP moonshots. Plus plenty of places are perfectly fine with tech dept and the AI fire hose is effectively tech d…
> long ago decided 80% is good enough to ship and we will figure the rest out later
Sure, we can eat that 80% if we made the debt and know where the bodies are buried. When AI does it, it’s more like “50% is good enough and hopefully someone smart enough can fix it up to 60% when it breaks”, which inevitably means we get it to 55%.
> while it creates it at scale it can also help in understanding it.
This feels like cope, and I’m not trying to be snarky. I also know that I had to train my brain to skip google’s “AI Summary” on searches because I’ve had a handful of wrong answers from it - not technically correct, not correct with caveats, just flat out wrong. So sure, AI could make a bigger mess and then we could trust it to help sort it out, but even if it finds three real problems and one non-existent one, it has still made even more work to sort out.
> Still software is not medicine even if software is required in basically every industry now. It should more conservative and wait till things settle down before jumping in.
Agree. We had a big company meeting about becoming an AI-focused company (we do healthcare-related software) and I’m honestly a little worried - I don’t use any AI in my work, and when I asked a colleague who implemented our new build process for help with migrating my own repo they said “I don’t know, I asked GPT to do it”. And that’s why the pipeline has a mega-long ternary for the name which doesn’t resolve, so all of our runs are titled “if [[ $matrixType === …” but they’re using AI, and they’re going to be celebrated for it
Re: Health NZ staff told to stop using ChatGPT to write clinical notes
#63Earlier quoted context omitted.
It's honestly such a big problem. One of my colleagues uses an AI scribe. I can't rely on any of his chart notes because the AI sometimes hallucinates (I've already informed him). It also tends to write a ridiculous amount of detail that are totally unnecessary and leave out important details such that I still need to comb through patient charts for (med rec, consults, etc). In the end it ends up creating more work f…
> I'm all for AI and it's great for things like copywriting, brainstorming and code generation That's funny. I would have said the same thing about your field prior to reading your comment.
You would have? I don’t necessarily think I like the idea of using AI for anything that I’m going to send to anyone else, be it prose or code. But I’d rather get an AI generated pull request than have anyone in my medical team using it for, well, anything.