I find the healthcare applications of this stuff so interesting. On the one hand, there are SO many reasons using LLMs to help people make health decisions should be an utterly terrible idea, to the point of immorality: - They hallucinate - They can't do mathematical calculations - They're incredibly good at being convincing, no matter what junk they are outputting And yet, despite being very aware of these limitatio…
Tell me you never taught service courses for pre-meds without telling me you never taught service courses for pre-meds ;)
> They hallucinate, They're incredibly good at being convincing, no matter what junk they are outputting
Describes about a third of the doctors I've interacted with, tbh.
> And the advice I got seemed useful, and helped kick off additional research and useful conversations with veterinary staff.
It's similar to "Dr. Google". Possible to misuse. But also, there's nothing magical about the medical guild initiation process. Lots of people are smart enough to learn and understand the bits of knowledge they need to accurately self-diagnose and understand tradeoffs of treatment options, then use a medical professional as a consultant to fill in the gaps and validate mental models.
Unfortunately, most medical professionals aren't willing to engage with patients in that mode and would rather misdiagnose than work with an educated patient. (My bil -- a medical doctor, and a fairly accomplished one at that -- has been chided for using "Dr Google" at an urgent care before.)
> Do the benefits outweight the risks? As with pretty much every ethical question involving LLMs, there are no obviously correct answers here.
At the end of the day, it doesn't matter. At least in the US, you won't have access to any meaningful treatment without going through the guild anyways.
I don't think that using LLMs for medical diagnosis is a good idea, but it's important to admit when the status quo is so thoroughly hollowed out of any moral or practical justification that even terrible ideas are better than the alternative of leaving things as they are.