Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

inflecthealth.medium.com

Post reply on HN
Print view View original

51–60 of 368 posts

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#51

Post by aflag »

ekidd wrote: post #15

Some of the best performances I've seen out of ChatGPT are essentially "junior programmer" level. But it still requires clear instructions and close supervision. But GPT's training data includes GitHub, and it's used to power Copilot. It has arguably been trained to be a programmer. In less familiar domains, like law or medicine, GPT has presumably undergone very limited training and tuning. It's essentially an "inte…

Not sure if I'd compare ChatGPT with a junior programmer. In my experience junior programmers tend to be builders. They will tend to code a lot of stuff and usually get reasonable results, but making some bad decisions that more experienced developers have already gone through. Inexperienced developers need supervision because otherwise they will just create heaps of code that will be hard to maintain later.

ChatGPT just doesn't do anything on its own and will never follow through with anything. So, it doesn't really need supervision. I feel like it's more like a professor or a very senior developer. Someone you'll consult with when you're having trouble. Obviously, our best specialists are still better than the AI, but if the current technology is perfected, it'd expect it to replace the specialist and not the junior programmer. Which obviously is a bit of a bleak future from a software engineer career's perspective.

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#52

Post by sixothree »

ChatGTP wrote: post #4

It’s funny because it’s almost the exact same problem I have with using it professionally for writing software.

I had an opportunity today that made it useful. I was trying to find a reliable method of counting the decimal places in a double (64 bit float). I couldn't help but feeling like the responses were not quite informed and possibly dangerous.

Chat GPT provided a solution, one that appeared better than most of what I had seen in the previous 15-30 minutes. I asked it twice to ensure safety and it improved its response.

I then asked it to explain a particular choice and it was thorough enough for me to feel comfortable. In the end I feel like it understood its reasoning better than some of the options I saw on SO.

This was GPT-4 and a fairly simple problem that was benefited by its understanding of the double type.

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#53

Post by colonCapitalDee »

bentcorner wrote: post #28
Turskarama wrote: post #7

I'm not really sure what he expected here, ChatGPT was not trained to be a doctor, it is far more general than that. Asking ChatGPT for medical advice is like asking someone who is very well read but has no experience as a doctor, and in that context it's doing very well. He also brings up one of the most salient points without really visiting it enough: ChatGPT does not ask for clarification, because it is not a kno…

> ChatGPT does not ask for clarification, because it is not a knowledge base trying to find an answer. All it does is figure out what character is statistically most likely to come next, it has no heuristic to know that there is a task it hasn't fully completed. It absolutely can, but it requires the right prompts. If you just say "give me a diagnosis" it won't, but if you prompt it with a "you are a doctor and I am…

ChatGPT's heuristic may lead it to emit a sequence of words that suggest to English speakers that they should provide clarification, but ChatGPT is congenitally incapable of determining that answering a question requires more information that was provided in the prompt and that clarification is needed. Rephrased, ChatGPT can ask for clarification, but only as a side effect of satisfying it's heuristic.

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#54

Post by flir »

jcims wrote: post #48

I know everyone scoffs at the concept of 'prompt engineer', but it really is an essential craft that we're going to have to come to terms with when interacting with large language models. Seeking suggestions on a more comprehensive prompt: https://sharegpt.com/c/sckAPvV Trying it out: https://sharegpt.com/c/LbpEIxi

Please see my "expert system" approach elsewhere in the thread. I felt it worked really well.

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#56

Post by yieldcrv »

ChatGTP wrote: post #4

It’s funny because it’s almost the exact same problem I have with using it professionally for writing software.

Yeah it bullshits a lot even on single liners

Its a faster and better stackoverflow for me, which is a big value add because the community and moderation aspect of SO is absurd

I love when it tells me about libraries and resources that I didn't know existed, when I didn’t necessarily ask the likely followup questions yet

Break big problems into smaller problems and let it tackle them

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#57

Post by intelVISA »

orcajerk wrote: post #19

Back in 2004, took a seminar in college regarding Decision Support Systems and how they a manager or doctor could ask it a question and get a response to help them make a decision. Went to the doctor couple a years ago and he charged $300 to google search the symptoms. No thanks.

You're expecting a doctor to have all relevant medical knowledge permanently memorized? That's the equivalent of coding interviews on random obscure topics where you can't look anything up.

Like a SWE their value is not perfect recall of every area of CS/medicine but ability to decipher arcane documentation into actionable outcomes.

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#58

Post by yieldcrv »

sixothree wrote: post #52
ChatGTP wrote: post #4

It’s funny because it’s almost the exact same problem I have with using it professionally for writing software.

I had an opportunity today that made it useful. I was trying to find a reliable method of counting the decimal places in a double (64 bit float). I couldn't help but feeling like the responses were not quite informed and possibly dangerous. Chat GPT provided a solution, one that appeared better than most of what I had seen in the previous 15-30 minutes. I asked it twice to ensure safety and it improved its response.…

protip: ask it to write the unit test for you

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#59

Post by [deleted] »

s0rce wrote: post #23
ekidd wrote: post #15

Some of the best performances I've seen out of ChatGPT are essentially "junior programmer" level. But it still requires clear instructions and close supervision. But GPT's training data includes GitHub, and it's used to power Copilot. It has arguably been trained to be a programmer. In less familiar domains, like law or medicine, GPT has presumably undergone very limited training and tuning. It's essentially an "inte…

training on uptodate.com would probably be a good start

[deleted]

Re: I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

#60

Post by dekhn »

gilbetron wrote: post #24

Being a scifi geek and AI geek and neuroscience geek for pretty much the past 40 years, I've read countless predictions and scenarios and stories about society's response as "true" AI begins to emerge. So watching it play out for real is creating this bizarre sense of deja vu combined with fascination and frustration and also some anxiety. This article and the comments in this thread are right up that alley. I mean,…

Ditto- I grew up reading Vernor Vinge, William Gibson, and John Varley and dedicated my career to applying ML to biological research problems while waiting for technology to reach the point where it could reliably be used to do some of the things that can be done in those books, biotechnologically (e.g. gene therapy for radical body modification).

I was very disappointed to learn when I first started (around '93-94) that neural networks were hard to design, and impossible to train more than a couple layers deep, and the labelled data wasn't there anyway. None of that really changed visibly for a couple decades, but multiple unrelated research projects made the necessary breakthroughs: straightforward NN models (CNNs, etc) that people could adapt, frameworks to train and predict without having to implement the entire ML stack from scratch, absurdly fast computers and acccelerator hardware, and massive labelled data sets.

So far, nothing I've seen is anywhere close to what I consider "true" AI. As we struggle with hard medical problems. But seeing computers handle text with such facility has been exciting because it's the sort of thing that normies can see and appreciate (and even be fooled by).

However, medical problems aren't something that ML is going to magically solve. Because medicine has huge structural issues that are harder to solve than making diagnoses.

It would be nice if James gleick wrote a new version of "What just happened" covering the time between the invention of Transformers and now.

Post reply on HN