Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

‘Why didn’t you think this baby was ill?’ Decision-making in acute paediatrics

ep.bmj.com

Post reply on HN
Print view View original

61–70 of 75 posts

Re: ‘Why didn’t you think this baby was ill?’ Decision-making in acute paediatrics

#61

Post by opo »

kashprime wrote: post #34

Emergency doctor here, I have to say that diagnosing a sick child is one of the hardest things you can do in medicine (aside from figuring out why someone is feeling 'dizzy'). The limiting step here is that we can't do blood tests and x-ray/CT on everyone that walks in the door (which itself can lead to harm: https://emergencymedicinecases.com/overinvestigation-emergen... ) I work in one of the busiest ER's in Canada…

>...Anyone who can build an ML model to catch these clues is going to make billions.

Yea you would think so... Unfortunately, in medicine, just because a computer program is better then a Dr at diagnosing a patient is no guarantee it will be used. The classic example here was the MYCIN expert system developed in the 1970s. MYCIN was shown to outperform infectious disease experts by 1979 in a blind test:

>... Eight independent evaluators with special expertise in the management of meningitis compared MYCIN's choice of antimicrobials with the choices of nine human prescribers for ten test cases of meningitis. MYCIN received an acceptability rating of 65% by the evaluators; the corresponding ratings for acceptability of the regimen prescribed by the five faculty specialists ranged from 42.5% to 62.5%. The system never failed to cover a treatable pathogen while demonstrating efficiency in minimizing the number of antimicrobials prescribed.

https://jamanetwork.com/journals/jama/article-abstract/36660...

https://en.wikipedia.org/wiki/Mycin

If MYCIN hadn't been rejected by the medical profession, I am sure by now that we would have developed software to assist Doctors in all areas of medicine...

Re: ‘Why didn’t you think this baby was ill?’ Decision-making in acute paediatrics

#62

Post by Fomite »

michaelscott wrote: post #42
Fomite wrote: post #9

Earlier quoted context omitted.

One of the things my veterinarian colleagues talk about a lot is how do you make a diagnosis with "My chicken is acting funny" as the only information you have.

Being a vet is a tough job; they need to be able to diagnose problems with patients that can't speak and sometimes have very, very different anatomies. In farming communities in particular they can sometimes be critical to whether a livestock farmer succeeds or fails a season.

One of my favorite moments eavesdropping on some vet students taking their boards was listening to them complaining about the question involving ultrasounding a shark.

Re: ‘Why didn’t you think this baby was ill?’ Decision-making in acute paediatrics

#63

Post by Fomite »

lmkg wrote: post #40
tialaramex wrote: post #23

Earlier quoted context omitted.

And this is why I write software for a living. Because "Take the chicken to pieces to find out why it's broken" makes sense to me, so I'd be a rubbish chicken doctor.

From what I understand, that actually is common for chickens because they're usually livestock rather than pets. If one chicken might be sick, you care more about whether it's contagious than the outcome to the one chicken, so you do the most thorough diagnostics you can even if that requires an autopsy. Now dogs on the other hand...

I like working with vets for my stuff (epidemiology) because there are really interesting constraints.

As you mention, one chicken is expendable, but you're worried about tens of thousands of birds.

A sheep is...mostly expendable. You have a low, market driven cap on what you can spend.

A dog or cat, less so - there's an emotional connection, but there's also an upper threshold (with a wide range, and who has what threshold is often surprising).

A race horse...well, let's just say equine ICUs are really pretty nice, all things considered.

Re: ‘Why didn’t you think this baby was ill?’ Decision-making in acute paediatrics

#64

Post by Fomite »

sametmax wrote: post #11
Fomite wrote: post #7

Earlier quoted context omitted.

I work with a lot of physician colleagues (as well as veterinarians) and one of the things I wish we could capture was what words someone uses to trigger those gut feelings. The way a patient describes their pain, how we try to wrap our language around complex disease concepts...

If you are focusing on words, you are missing the big picture. 'Guts feelings' are what you get when your brain process a butt load of informations in a fuzzy way. This includes subtle smells, colors, moves, timing, shapes, sounds, chain of events and how they interact with each others. Words can be a tiny part of it, but usually gut feelings are about all those things you can't process as easily with rational thinki…

I chose words for two reasons:

- First, they're super-hard to capture. "What did a patient say to me?" doesn't end up on medical record systems.

- Second, that's what they often describe to me. That the patient was describing something, and they weren't sure why, but it made them worried.

Re: ‘Why didn’t you think this baby was ill?’ Decision-making in acute paediatrics

#65

Post by Fomite »

hycaria wrote: post #44
Fomite wrote: post #9

Earlier quoted context omitted.

One of the things my veterinarian colleagues talk about a lot is how do you make a diagnosis with "My chicken is acting funny" as the only information you have.

If I may ask, what exactly is it that you're doing ?

I think this is directed at me. If not, I apologize. I work on developing computational models for hospital infection control - both in human and veterinary settings.

Re: ‘Why didn’t you think this baby was ill?’ Decision-making in acute paediatrics

#66

Post by Bartweiss »

philipodonnell wrote: post #41
freeone3000 wrote: post #37

Earlier quoted context omitted.

That's not a great definition of parity. We'd want accuracy and specificity numbers linked to outcomes, not concurrence. The times when Watson agrees with doctors is effectively irrelevant - results would be the same whether or not he was added. We need to highlight whether, given a disagreement, Watson was better or worse for outcomes.

The trick is whether the cases of disagreement were themselves predictable! Given a fixed number of oncologists and deploying Watson only to support those oncologists, yes, you're correct that its only useful if it outperforms them. But I think of it more like Watson is a single hive-mind team of like a thousand med students near the end of residency: they get most things right but there are a few places where more e…

I think part of the problem for Watson is that it needs someone to gather the data, which is usually a doctor. So if you're pairing each patient with an oncologist for intake anyway, it's not clear that "examination plus enter all data into Watson" is a benefit over "examination plus make a decision".

I guess the ideal outcome for Watson (if it doesn't outpace expert oncologists) would be something like "experience nurse practitioner does an exam, and enters data into Watson" or maybe even "special oncology-trained NP does an exam with Watson".

The other part I don't know is what oncology accuracy rates look like. If the reason to not majorly expand screening is cost and availability, Watson could be huge. If it's false positives from our existing rate, there's a lot less value.

Re: ‘Why didn’t you think this baby was ill?’ Decision-making in acute paediatrics

#67

Post by Bartweiss »

Dylan16807 wrote: post #59
freeone3000 wrote: post #37

Earlier quoted context omitted.

That's not a great definition of parity. We'd want accuracy and specificity numbers linked to outcomes, not concurrence. The times when Watson agrees with doctors is effectively irrelevant - results would be the same whether or not he was added. We need to highlight whether, given a disagreement, Watson was better or worse for outcomes.

> The times when Watson agrees with doctors is effectively irrelevant If your question is "can Watson improve treatment over an oncologist?" (ignoring issues of expense and availability), then it doesn't matter. When you're talking about how close it is to "parity", it matters a lot . If Watson exactly matched the oncologist in all but one out of a million cases, and in that millionth case caused the patient to explo…

> When you're talking about how close it is to "parity", it matters a lot.

I guess the question there is how valuable 'close to parity' is. If you can treat more patients, or treat them faster, or even much cheaper, then close to parity is a big deal. If examining patients and avoiding destructive false positives are the limiting factors, it's not clear that parity-level decisions are a significant benefit.

Re: ‘Why didn’t you think this baby was ill?’ Decision-making in acute paediatrics

#68

Post by opportune »

j16sdiz wrote: post #36
philipodonnell wrote: post #35

Earlier quoted context omitted.

> Anyone who can build an ML model to catch these clues is going to make billions. You'd be surprised. IBM poured billions into Watson and appears to have been pretty successful in nearly reaching parity with a certified oncologist, but the results were dismissed because it didn't outperform them. > At first, Manipal used Watson to recommend treatment options for all cancer patients, said oncologist S.P. Somashekhar.…

It is used to recommend treatment options, not diagnosis

That’s just liability-related verbiage

Re: ‘Why didn’t you think this baby was ill?’ Decision-making in acute paediatrics

#69

Post by ggm »

philipodonnell wrote: post #35
kashprime wrote: post #34

Emergency doctor here, I have to say that diagnosing a sick child is one of the hardest things you can do in medicine (aside from figuring out why someone is feeling 'dizzy'). The limiting step here is that we can't do blood tests and x-ray/CT on everyone that walks in the door (which itself can lead to harm: https://emergencymedicinecases.com/overinvestigation-emergen... ) I work in one of the busiest ER's in Canada…

> Anyone who can build an ML model to catch these clues is going to make billions. You'd be surprised. IBM poured billions into Watson and appears to have been pretty successful in nearly reaching parity with a certified oncologist, but the results were dismissed because it didn't outperform them. > At first, Manipal used Watson to recommend treatment options for all cancer patients, said oncologist S.P. Somashekhar.…

A recent post here suggested Watson was recommending highly disadvantageous treatments and MDs became reluctant to continue because it wasn't working.

Re: ‘Why didn’t you think this baby was ill?’ Decision-making in acute paediatrics

#70

Post by hycaria »

Fomite wrote: post #65
hycaria wrote: post #44

Earlier quoted context omitted.

If I may ask, what exactly is it that you're doing ?

I think this is directed at me. If not, I apologize. I work on developing computational models for hospital infection control - both in human and veterinary settings.

Yes, it was :)

Interesting, public or private ? How closely do you work with practitionners ? Is their knowledge of value regarding your job ?

I ask because I happen to be a veterinarian who somehow ended up being a developer.

Post reply on HN