Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FDA permits marketing of AI-based device to detect diabetes-related eye problems

fda.gov

Post reply on HN
Print view View original

61–70 of 104 posts

Re: FDA permits marketing of AI-based device to detect diabetes-related eye problems

#61

Post by SilasX »

bena wrote: post #39
j32fun wrote: post #32

Ultimately, you will need a doctor somewhere along the diagnostic process so that someone is there to assume liability for incorrect diagnoses.

What's crazy is that if AI is better than doctors by some significant degree, what do we do when the doctor and AI disagree? Like if doctors are right 85% of the time, but the AI is 90%. I guess we treat it as another doctor? Like if we have 4 opinions that agree, we go with that one regardless of the source of those opinions (as long as they meet some minimum competence threshold).

In practice? Pessimistic answer:

a) Usage still requires the presence of the Doctor.

b) The doctor does nothing but relay the AI’s message.

c) The doctor continues to charge the same and treat the same number of patients.

d) Everyone who expresses “hey isn’t the doctor redundant now? Shouldn’t we be treating more patients for cheaper” gets ridiculed as “one of those people”.

e) Edit: Also, the doctors’ association devotes significant resources to come up with memetically virulent reasons why the world would end if we took doctors out of the loop.

I mean, that’s how a lot of obviated jobs are currently treated...

Re: FDA permits marketing of AI-based device to detect diabetes-related eye problems

#62

Post by ska »

Herodotus38 wrote: post #28

I think the title is fine, but a lot of the comments are applying what the software does to medicine in general. What this software is used for is very specific, but also very useful in that it is a common medical problem. It is used only to help diagnose diabetic retinopathy (ie eye damage caused by diabetes). This is AI Vision software used to analyze a photograph of someone's retina to detect damage. In essence it…

PHI is a real barrier to your wish list, it's harder to get right than most people think and it is very very hard to get right in a distributed environment. And all the current advances in these general areas (voice recognition, transcription, semantic reasoning, etc.) are leaning heavily on distributed processing.

It's definitely a well identified market and there are people working on it, but I haven't seen much progress (not that I'm looking terribly closely at the moment).

From the point of the view of the last step, the ML one, I suspect the consenting is as much an issue as the PHI. Getting any real traction on this will likely require massive data sets and significant clinician time to aid training.

Re: FDA permits marketing of AI-based device to detect diabetes-related eye problems

#63

Post by YeGoblynQueenne »

bena wrote: post #39
j32fun wrote: post #32

Ultimately, you will need a doctor somewhere along the diagnostic process so that someone is there to assume liability for incorrect diagnoses.

What's crazy is that if AI is better than doctors by some significant degree, what do we do when the doctor and AI disagree? Like if doctors are right 85% of the time, but the AI is 90%. I guess we treat it as another doctor? Like if we have 4 opinions that agree, we go with that one regardless of the source of those opinions (as long as they meet some minimum competence threshold).

>> Like if doctors are right 85% of the time, but the AI is 90%.

There is always the possiblity that the doctors and the device are both right 90% of the time, but not the same 90% of the time.

Or that either the doctors or the device are right most of the time for the most severe cases but the other party is right only for the milder cases, etc.

It's not easy to look at absolute numbers here.

Re: FDA permits marketing of AI-based device to detect diabetes-related eye problems

#64

Post by pontus »

asperous wrote: post #5

The company is saying you don't need a specialist, but after bayes theorem (using 90%TN 87%TP and D(A)= 200,000 complication / 29,100,000 diabetes), the chance you have this condition after the machine says you do is 0.83%.

Keep in mind that the test wouldn't be administered on any randomly chosen diabetic patient but presumably rather on those who already exhibit some type of vision imparement consistent with the disease. As a result, the prior of 200k / 29M is not quite right and I'm guessing that the true prior is likely much higher.

Re: FDA permits marketing of AI-based device to detect diabetes-related eye problems

#65

Post by YeGoblynQueenne »

bobowzki wrote: post #40
laythea wrote: post #36

I wonder if doctors will become like pilots? Ie. Almost redundant, but essential, at the same time.

I'm a doctor and very much pro ML/AI. I'd love to have an autopilot I could watch in awe. Still a lot of practical tasks though that will be much harder to automate. And for the first few generation of AIs I guess someone will have to babysit them.

Ah, but who's going to treat the next few generations of AI, when there are no more doctors left who can "fly" without their "autopilot"?

Re: FDA permits marketing of AI-based device to detect diabetes-related eye problems

#66

Post by zaphod12 »

bena wrote: post #39
j32fun wrote: post #32

Ultimately, you will need a doctor somewhere along the diagnostic process so that someone is there to assume liability for incorrect diagnoses.

What's crazy is that if AI is better than doctors by some significant degree, what do we do when the doctor and AI disagree? Like if doctors are right 85% of the time, but the AI is 90%. I guess we treat it as another doctor? Like if we have 4 opinions that agree, we go with that one regardless of the source of those opinions (as long as they meet some minimum competence threshold).

As others have said - perform more tests. I've worked around molecular pathology testing for cancer (which comes up with a diagnosis based on data analysis after running DNA or RNA sequencing). If the molecular report differs from what the surgical pathologist saw when looking under the microscope, it's typically not 'it's cancer' vs 'it is not cancer', but more 'this is molecularly non-small cell lung cancer' vs 'this appears to be prostate cancer'. So what will happen is they'll do more staining on the sample, specific to what the molecular report came out with - and a lot of times - bam. That tumor in someone's prostate is actually lung cancer and needs to be treated that way.

Re: FDA permits marketing of AI-based device to detect diabetes-related eye problems

#67

Post by YeGoblynQueenne »

asperous wrote: post #5

The company is saying you don't need a specialist, but after bayes theorem (using 90%TN 87%TP and D(A)= 200,000 complication / 29,100,000 diabetes), the chance you have this condition after the machine says you do is 0.83%.

Results are reported as .87 accuracy, not true positive rate, at least in the Verge article.

I guess they may be misreporting sensitivity as accuracy.

Re: FDA permits marketing of AI-based device to detect diabetes-related eye problems

#68

Post by maxerickson »

asperous wrote: post #50
maxerickson wrote: post #19

Earlier quoted context omitted.

They are marketing the device strictly to make referrals to specialists. The FDA press release says as much anyway: https://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/u... If the images are of sufficient quality, the software provides the doctor with one of two results: (1) “more than mild diabetic retinopathy detected: refer to an eye care professional” or (2) “negative for more than mild diabetic retinopath…

Yeah from the article: > IDx-DR founder Michael Abràmoff told Science News. “It makes the clinical decision on its own.” But I would guess a specialist would still need to be involved since it's not a fool-proof system. A specialist might take other symptoms or variables into account when making the diagnosis or order further tests. While this tool might be useful for blanket screening considering that it is harmless…

It's specifically making the clinical decision I quote from the FDA press release, whether or not to refer to a specialist for further diagnosis.

It's not something primary care doctors currently do.

Re: FDA permits marketing of AI-based device to detect diabetes-related eye problems

#69

Post by bobowzki »

laythea wrote: post #45
bobowzki wrote: post #40

Earlier quoted context omitted.

I'm a doctor and very much pro ML/AI. I'd love to have an autopilot I could watch in awe. Still a lot of practical tasks though that will be much harder to automate. And for the first few generation of AIs I guess someone will have to babysit them.

I would imagine having such an autopilot would make one less professed at any particular subject.

Or perhaps it could lead to new insights?

Re: FDA permits marketing of AI-based device to detect diabetes-related eye problems

#70

Post by derefr »

lucio wrote: post #51

>AI software that helps doctors diagnose diabetic retinopathy like specialists is approved by FDA (theverge.com)

Well, sure, but the software's architecture probably isn't too particular to this use-case; it's just computer vision. Given FDA approval for using CV for this, we'll probably quickly see many other companies attempting to drive similar technologies to market.

Or, rather, we would if there were any existing companies in this space that could take advantage of this. The creator of this device, IDx (https://www.eyediagnosis.net), seems to be rather unique in being an entrepreneurial medical-device manufacturer; MDMs are a rather hide-bound lot.

Honestly, if there's going to be a wave of innovation in this space, I might expect it to come from the inorganic-chem-focused pharma companies, since they have the expertise in both materials science and machine-learning (from doing novel small-molecule detection studies) required to come up with the innovations. I expect they'd likely partner with one or more of the MDMs to build the hardware, but they would write the software.

Post reply on HN