Live data from Hacker News

FDA permits marketing of AI-based device to detect diabetes-related eye problems

fda.gov

31–40 of 104 posts

Re: FDA permits marketing of AI-based device to detect diabetes-related eye problems

#31
post #6

Earlier quoted context omitted.

Seems like they are using this to get a proper referral to a specialist rather than using this as sole diagnosis. The code itself is probably 15 lines using Tensorflow or other framework I'm guessing, but could be wrong.

If it is that simple I am wasting my time in my current job.

The algorithm could be simple. Evaluating all failure cases, analysing them, and getting the whole lot approved certainly isn't simple.

Re: FDA permits marketing of AI-based device to detect diabetes-related eye problems

#33
post #25
post #15

Earlier quoted context omitted.

I think the number is closer to 6%. 90% of people are accurately detected as not having the disease, i.e. 10% FP rate. So ~3m people would be falsely diagnosed as having the complication, to 200k/3m have the disease. 10% isn't a great number, but it isn't clear from this coverage whether this complication is generally asymptomatic or not. If there are symptoms to go with it, the numbers may be far better.

Also keep in mind that if there are positives general practitioners would refer to a specialist anyways for treatment. These specialists would be more than equipped to detect false positives. Teleretina imaging is becoming more and more prevalent as well with eyePACS and Welch-Allyn having dedicated interpretation services, so patients wouldn't have to necessarily go somewhere for verification. I'm more worried about…

I guess the baseline for comparing the 13% would be the nonspecialist rate of the same.

Re: FDA permits marketing of AI-based device to detect diabetes-related eye problems

#34

I think the title is fine, but a lot of the comments are applying what the software does to medicine in general. What this software is used for is very specific, but also very useful in that it is a common medical problem. It is used only to help diagnose diabetic retinopathy (ie eye damage caused by diabetes). This is AI Vision software used to analyze a photograph of someone's retina to detect damage. In essence it…

> What this isn't, is diagnosing a patient based on taking a history and inputting examination findings and labs, etc... We are still quite a bit of a way from that but I'm sure people are working hard on that as well.

This is one of the classic AI application domains -- for example this is why Stanford's Knowledge Systems Lab was near to the medical school and why the mainframe Zork was developed was owned by MIT's Medical Decision Making group.

Re: FDA permits marketing of AI-based device to detect diabetes-related eye problems

#35
post #5

The company is saying you don't need a specialist, but after bayes theorem (using 90%TN 87%TP and D(A)= 200,000 complication / 29,100,000 diabetes), the chance you have this condition after the machine says you do is 0.83%.

I'm more worried about the false-negatives.

What if you have the condition, the GP runs the test, and it turns out negative, and then the GP decides to not send you to a specialist ...

Can anybody infer what the confusion matrix looks like from reading the text?

Re: FDA permits marketing of AI-based device to detect diabetes-related eye problems

#37
post #5

The company is saying you don't need a specialist, but after bayes theorem (using 90%TN 87%TP and D(A)= 200,000 complication / 29,100,000 diabetes), the chance you have this condition after the machine says you do is 0.83%.

Yeah I was wrong on the math:

P(H) = .0069

P(D|H) = .87

P(D|H')= .1 (10% chance the test says you have it but you don't)

P(H|D) = 0.057 or about 6% probability you have complication after machine says you do

Re: FDA permits marketing of AI-based device to detect diabetes-related eye problems

#38
post #32

Ultimately, you will need a doctor somewhere along the diagnostic process so that someone is there to assume liability for incorrect diagnoses.

Insurance companies assume the liability for the doctor's diagnoses. I'm not sure why they'd be unwilling to do the same for the software's diagnosis. Somewhere, an actuary is willing to estimate that risk.

Re: FDA permits marketing of AI-based device to detect diabetes-related eye problems

#39
post #32

Ultimately, you will need a doctor somewhere along the diagnostic process so that someone is there to assume liability for incorrect diagnoses.

What's crazy is that if AI is better than doctors by some significant degree, what do we do when the doctor and AI disagree? Like if doctors are right 85% of the time, but the AI is 90%.

I guess we treat it as another doctor? Like if we have 4 opinions that agree, we go with that one regardless of the source of those opinions (as long as they meet some minimum competence threshold).

Re: FDA permits marketing of AI-based device to detect diabetes-related eye problems

#40
post #36

I wonder if doctors will become like pilots? Ie. Almost redundant, but essential, at the same time.

I'm a doctor and very much pro ML/AI. I'd love to have an autopilot I could watch in awe. Still a lot of practical tasks though that will be much harder to automate. And for the first few generation of AIs I guess someone will have to babysit them.
Post reply on HN