Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Medicine's Machine Learning Problem

bostonreview.net

Post reply on HN
Print view View original

51–60 of 112 posts

Re: Medicine's Machine Learning Problem

#51

Post by sungam »

sjg007 wrote: post #49
sungam wrote: post #39

Earlier quoted context omitted.

Situation may be slightly different in the NHS (national health service) where there is an overwhelming number of referrals from general practitioners for suspected skin cancer most of which turn out to be benign. As a consequence there is lack of capacity to see patients with other skin conditions. Of course it's always possible to see a private Dermatologist if you have health insurance or are happy to pay.

If that’s true that sounds like a different problem. Maybe they need to train more dermatologists? And if there are appointments available privately well... I don’t know what to say. Seems like an structural systemic failure which is odd. Maybe dermatologists are gaming the system to induce private pay..

The number of Dermatologists trained in the UK is entirely decided (and paid for) by central government. UK Dermatologists have for many years highlighted the need for training of more consultants.

Re: Medicine's Machine Learning Problem

#52

Post by n3ur0n »

gdebel wrote: post #14

I'm an eye surgeon and self-taught machine learning practitioner, I started to learn Python in 2016 when the deep learning hype was at his highest. After 3 years of research, playing with datasets, extracting and cleaning data from EMR and from different machines, I not sure that the biggest problem with the so-called "AI" is the inequalities that it can induce ; it is rather, is it useful at all ? This is a little b…

I do respect your experience and take on the matter, however, let's replace this statement:

"I'm an eye surgeon and self-taught machine learning practitioner, I started to learn Python in 2016 when the deep learning hype was at his highest."

with:

I'm a [machine learning researcher] and self-taught [ophthalmologist], I started to learn [ophthalmology] in 2016 when the [clinical medicine] hype was at his highest.

In this hypothetical situation, I bet you would instantly discount what I would have to say about ophthalmology because I clearly would not have the depth or experience to have an informed opinion on ophthalmology.

Over the past few years with the ML hype, I have noticed quite a few clinicians who have self taught some deep learning methods claim expertise in the subject area (not targeting you, a general observation). I feel like many clinicians do not understand the breadth of machine learning approaches. There is just so much to know! from robust statistics, non-parametric methods, to kernel methods. Deep learning and deep generative models are by no means the only tools at our disposal.

I absolutely agree with you though. Applied machine learning practitioners have been over selling their accomplishments -- which I believe is detrimental to progress in the field.

I would highly encourage you to collaborate with ML researchers who have spent a decade or more working on hard problems. From the other side, I can tell you I gained a lot discussing ideas with domain experts (neurologists, radiologists, functional neurosurgeons). They have insights that I could never have picked up by self teaching.

Re: Medicine's Machine Learning Problem

#53

Post by YeGoblynQueenne »

sungam wrote: post #46
YeGoblynQueenne wrote: post #41

Earlier quoted context omitted.

Is it necessary to go so far as making a diagnosis at all? Wouldn't it suffice to detect -and alert the user- that some of her moles have changed shape and she might need to have them looked at more carefully by an expert? This is a task that is very difficult to perform with the naked eye, especially for people with skin types that have lots of moles and an automated decision that could be relied on to detect otherw…

Yes this is the idea of mole mapping and there are 3D whole body photo imaging systems available for this with automated detection of changing lesions. It's harder to do on a smart phone but maybe possible.

Thanks - I'll have a look at "mole mapping" now that I know the term.

Re: Medicine's Machine Learning Problem

#54

Post by rscho »

n3ur0n wrote: post #52
gdebel wrote: post #14

I'm an eye surgeon and self-taught machine learning practitioner, I started to learn Python in 2016 when the deep learning hype was at his highest. After 3 years of research, playing with datasets, extracting and cleaning data from EMR and from different machines, I not sure that the biggest problem with the so-called "AI" is the inequalities that it can induce ; it is rather, is it useful at all ? This is a little b…

I do respect your experience and take on the matter, however, let's replace this statement: "I'm an eye surgeon and self-taught machine learning practitioner, I started to learn Python in 2016 when the deep learning hype was at his highest." with: I'm a [machine learning researcher] and self-taught [ophthalmologist], I started to learn [ophthalmology] in 2016 when the [clinical medicine] hype was at his highest. In t…

The troubles we are seeing with medical AI integration are not stemming from lack of personal abilities, though. The problem is clearly systemic, with medical data being currently mostly unusable (for both humans and machines, although humans often believe otherwise). So you can be as good as you want either in medicine or ML or both, material support is lacking for wide applicability of medical AI.

Re: Medicine's Machine Learning Problem

#55

Post by rscho »

krcz wrote: post #45
rscho wrote: post #34

Earlier quoted context omitted.

Then technicians will just put whatever diagnosis in the relevant text field and let the "AI" do their job (if the "AI" is deemed good enough). I've been working in healthcare for 15 years, and I don't have a single doubt that that's what would happen. Conversely, if the "AI" is deemed not good enough, it will be business as usual and nobody will so much as glance at the "AI" results.

My idea was: 1. Technician writes down their diagnosis 2. They submit it to the system 3. AI comes with its own analysis 4. Technician sees the outcome, they can update their assessment 5. Everything is saved into the system If one of technicians has too much errors in their initial assessments, it should raise a concern.

> 4. Technician sees the outcome, they can update their assessment

Will result in exactly what I described above.

> If one of technicians has too much errors in their initial assessments, it should raise a concern.

People will refuse AI oversight if there are associated sanctions. People will make every effort to game the system. Following that, you'll be left with:

a. Pay techs more, so they accept the new working conditions.

b. Fire all techs and make do with a (potentially suboptimal) AI system.

Yes, this is very much gate keeping at work.

Re: Medicine's Machine Learning Problem

#56

Post by dontreact »

usrnm wrote: post #18
dontreact wrote: post #3

It's a hard problem to work around which is rooted in the data available. I published this paper while I was at Google: https://www.nature.com/articles/s41591-019-0447-x The only data we were able to get at the time was mostly white patients. We talked to many hospitals but many were/are reluctant to share anonymized data for research. I'm not at Google so I'm not sure the status of the project now, but there was a r…

Honest question: does it really matter for lung cancer? Is there much difference between races in this particular field?

I spoke with one of the doctors who designed the criteria for determining whether a lung module found not through screening is cancer. He mentioned that they very nearly added a different criteria for Asian women, but were too worried about the potential backlash.

Re: Medicine's Machine Learning Problem

#57

Post by dontreact »

lhomdee wrote: post #25
dontreact wrote: post #3

It's a hard problem to work around which is rooted in the data available. I published this paper while I was at Google: https://www.nature.com/articles/s41591-019-0447-x The only data we were able to get at the time was mostly white patients. We talked to many hospitals but many were/are reluctant to share anonymized data for research. I'm not at Google so I'm not sure the status of the project now, but there was a r…

> Fundamentally, it seems to me like there just aren't as many lung cancer screening scans out there for non-white patients as there are for white patients. Just to qualify, you mean for the USA alone? It seems to me that part of the challenge is recognizing that the research needs to take place beyond just Western countries, or acknowledging it where such research is already occurring. Understandably many people wou…

At the time we were conducting this research lung cancer screening existed mostly in Europe, China and the U.S.

Note that if we had to conduct a 5 year multi site lung cancer screening trial ourselves in addition to doing the research, there would be basically no way of getting private funding for that. Those trials are very, very expensive and take several years to reach a conclusion.

Add to that the potential optics of Google “experimenting” in developing countries and the blowback risk from that...

Re: Medicine's Machine Learning Problem

#58

Post by dontreact »

DoreenMichele wrote: post #27
dontreact wrote: post #3

It's a hard problem to work around which is rooted in the data available. I published this paper while I was at Google: https://www.nature.com/articles/s41591-019-0447-x The only data we were able to get at the time was mostly white patients. We talked to many hospitals but many were/are reluctant to share anonymized data for research. I'm not at Google so I'm not sure the status of the project now, but there was a r…

How do we improve on the situation? Given economic realities and racist history (consider what happened in Tuskegee as one example), in the US you would need to provide free screenings to poor people under circumstances that convinced people of color they can trust you while signing the documents to let you have their data. This is a fairly high bar to meet and one most studies are probably making zero effort to real…

Note that lung cancer screening is covered my Medicare and thus already free for anyone over 65 who smoked a pack a day for 30 years (or equivalent aka more in less time).

My understanding is that there are many reasons that screening is not deployed more widely but the fact that it requires a 40 minute discussion with a physician, and those physicians in communities in need have very limited time.

Then there is the issue of getting people to show up and take part in preventative care which is itself tricky.

In any case, it was not something we were in a position to do much about as a small AI research team. Where I work now there is also a focus on trying to address this issue by reaching out to more hospitals to gather more diverse data, but there are still a lot of roadblocks to sharing data we have to work through and it’s a very slow process.

Re: Medicine's Machine Learning Problem

#59

Post by dontreact »

gdebel wrote: post #14

I'm an eye surgeon and self-taught machine learning practitioner, I started to learn Python in 2016 when the deep learning hype was at his highest. After 3 years of research, playing with datasets, extracting and cleaning data from EMR and from different machines, I not sure that the biggest problem with the so-called "AI" is the inequalities that it can induce ; it is rather, is it useful at all ? This is a little b…

I think you are not being creative enough about how AI can influence medical care, and also not aware of existing deployed solutions making significant clinical impact.

For example, viz.ai has a solution to help get brain bleeds spotted to the eyes of surgeons more quickly. It is deployed and has cut the average length of stay in the neuro ICU significantly

https://mobile.twitter.com/viz_ai/status/1314710308603133953

I work at Caption Health, where we are enabling novices to take echocardiography scans. The doctors who work with our technology found it extremely useful to help diagnose cardiac involvement during covid.

https://captionhealth.com/education/

As much as I have respect for the expertise of medical doctors, I would ask that you have respect for folks working to apply AI in medicine.

Re: Medicine's Machine Learning Problem

#60

Post by dontreact »

dnautics wrote: post #20
ps2fats wrote: post #17

Earlier quoted context omitted.

May I suggest, in response to your sentiment that applications of AI to medicine are lacking, is that you are seeing applications replace current medical practices. An AI diagnosis of a medical image seems redundant indeed, however in this situation a patient has seen a doctor out of complaints and has been sent to the radiologist for further investigation. This medical practice is reactionary, and suspicions are alr…

I disagree. You don't really want to routinize that level of medical surveillance, due to the classical Bayesian predictive power problem. When you come in with a complaint, it changes the prior and is additional evidence to revise the diagnosis on top of the screening information. What you do want out of AI is to flag areas of interest in imaging for example and help identify when records are at risk of being incorr…

Screening has been shown to be effective for lung cancer. With enough data, we can improve the posterior enough for certain applications that we don’t need the stronger prior of complaints.

Over time as AI improves, more and more diagnoses can look like this.

Post reply on HN