Medicine's Machine Learning Problem • Hacker News Board

Re: Medicine's Machine Learning Problem

#61

Post by matheusmoreira » Fri, Jan 08, 2021, 3:26 PM UTC

gdebel wrote: post #14

I'm an eye surgeon and self-taught machine learning practitioner, I started to learn Python in 2016 when the deep learning hype was at his highest. After 3 years of research, playing with datasets, extracting and cleaning data from EMR and from different machines, I not sure that the biggest problem with the so-called "AI" is the inequalities that it can induce ; it is rather, is it useful at all ? This is a little b…

> I also realized that it is extremely difficult, if not impossible, to use data from EMR out of the box.

This is my biggest complaint with the EMR systems I've used and I've always wanted to improve this. I wonder if fellow doctors would be okay with using a simple structured language to describe data in an EMR.

For example:

  Height: 175 cm
  Weight: 70 kg

  Ethnicity: white
  Age: 40 years
  Creatinine: 0.9 mg/dl

An inference engine could use that data to calculate lots of things. Simple stuff like body mass index and creatinine clearance. The patient could be automatically classified in all possible scores given available data.

Doctors already do this work, we even input this exact same data into calculator apps. The innovation would be recognizing this data in the EMR text and doing it automatically. I think it would be a huge gain.

Re: Medicine's Machine Learning Problem

#62

Post by dontreact » Fri, Jan 08, 2021, 3:29 PM UTC

rahulnair23 wrote: post #38

ps2fats wrote: post #17

Earlier quoted context omitted.

May I suggest, in response to your sentiment that applications of AI to medicine are lacking, is that you are seeing applications replace current medical practices. An AI diagnosis of a medical image seems redundant indeed, however in this situation a patient has seen a doctor out of complaints and has been sent to the radiologist for further investigation. This medical practice is reactionary, and suspicions are alr…

Health policy is fraught with counter-intuitive phenomenon - and screening is one of them. Seems like it should help, but in practice leads to over-diagnosis. For example - Cancer rates jumped in Korea after screening with no impact on patient outcomes [1]. There are several others. [1] Lee, J. H., & Shin, S. W. (2014). Overdiagnosis and screening for thyroid cancer in Korea. The Lancet, 384(9957), 1848.

This is a false blanket statement. Also one that could change as we start to see human+ai performance be better than just human performance.

For lung cancer screening, NLST showed a 20% reduction in mortality and now NELSON has shown even stronger results in Europe.

This “all screening is bad” is FUD in the medical field, frankly. Yes it has to be studied and implemented carefully, but to make blanket statements about screening as a whole is factually incorrect.

Re: Medicine's Machine Learning Problem

#63

Post by epmaybe » Fri, Jan 08, 2021, 3:45 PM UTC

jsinai wrote: post #26

epmaybe wrote: post #13

Earlier quoted context omitted.

I certainly see and empathize where you are coming from. However, I would like to add that it kind of makes sense that you’d have more white people with scans available. Focusing on the USA for a second (and note that this likely applies elsewhere too, since screening programs are really only in full force in developed countries which, surprise surprise, are predominantly white). Non white patients don’t get screened…

> Non white patients are inherently fewer than white patients Look at global population statistics. While there are no official global figures for ethnicity, we can make some simple inferences based on continental distribution [1]: North America + Europe combined (17.19%) is barely as much as Africa (17.2%), and this is ignoring the fact that a good part of the North American population is non-white. There is nothing…

I think I am agreeing with you, based on your comment on imbalanced access to healthcare and screening programs. I’m saying the same thing in that data collection for ct scans is really only happening in countries that are predominantly white, not that it isn’t possible for other countries to implement programs and collect that data for training purposes.

Edit: unless of course you have found large databases that suggest my intuition is wrong?

Re: Medicine's Machine Learning Problem

#64

Post by gdebel » Fri, Jan 08, 2021, 3:54 PM UTC

n3ur0n wrote: post #52

gdebel wrote: post #14

I'm an eye surgeon and self-taught machine learning practitioner, I started to learn Python in 2016 when the deep learning hype was at his highest. After 3 years of research, playing with datasets, extracting and cleaning data from EMR and from different machines, I not sure that the biggest problem with the so-called "AI" is the inequalities that it can induce ; it is rather, is it useful at all ? This is a little b…

I do respect your experience and take on the matter, however, let's replace this statement: "I'm an eye surgeon and self-taught machine learning practitioner, I started to learn Python in 2016 when the deep learning hype was at his highest." with: I'm a [machine learning researcher] and self-taught [ophthalmologist], I started to learn [ophthalmology] in 2016 when the [clinical medicine] hype was at his highest. In t…

Haha, you are perfectly right. I totally admit that I'm an amateur with a low level of ML expertise.

One the other hand, ML researchers with a deep knowledge expertise are extremely hard to find, even among statisticians / programmers. I suppose that the people with a real expertise are working on their own startup or in FAANG.

This leads to a situation where the medical research involving ML is largely without interest or full of bias. It is easy to spot in the literature.

Re: Medicine's Machine Learning Problem

#65

Post by gdebel » Fri, Jan 08, 2021, 4:02 PM UTC

enriquto wrote: post #40

gdebel wrote: post #14

I'm an eye surgeon and self-taught machine learning practitioner, I started to learn Python in 2016 when the deep learning hype was at his highest. After 3 years of research, playing with datasets, extracting and cleaning data from EMR and from different machines, I not sure that the biggest problem with the so-called "AI" is the inequalities that it can induce ; it is rather, is it useful at all ? This is a little b…

> First, it took me a very long time to really, fully get that AI is not fundamentally different from a simple linear regression (infering a rule from data). I'm quite surprised by this. Doesn't each AI tutorial start by stating that very thing?

I assumed that basic ML was similar to the statistics I already knew, and that deep learning was inherently different. It had to be different, given how people talked about it. It is just an illustration of how the fuss made about AI at this time impacted researcher's minds with no expertise in ML. This is going down, fortunately.

Re: Medicine's Machine Learning Problem

#66

Post by gdebel » Fri, Jan 08, 2021, 4:12 PM UTC

dontreact wrote: post #59

gdebel wrote: post #14

I'm an eye surgeon and self-taught machine learning practitioner, I started to learn Python in 2016 when the deep learning hype was at his highest. After 3 years of research, playing with datasets, extracting and cleaning data from EMR and from different machines, I not sure that the biggest problem with the so-called "AI" is the inequalities that it can induce ; it is rather, is it useful at all ? This is a little b…

I think you are not being creative enough about how AI can influence medical care, and also not aware of existing deployed solutions making significant clinical impact. For example, viz.ai has a solution to help get brain bleeds spotted to the eyes of surgeons more quickly. It is deployed and has cut the average length of stay in the neuro ICU significantly https://mobile.twitter.com/viz_ai/status/1314710308603133953…

Hi, I did not intend to be disrespectful, sorry if you read my message like this.

I mainly intended to underline the fact that we (doctors) were promised a revolution in healthcare (AKA : to disappear) and we ended with diagnostic scores.

However, I gladly admit that I exaggerated and that AI technologies can be helpful in some cases, of course.

Re: Medicine's Machine Learning Problem

#67

Post by sjg007 » Fri, Jan 08, 2021, 4:16 PM UTC

sungam wrote: post #51

sjg007 wrote: post #49

Earlier quoted context omitted.

If that’s true that sounds like a different problem. Maybe they need to train more dermatologists? And if there are appointments available privately well... I don’t know what to say. Seems like an structural systemic failure which is odd. Maybe dermatologists are gaming the system to induce private pay..

The number of Dermatologists trained in the UK is entirely decided (and paid for) by central government. UK Dermatologists have for many years highlighted the need for training of more consultants.

Sure but the US has similar restrictions and problems. Medicare DME pays for almost all of the residency spots. In 2015 there were 400 dermatology spots in the USA. I guess one issue is travel time. A lot of folks don't live near cities and have access to specialty care.

Re: Medicine's Machine Learning Problem

#68

Post by gdebel » Fri, Jan 08, 2021, 4:17 PM UTC

rahulnair23 wrote: post #38

ps2fats wrote: post #17

Earlier quoted context omitted.

May I suggest, in response to your sentiment that applications of AI to medicine are lacking, is that you are seeing applications replace current medical practices. An AI diagnosis of a medical image seems redundant indeed, however in this situation a patient has seen a doctor out of complaints and has been sent to the radiologist for further investigation. This medical practice is reactionary, and suspicions are alr…

Health policy is fraught with counter-intuitive phenomenon - and screening is one of them. Seems like it should help, but in practice leads to over-diagnosis. For example - Cancer rates jumped in Korea after screening with no impact on patient outcomes [1]. There are several others. [1] Lee, J. H., & Shin, S. W. (2014). Overdiagnosis and screening for thyroid cancer in Korea. The Lancet, 384(9957), 1848.

This is exacerbated by the fact that if the AI told the doctor that there is a doubt, no doctor will take the risk of not doing a biopsy / scanner / MRI / surgery (depending on the case). Because, how would you defend yourself in front of the judge ? This is something we always have in mind.

This is how you end with false positives and over-diagnosis.

Re: Medicine's Machine Learning Problem

#69

Post by dontreact » Fri, Jan 08, 2021, 4:27 PM UTC

gdebel wrote: post #66

dontreact wrote: post #59

Earlier quoted context omitted.

I think you are not being creative enough about how AI can influence medical care, and also not aware of existing deployed solutions making significant clinical impact. For example, viz.ai has a solution to help get brain bleeds spotted to the eyes of surgeons more quickly. It is deployed and has cut the average length of stay in the neuro ICU significantly https://mobile.twitter.com/viz_ai/status/1314710308603133953…

Hi, I did not intend to be disrespectful, sorry if you read my message like this. I mainly intended to underline the fact that we (doctors) were promised a revolution in healthcare (AKA : to disappear) and we ended with diagnostic scores. However, I gladly admit that I exaggerated and that AI technologies can be helpful in some cases, of course.

Geoffrey Hinton really made things hard for folks on the AI side and even walked back that promise.

I think it’s the classic thing where it’s overestimated in the short term and underestimated in the long (longggg) term.

My sense is that for AI to have the full impact it will one day reach, it will take rethinking medical care entirely with online machine learning and data at the core of how decisions are made.

ML was able to revolutionize how ads are delivered (for better or worse, but at least reaching the objectives of those who deployed it) because you can update and deploy the models multiple times a day.

If we can one day get to a world like that where an ML mode is constantly learning and updating itself, and has seen far more patients than any individual doctor, then maybe we will see the sorts of bigger shifts that were imagined shortly after we started to see ML surpass human ability on long standing difficult tasks like object recognition.

Getting there is a long, long road where we need to learn to work together with AI and figure out where the holes are in terms of robustness, earn trust over years of successful deployment, and figure out how to properly put safety bounds around more frequent updates to models.

Re: Medicine's Machine Learning Problem

#70

Post by gdebel » Fri, Jan 08, 2021, 4:47 PM UTC

blueblisters wrote: post #16

gdebel wrote: post #14

I'm an eye surgeon and self-taught machine learning practitioner, I started to learn Python in 2016 when the deep learning hype was at his highest. After 3 years of research, playing with datasets, extracting and cleaning data from EMR and from different machines, I not sure that the biggest problem with the so-called "AI" is the inequalities that it can induce ; it is rather, is it useful at all ? This is a little b…

This is an interesting perspective. Since you're an eye surgeon, this might be a relevant question. What do you think of the relative success of Diabetic Retinopathy (DR) diagnostic models, especially the FDA approval of the clinical trials Digital Diagnostics (formerly IDxDR) [1]? Their approach to the model architecture was slightly different from the black-box approach of other labs, wherein IDxDR's model is train…

Honestly, I don't know what to think. Ophthalmology is a great field for AI researchers (lots of images: the eye is an organ that you can photograph and analyze visually in every angles, almost like in dermatology). In Ophthalmology, diabetic retinopathy is an evident take : lots of people involved, lots of annotated pictures available, screening programs.

However, I would like to see the performances of the algorithm on different fundus camera. It is also important to realize that diabetic retinopathy classification is very easy to learn, to the point that if the screening is such a problem, it is easier to ask the person that takes the pictures (In France, a nurse or an orthoptist) to phone the doctor when he/she sees something strange on the eye fundus.