Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stanford computer diagnoses breast cancer more accurately than human doctor

extremetech.com

Post reply on HN
Print view View original

21–30 of 52 posts

Re: Stanford computer diagnoses breast cancer more accurately than human doctor

#21

Post by marshallp »

2mur wrote: post #16

I'm a pathologist and an avocational programmer. This is pretty neat material and is very relevant to me, as I have been trying to bone up my math chops with Khan Academy videos so that I can tackle some computer vision related work in pathology. With regards to the study, I will just point out that the system is not diagnosing breast carcinoma, but rather is producing a score which reflects the prognosis as relates…

I've seen a lot of doctors chime in various threads and say their jobs couldn't possible be done by machine learning. The same thing was said about self driving cars before the darpa challenges - when some profs actually put their mind to it, it was done in a couple of years. . If the data was available, there is probably quite a few people who can actually detect cancer in slides.

Re: Stanford computer diagnoses breast cancer more accurately than human doctor

#23

Post by hugh3 »

separated wrote: post #14

The funny thing is, most rads physicians would probably welcome this, as mammography is often the least preferred sub-specialty (my wife is a radiologist and feels this way, and I've encountered many radiologists with the same outlook).

mammography is often the least preferred sub-specialty (my wife is a radiologist and feels this way, and I've encountered many radiologists with the same outlook).

Why? Because breast cancer is hard to diagnose? Because nobody likes looking at floppy old-lady boobs all day?

Re: Stanford computer diagnoses breast cancer more accurately than human doctor

#24

Post by _delirium »

kenjackson wrote: post #13

One thing the article doesn't point out is that AI methods in the past did a poor job of detecting breast cancer. This is an important new result and big step for the field.

Unfortunately this still doesn't look like it's detecting breast cancer; it's scoring the severity of tumors, given prior knowledge that it's being presented tumors to analyze (but tumors of unknown severity).

Re: Stanford computer diagnoses breast cancer more accurately than human doctor

#25

Post by malingo »

2mur wrote: post #16

I'm a pathologist and an avocational programmer. This is pretty neat material and is very relevant to me, as I have been trying to bone up my math chops with Khan Academy videos so that I can tackle some computer vision related work in pathology. With regards to the study, I will just point out that the system is not diagnosing breast carcinoma, but rather is producing a score which reflects the prognosis as relates…

with your knowledge as a pathologist, would you agree that the success of this approach depends on the quality of the biopsy (i.e. obtaining a sample that faithfully represents the entire volume of tissue being investigated)? From my experience with prostate cancer screening research I know that biopsy scoring of prostate tumors can be very inaccurate compared to pathology scoring after a prostatectomy.

Re: Stanford computer diagnoses breast cancer more accurately than human doctor

#26

Post by tptacek »

marshallp wrote: post #21
2mur wrote: post #16

I'm a pathologist and an avocational programmer. This is pretty neat material and is very relevant to me, as I have been trying to bone up my math chops with Khan Academy videos so that I can tackle some computer vision related work in pathology. With regards to the study, I will just point out that the system is not diagnosing breast carcinoma, but rather is producing a score which reflects the prognosis as relates…

I've seen a lot of doctors chime in various threads and say their jobs couldn't possible be done by machine learning. The same thing was said about self driving cars before the darpa challenges - when some profs actually put their mind to it, it was done in a couple of years. . If the data was available, there is probably quite a few people who can actually detect cancer in slides.

That's not at all what he said. He said that this particular research involved a simpler (for CS) problem than either the title or his day to day job tackles.

Do you have a background in bio or medicine or computer vision? It's very interesting to see two informed people disagree about applied computer science, so I'd love you to contribute something more specific to the thread.

Re: Stanford computer diagnoses breast cancer more accurately than human doctor

#27

Post by pak »

Please: there ought to be a rule on HN that instead of linking to a popsci re-interpretation of a scientific result, you instead link directly to the paper.

http://stm.sciencemag.org/content/3/108/108ra113.full

If it is behind a paywall, as it probably is for this paper if you are not at a university, perhaps look around for the least hyperbolic re-interpretation of the paper, and link that instead.

http://www.genengnews.com/gen-news-highlights/image-analysis...

Even if you do not have access to the full paper because of the paywall, you should be able to still read the abstract and pick the popsci article that is fairest to what the paper actually says.

Otherwise, this happens: http://www.phdcomics.com/comics.php?f=1174 and I really don't like it when HN blindly follows the hivemind in furthering that phenomenon.

Let's examine what the authors actually say.

To directly compare the performance of the C-Path system to pathological grading on the exact same set of images, we applied standard pathological grading criteria to the TMA images used in the C-Path analysis. [...] the pathologist grading the images was blinded from the survival data. Although the C-Path predictions on the NKI data set were strongly associated with survival, the pathologic grade derived from the same TMA images showed no significant association with survival (log-rank P = 0.4), highlighting the difficulty of obtaining accurate prognostic predictions from these small tumor samples.

That's it. They make one remark about it, and do not focus on this at all elsewhere in the paper, because it was not the point of their study and the methodology for this little result is far from robust. Note they used one pathologist to run this little test. Also note that a high p-value is not evidence that the null hypothesis is true--it is quite possible that there is a relationship but the study is underpowered; this is a frequent point of confusion.

Let's please keep scientific statements in context. The original paper says nothing tantamount to computers diagnosing breast cancer more accurately than doctors. It is, principally, about a new morphological feature that the researchers believe is tied more strongly to survival according to their computational model.

Re: Stanford computer diagnoses breast cancer more accurately than human doctor

#28

Post by law »

2mur wrote: post #16

I'm a pathologist and an avocational programmer. This is pretty neat material and is very relevant to me, as I have been trying to bone up my math chops with Khan Academy videos so that I can tackle some computer vision related work in pathology. With regards to the study, I will just point out that the system is not diagnosing breast carcinoma, but rather is producing a score which reflects the prognosis as relates…

So I am actually far less concerned about a computer doing my job very well, which is actually looking at a piece of tissue on a slide and making a tumor versus not-tumor distinction. This is very hard to do and I think will continue to be even harder for computers/computer-vision/AI to do for a long time to come.

As a machine learning researcher, I do not disagree that it's "very hard" to make a single tumor-versus-not-tumor classifications. However, machine-learned classifiers are becoming much more adept at making these types of classifications, and will only improve with time. I think most of the problem right now is in creating a corpus with enough attributes and precise/reliable measurements to accurately train a classifier. Presently, computer vision algorithms are quite sensitive to noise; the human eye is much better at separating this noise from the actual signal.

Right now, I'm doing a much different kind of research: applying machine learning algorithms to legal documents (court opinions, statutes, patents, etc.) to generate legal analyses. To not worry about a computer "doing your job very well" is, in my opinion, a naive perspective. It's indisputable that one day, computers will be doing classification-type jobs very well. Embracing the technology will keep you ahead of the curve; dismissing it is suicide.

Re: Stanford computer diagnoses breast cancer more accurately than human doctor

#29

Post by marshallp »

tptacek wrote: post #26
marshallp wrote: post #21

Earlier quoted context omitted.

I've seen a lot of doctors chime in various threads and say their jobs couldn't possible be done by machine learning. The same thing was said about self driving cars before the darpa challenges - when some profs actually put their mind to it, it was done in a couple of years. . If the data was available, there is probably quite a few people who can actually detect cancer in slides.

That's not at all what he said. He said that this particular research involved a simpler (for CS) problem than either the title or his day to day job tackles. Do you have a background in bio or medicine or computer vision? It's very interesting to see two informed people disagree about applied computer science, so I'd love you to contribute something more specific to the thread.

Actually, he also said that he is not concerned that machine learning could initially detect cancer anytime soon. Also, if you've been machine learning trends recently(past 5 years), you'll see that deep learning methods (hinton, lacunn, ng, bengio) have actually made a huge leap over what came before, and are believed to be that "final" in some sense algorithm that can allow to tackle any learning problem. These just haven't spread widely anough yet.

Re: Stanford computer diagnoses breast cancer more accurately than human doctor

#30

Post by 2mur »

law wrote: post #28
2mur wrote: post #16

I'm a pathologist and an avocational programmer. This is pretty neat material and is very relevant to me, as I have been trying to bone up my math chops with Khan Academy videos so that I can tackle some computer vision related work in pathology. With regards to the study, I will just point out that the system is not diagnosing breast carcinoma, but rather is producing a score which reflects the prognosis as relates…

So I am actually far less concerned about a computer doing my job very well, which is actually looking at a piece of tissue on a slide and making a tumor versus not-tumor distinction. This is very hard to do and I think will continue to be even harder for computers/computer-vision/AI to do for a long time to come. As a machine learning researcher, I do not disagree that it's "very hard" to make a single tumor-versus-…

Thanks. I probably should have caveated that statement with some qualifier. I'm not particularly worried about it in the short-term timeframe in which I expect to be earning my bread as a practicing surgical pathologist. I absolutely expect at some point, machine learning will have progressed to the point to be able to perform my job as well as I can. I just think that using it in a clinical setting is a long way off, and far before we get to that point, we will be using pure molecular techniques to diagnose and classify cancer. If I had to list the tasks that I would imagine machine learning to be able to do at an expert level before my eye/brain combination could be replaced for diagnosing cancer on glass slides it may look like:

  - Segregate tissue from non-tissue (ie. blank slide space)
  
  - Classify the tissue (epithelial versus mesenchymal/stromal, mucin/matrix,
  exogenous material, etc).

  - Decide if it is normal or abnormal 

  - Decide if it is native to whatever organ that it reportedly came from or is it
  a metastases from somewhere else.

  - Select appropriate immunohistochemical stains or other ancillary tests (eg. 
  molecular assays).
  
  - Appropriately interpret the immunohistochemical stain (eg. is it a nuclear stain
  or cytoplasmic, or both, or membranous).
  Is that staining true staining or artifactual.

  - Interpret that in the clinical context of the patient's history.
And it has to do all of those things and correct and account for bad preparations from the laboratory, inconsistent staining from day-to-day and slide-to-slide, non-representative biopsies from the surgeons etc.

I think at the end of my career, they will eventually haul my carcass off of my microscope (hopefully!) in 40 to 50 years hence. I imagine that we will have made a ton of progress with computer vision in pathology, and will be using it in an adjunct fashion. To help with analyzing results and other tedious tasks that computer vision is well-suited for. But a lot of medicine, and especially pathology, is making intuitive connections and conclusions with inconsistent and incomplete evidence in day-to-day work on individual patients.

Post reply on HN