I was under the impression companies engaged in medical image diagnostics train their algos on verified clinical data (i.e for radiology the training set would have Biopsy results). Is crowd labeling valuable for medical diagnosis?
Absolutely. Biopsy proven is great, but often not available. This is especially true when you want a very large dataset, or if it’s something like a fracture or a bleed and not cancer.
Really glad to hear you're working on this problem. I've done a bit of grooming medical imaging datasets for AI projects. A big chunk of time is spent working on pipelines to properly de-identify the images. Everything from PHI hidden deep in the dicom headers to patient name burned into the image by the scanner or some workstation that opened it. How are you dealing with those challenges?
Really glad to hear you're working on this problem. I've done a bit of grooming medical imaging datasets for AI projects. A big chunk of time is spent working on pipelines to properly de-identify the images. Everything from PHI hidden deep in the dicom headers to patient name burned into the image by the scanner or some workstation that opened it. How are you dealing with those challenges?
That is certainly a challenge. Automated approaches of removing PHI often miss some things for the reasons you mentioned, and at the end of the day you need a person to verify that the image is free of PHI. Right now we depend on our clients to remove PHI, but we’re also working on a process where we verify some users credentials and have those users review cases for PHI before we release a potentially sensitive case to the crowd.
We're a medical imaging AI startup and one of the toughest aspects of getting quality training data is peer review of ground truth labeling. I love that your platform solves this problem with more than usual 3 annotators! Will be in touch shortly