In addition to what other commenters have written, you also have the problem that your inputs aren't clean or straightforward. People don't always answer fully truthfully, and sometimes give extraneous information that's not germane to their particular complaint. So, even if you get past the regulatory hurdles of obtaining and using data, and you have enough data to train your neural network, and you've handled the huge class imbalance issue with rare diseases, you
still have to deal with the fact that the inputs to your model are subject to all kinds of bias and confounding that's associated with human-generated data.
To make this concrete: I used to do a lot of HIV testing and counseling. Whenever I'd ask someone "how often do you use condoms?" they'd 100% of the time say "every time!" When I'd then switch to asking "when was the last time you didn't use a condom?" they'd often reply along the lines of "last week."
This kind of issue happens not just with awkward questions, but also with more "objective" data like labs and medications.
- Was the antibiotic script written for a patient necessary for their condition, or was the physician tired and couldn't fend off a particularly assertive patient who was convinced antibiotics would solve their viral infection?
- Are labs randomly ordered, or ordered in a targeted fashion based on the "hunch" of a physician? If the latter, then the presence of lab result would itself be correlated with having some disease, and could thus throw off your entire model.
- How does the patient's insurance status influence the tests they have ordered? At one clinic I'm aware of, they typically order HIV/GC/Chlam screenings as a bundle, but if the patient has a PPO insurance they are also more likely to throw in a syphilis screening.
- How does patient preference factor into what gets diagnosed, ordered, and treated? You might have two early stage, equal risk breast cancer patients, one whom opts for a double mastectomy because she saw what happened to a coworker who had breast cancer and doesn't want to risk recurrences, whereas the other opts for a lumpectomy because she thinks her risks are low enough to be more conservative in treatment.
These issues are all solvable, but for AI to have a useful impact in healthcare, there's a lot more work than just throwing a bunch of data into a deep learning model.