Live data from Hacker News

Ask HN: What medical datasets do you need?

news.ycombinator.com

131–139 of 139 posts

Re: Ask HN: What medical datasets do you need?

#131
post #128

Earlier quoted context omitted.

If you train all of them on the same data, you will get similar answers. That doesn't mean that those answers will be more right than a less sure doctor.

Indeed. However this post is a "request for data" and I assume the algorithms of the future will be trained adequately :)

I suppose all algorithms will be trained on all available data, making the results pretty similar.

Re: Ask HN: What medical datasets do you need?

#133

Please oh please, Chronic Fatigue Syndrome. The potential for ML is absolutely huge. Some people have already collected or are working to collect genetic and other data on a large scale [1] [2] [3], so it does exist. CFS is interesting because: a) Patients' symptoms appear to fluctuate "randomly" but are actually typically a complex function of genetics, blood markers, exercise, diet, medication and other factors. b)…

If you are interested in working on ME /CFS, please contact me.

Re: Ask HN: What medical datasets do you need?

#134

Earlier quoted context omitted.

One potential problem with this - the question of liability, and who is responsible for diagnostic accuracy? In this case, for some "Lab on a Chip" device providing a patient directly with diagnostic information without the vetting of a human doctor, liability would sit with the company. IBM's Watson at MD Anderson Cancer center did not work out real well for them. In other words, using AI in the realm of medical dia…

What about the treatment side? Once you have a diagnosis, could we use AI to review the patient's medical record, compare outcomes of past patients with the same diagnosis and similar histories, and suggest adjustments of personalized treatments to optimize outcomes? Overall, of course, you're right. Liability is the problem with my suggestion. Doctors prescribe to treat, they also prescribe to meet the legally manda…

Fair points.

Extracting data from the EMR is very difficult because all EMR was originally intended to only be a storage place for data - not designed to output data back to a user.

Re: Ask HN: What medical datasets do you need?

#135

Earlier quoted context omitted.

Did you use something like Differential Privacy?

I would be very interested to know if anyone is using that in practice. It seems like a promising concept.

Apparently, DP has some detractors. I was told by my signal processing professor that differential privacy wasn't really a solution for privacy preserving data analysis. He said something along these lines: "if I know something about the underlying data distribution (Gaussian, etc.), it is possible to wash out the randomness."

Now, I don't understand DP well enough and information theory/signal processing still seems a bit like "dragons be here" to me. But, I want to take a stab at trying to reason why he said that.

For example, take randomized response (the only DP technique I understand). That is vulnerable to a longitudinal attack: a person can query repeatedly to wash out the randomness. If you think about it, isn't it the almost the inverse of a repetition code (error correction)? There, you're trying to use redundancy (repetition) to remove noise.

Re: Ask HN: What medical datasets do you need?

#136
post #53

Costs of all the procedures for each hospital. Whatever people get charged.

Medicare provider utilization and payment data (which is a large percentage of the total market in the United States) has been publicly available[1] for several years now. The Wall Street Journal won a Pulitzer Prize[2] for the analysis they did of the public data sets. [1]: http://graphics.wsj.com/medicare-billing/ [2]: https://www.wsj.com/articles/wsj-new-york-times-win-pulitzer...

That's medicare only, unfortunately. I have that dataset.

Re: Ask HN: What medical datasets do you need?

#137

Please oh please, Chronic Fatigue Syndrome. The potential for ML is absolutely huge. Some people have already collected or are working to collect genetic and other data on a large scale [1] [2] [3], so it does exist. CFS is interesting because: a) Patients' symptoms appear to fluctuate "randomly" but are actually typically a complex function of genetics, blood markers, exercise, diet, medication and other factors. b)…

For many autoimmune-related conditions patient’s symptoms also appear to fluctuate randomly, with symptoms such as pain and fatigue coming seemingly out of the blue. This includes chronic diseases such as Lupus, Rheumatoid Arthritis, Fibromyalgia and a long tail of other conditions [1].

People with these lifelong illnesses typically experience a roller-coaster of recurring symptom flare-ups, wreaking havoc with their lives. Yet there are patterns to the flare-ups. This is an opportunity to make a big difference for millions of people [2].

[1] https://www.aarda.org/disease-list/

[2] https://www.aarda.org/autoimmune-information/autoimmune-stat...

Re: Ask HN: What medical datasets do you need?

#138

Please oh please, Chronic Fatigue Syndrome. The potential for ML is absolutely huge. Some people have already collected or are working to collect genetic and other data on a large scale [1] [2] [3], so it does exist. CFS is interesting because: a) Patients' symptoms appear to fluctuate "randomly" but are actually typically a complex function of genetics, blood markers, exercise, diet, medication and other factors. b)…

For many autoimmune-related conditions patient’s symptoms also appear to fluctuate randomly, with symptoms such as pain and fatigue coming seemingly out of the blue. This includes chronic diseases such as Lupus, Rheumatoid Arthritis, Fibromyalgia and a long tail of other conditions [1]. People with these lifelong illnesses typically experience a roller-coaster of recurring symptom flare-ups, wreaking havoc with their…

The key to make sense of the data for these diseases is a record of patient’s symptoms. Assembling useful datasets is not only a question about access, but also about resolving human factors to successfully collect the essential information from patients.

A major challenge is to get a large number of patients to continuously track their symptoms. Most want to know what’s in it for them. It takes substantial incentives for people to regularly report outcomes and use wearables for data collection. Until we can make the marginal cost hit zero, they need to benefit from their efforts and investment, preferably instantly.

Re: Ask HN: What medical datasets do you need?

#139

Earlier quoted context omitted.

I would be very interested to know if anyone is using that in practice. It seems like a promising concept.

Apparently, DP has some detractors. I was told by my signal processing professor that differential privacy wasn't really a solution for privacy preserving data analysis. He said something along these lines: "if I know something about the underlying data distribution (Gaussian, etc.), it is possible to wash out the randomness." Now, I don't understand DP well enough and information theory/signal processing still seems…

You're right that with repeated sampling you can learn more about the data set. If I understand it correctly, I think the solution to that in DP is that you have a limit to how many times you can make your repeated query before you have spent your privacy budget and are cut off. The idea is that for a given budget level you are limited to fewer queries than you would need to learn enough to de-anonymize the data set.

If your signal processing professor was already taking that into account then I would be curious to know how that attack would work.

Post reply on HN