Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google’s DeepMind made ‘inexcusable’ errors handling UK health data, says report

theverge.com

Post reply on HN
Print view View original

61–70 of 72 posts

Re: Google’s DeepMind made ‘inexcusable’ errors handling UK health data, says report

#61

Post by dragonwriter »

aub3bhat wrote: post #36
Silhouette wrote: post #30

Earlier quoted context omitted.

From your comments here, it appears that you're doing PhD research dealing with large sets of healthcare data. It also appears that you are casually dismissive of concerns about the risks of that data being de-anonymised. Your best rebuttal seems to be some vague allusion that the parent poster, who was essentially correct, didn't know what they were talking about. (This is something of a digression anyway, because i…

Concerns about de-anonymization are NOT AT ALL relevant in cases where patient level data is shared. Since its common knowledge that this data is ripe for misuse and abuse. As a result government agencies have developed a set of legal requirements and contracts to be used when sharing such data. Talking about differential-privacy and de-identification-is-not-de-anonymization is meaningless in this context since all p…

> And regarding your concern about me, if anything I should be the one engaging in this ridiculous witch hunt against Deep Mind

I dunno; if your cavalier attitude toward privacy protections around personal health data reflect how your research was managed, there could be a very strong personal-interest reason for you to be waving around the "no big deal" flag you are waving about DeepMind.

Re: Google’s DeepMind made ‘inexcusable’ errors handling UK health data, says report

#62

Post by aub3bhat »

throwaway729 wrote: post #60
aub3bhat wrote: post #53

Earlier quoted context omitted.

Number of git commits!!! hahhahah. Chill dude, chill. Software changes the world.

> Number of git commits!!! Well, it's true. Differential privacy is a nice idea, but in this particular case, you're disparaging a style of research that -- to date -- has had a much greater impact on improving actual, real world privacy than all the fanciest query engines in the world. > Software changes the world. The most important thing to know about differential privacy is that when in comes to privacy, software…

LOL

I actually don't use differential privacy at all!

My software implements legal requirements as stated by the agency providing the data. Which are equivalent to a stronger version of K-Anonymity.

I am not at all against Policy Research, the issue is that this particular paper is a spectacularly BAD example of Policy Research. Policy research should not be driven by FUD around AI (e.g. the quote "did not constrain the company from using AI analytical techniques on the data") or some corporation.

Re: Google’s DeepMind made ‘inexcusable’ errors handling UK health data, says report

#63

Post by 3JPLW »

throwaway729 wrote: post #41
3JPLW wrote: post #28

Earlier quoted context omitted.

Indeed. The paper itself details seven "transgressions:" > 1) We do not know––and have no power to find out––what Google and DeepMind are really doing with NHS patient data, nor the extent of Royal Free’s meaningful control over what Google and DeepMind are doing; > 2) Any assurances about use of the dataset come from public relations statements, rather than independent oversight or legally binding documents; > 3) Th…

> Quite a few of these strike me as rather absurd In a "that can't possibly be true" sense? Well, yeah, that's kind of the point... 1-3 seem like the sorts of things that even the least privacy-sensitive person can agree are troublesome. If Google is willing to give anyone who signs a set of modest legal agreements carte blanc unaudited access to data stored on their servers, I'll begin to even remotely consider ente…

Amusingly, I find point #4 (which you skipped) completely reprehensible and unambiguously the worst offender. All these sorts of arrangements are done within a legal context. If they satisfied the legal requirements, then the other points lose their punch.

3. If you want to learn new insights, you — by definition — need to include data that a priori don't seem directly related. This point even notes that the data was technically and legally well-scoped.

5. Sounds like they were within the terms of the existing data privacy agreements given 6. Again, I don't know UK privacy laws.

6. My impression from the paper is that they're not only trying to manage AKI but also improve the detection of it. Ok, sure, they're not going to improve detection in deceased or transferred patients. Those probably should have been minimized.

Re: Google’s DeepMind made ‘inexcusable’ errors handling UK health data, says report

#64

Post by Gatsky »

"In July 2015, clinicians from British public hospitals within the Royal Free London NHS Foundation Trust approached Google DeepMind Technologies Limited, an artificial intelligence company with no experience in providing healthcare services, about developing software using patient data from the Trust."

Actually, the institution which collects and stores the data handed it over without due process. The article keeps trying to blame DeepMind, I guess criticising the NHS is a little stale. I think this is in a journal because it is too long for an op-ed but not substantial enough for a long form article.

Re: Google’s DeepMind made ‘inexcusable’ errors handling UK health data, says report

#65

Post by aub3bhat »

dragonwriter wrote: post #61
aub3bhat wrote: post #36

Earlier quoted context omitted.

Concerns about de-anonymization are NOT AT ALL relevant in cases where patient level data is shared. Since its common knowledge that this data is ripe for misuse and abuse. As a result government agencies have developed a set of legal requirements and contracts to be used when sharing such data. Talking about differential-privacy and de-identification-is-not-de-anonymization is meaningless in this context since all p…

> And regarding your concern about me, if anything I should be the one engaging in this ridiculous witch hunt against Deep Mind I dunno; if your cavalier attitude toward privacy protections around personal health data reflect how your research was managed, there could be a very strong personal-interest reason for you to be waving around the "no big deal" flag you are waving about DeepMind.

Or maybe maybe I have deeper knowledge of intricacies & current state of healthcare data.

Even the expert from Wellcome Trust thinks its “overly-critical”.

Re: Google’s DeepMind made ‘inexcusable’ errors handling UK health data, says report

#66

Post by throwaway729 »

aub3bhat wrote: post #62
throwaway729 wrote: post #60

Earlier quoted context omitted.

> Number of git commits!!! Well, it's true. Differential privacy is a nice idea, but in this particular case, you're disparaging a style of research that -- to date -- has had a much greater impact on improving actual, real world privacy than all the fanciest query engines in the world. > Software changes the world. The most important thing to know about differential privacy is that when in comes to privacy, software…

LOL I actually don't use differential privacy at all! My software implements legal requirements as stated by the agency providing the data. Which are equivalent to a stronger version of K-Anonymity. I am not at all against Policy Research, the issue is that this particular paper is a spectacularly BAD example of Policy Research. Policy research should not be driven by FUD around AI (e.g. the quote "did not constrain…

> I actually don't use differential privacy at all!

I was just going by your own description on the product website. Anyways, kind of irrelevant since your "software implements legal requirements as stated by the agency providing the data". That's basically my whole point.

> ...is a spectacularly BAD example of Policy Research

I tend to judge research by its merit and its impact.

Merit is discussed at length elsewhere, and IMO you're flat out wrong about the normalcy of this particular agreement. But we can leave that to other threads.

In this thread, you're disparaging the research based upon its impact ("real debate", "real systems"), when in fact it has had a direct impact.

Re: Google’s DeepMind made ‘inexcusable’ errors handling UK health data, says report

#67

Post by throwaway729 »

3JPLW wrote: post #63
throwaway729 wrote: post #41

Earlier quoted context omitted.

> Quite a few of these strike me as rather absurd In a "that can't possibly be true" sense? Well, yeah, that's kind of the point... 1-3 seem like the sorts of things that even the least privacy-sensitive person can agree are troublesome. If Google is willing to give anyone who signs a set of modest legal agreements carte blanc unaudited access to data stored on their servers, I'll begin to even remotely consider ente…

Amusingly, I find point #4 (which you skipped) completely reprehensible and unambiguously the worst offender. All these sorts of arrangements are done within a legal context. If they satisfied the legal requirements, then the other points lose their punch. 3. If you want to learn new insights, you — by definition — need to include data that a priori don't seem directly related. This point even notes that the data was…

As you noted, the fundamental problem underlying 3,5,6 is that they're trying to detect AKI. That requires everyone's data.

I'm not opposed to that in general, but there really ought to be 1) an opt-in or at least a well-advertised opt-out mechanism; and 2) an independent audit of how data is used.

FWIW I think this was a healthy push-back against "just trust us" and hope the result is a cleaner template for future similar projects.

Re: Google’s DeepMind made ‘inexcusable’ errors handling UK health data, says report

#68

Post by halhodson »

aub3bhat wrote: post #36
Silhouette wrote: post #30

Earlier quoted context omitted.

From your comments here, it appears that you're doing PhD research dealing with large sets of healthcare data. It also appears that you are casually dismissive of concerns about the risks of that data being de-anonymised. Your best rebuttal seems to be some vague allusion that the parent poster, who was essentially correct, didn't know what they were talking about. (This is something of a digression anyway, because i…

Concerns about de-anonymization are NOT AT ALL relevant in cases where patient level data is shared. Since its common knowledge that this data is ripe for misuse and abuse. As a result government agencies have developed a set of legal requirements and contracts to be used when sharing such data. Talking about differential-privacy and de-identification-is-not-de-anonymization is meaningless in this context since all p…

"But unlike the authors I want real debate"

Hello. I am one of the authors. We are on the record saying that we want a real debate.

See: http://uk.businessinsider.com/deepmind-royal-free-london-nhs... https://www.theregister.co.uk/2017/03/16/googles_deepmind_an... http://www.bbc.co.uk/news/technology-39301901

Re: Google’s DeepMind made ‘inexcusable’ errors handling UK health data, says report

#69

Post by killjoywashere »

throwaway729 wrote: post #24
aub3bhat wrote: post #22

Earlier quoted context omitted.

Again take a look at CMS Qualified Entity program which provides identifiable data. In fact most State government health agencies have a special board which routinely meets to discuss identifiable data requests. Finally their is nothing wrong in writing a paper about a politically charged topic, but completely dismissing existing standards to paint a picture of doom and gloom is ridiculous. E.g. just look at the refe…

> Again take a look at CMS Qualified Entity program which provides identifiable data From what I understand, this program has exactly the sort of contractual legal limits on use that the authors are asking for: "Qualified entities may use the information obtained under section 1874(e) of the Act for the sole purpose of evaluating the performance of providers of services and suppliers , and to generate specified publi…

> If NHS hands health data over to private companies, there should be very strong legal protections governing the use of that data.

NHS hands health data over to lots of private companies. Like any corporate entity, NHS would be expected to perform better when it outsources non-core work. The core work of the hospitals is patient care. The core work of the health care system is coordination, neither of which sounds like "algorithm development".

Keep in mind there is a vast standing body of law and precedent around these issues in the US, Britain, and anywhere else that medical research is undertaken.

Re: Google’s DeepMind made ‘inexcusable’ errors handling UK health data, says report

#70

Post by KCFforecast »

desas wrote: post #55
KCFforecast wrote: post #51

Under Spain protection of personal data law, any file with personal information must be accessible for the person, and the person has the right to know, modify and delete that file, and to deny the access to that information for any purpose.

There must be qualifiers, you can't tell your bank to forget the money you owe it.

You are right, more information here: http://uk.practicallaw.com/1-520-8264
Post reply on HN