How is this related to and different from differential privacy?
Artificial data give the same results as real data without compromising privacy
11–20 of 48 posts
Re: Artificial data give the same results as real data without compromising privacy
#12Does someone have a link to the preprint / arxiv? The link in the story is a 404 (I presume that the paper just hasn't been posted yet or something?)
I've found these documents:
- https://dspace.mit.edu/handle/1721.1/109616#files-area
- https://pdfs.semanticscholar.org/64ad/643e8084486ca7d3312ed4...
Re: Artificial data give the same results as real data without compromising privacy
#13I haven't read the original paper (yet), but something doesn't sit right with the work, if the way it is portrayed is indeed faithful to it and I'm not missing something important.
- It looks like the work of the data scientists will be limited to the extent of the modeling already done by recursive conditional parameter aggregation. (edit: So why not just ship that model and adapt it instead of using it to generate data?)
- Its "validation" appears to be doubly proxied - i.e. the normal performance measures we use are themselves a proxy, and now we're comparing those against these performance measures derived from models built out of the data generated by these models. I'm not inclined to trust a validation that is so removed.
Any one who can explain this well?
Re: Artificial data give the same results as real data without compromising privacy
#14I'm highly dubious of the ability for synthetic data to model accurately datasets without introducing unexpected bias, esp. to account for causality.
If you dig through the original paper, the conclusion is on the line with that:
“For 7 out of 15 comparisons, we found no significant difference between the accuracy of features developed on the control dataset vs. those developed on some version of the synthesized data; that is, the result of the test was False.”
So, on the tests they developed, the proposed method doesn't work 8 times out of 15…
Re: Artificial data give the same results as real data without compromising privacy
#15I haven't read the original paper (yet), but something doesn't sit right with the work, if the way it is portrayed is indeed faithful to it and I'm not missing something important. - It looks like the work of the data scientists will be limited to the extent of the modeling already done by recursive conditional parameter aggregation. (edit: So why not just ship that model and adapt it instead of using it to generate…
I think they just invented the political representative in modelling
Re: Artificial data give the same results as real data without compromising privacy
#16They use real data to create artificial data. So, real data is still more useful.
The idea is to sidestep the need to access private information in order for researchers to do their work. So in this case, the artificial data is more useful, since the real data is inaccessible.
But the artificial data must come from somewhere? It can be modeled from real data in order to take into account outliers and to avoid cognitive biases in generation, but then there's still an initial reliance on the real data.
Re: Artificial data give the same results as real data without compromising privacy
#17The abstract claims there was no difference only 70% of the time. So 30% of the time there was a difference. Unsurprisingly it greatly limits the kind of data analysis that was allowed, which greatly reduces the applicability even if you believe it. I'm pretty dubious of this work anyway.
Re: Artificial data give the same results as real data without compromising privacy
#18How is this related to and different from differential privacy ?
Differential privacy is a formal guarantee of an algorithm. Roughly, given algorithm A that takes input database X, we say A is differentially private if m, for any X' differing in at most one row from X, the output distributions of A(X) and A(X') are similar. So to say an algorithm is differentially private you need to prove a claim like this.
It's hard to compare to this paper, because this paper's privacy claims appear to be heuristic, not formal. This isn't necessarily bad, since existing approaches for constructing synthetic data in a differentially private way is still not very practical. But heuristics do necessarily lack provable privacy guarantees, so there's no proof that something very bad privacy-wise can't happen with sufficiently clever processing of the synthetic data.
Re: Artificial data give the same results as real data without compromising privacy
#19I'm highly dubious of the ability for synthetic data to model accurately datasets without introducing unexpected bias, esp. to account for causality. If you dig through the original paper, the conclusion is on the line with that: “For 7 out of 15 comparisons, we found no significant difference between the accuracy of features developed on the control dataset vs. those developed on some version of the synthesized data…
I couldn't read the paper (seemed to be missing), but has anyone else noticed that MIT seems to have big problems with open science?
I mean I have formed an association specifically with the MIT brand now, so this type of work coming out of there doesn't surprise me. I couldn't tell you exactly what has lead to this association though.
Re: Artificial data give the same results as real data without compromising privacy
#20I'm highly dubious of the ability for synthetic data to model accurately datasets without introducing unexpected bias, esp. to account for causality. If you dig through the original paper, the conclusion is on the line with that: “For 7 out of 15 comparisons, we found no significant difference between the accuracy of features developed on the control dataset vs. those developed on some version of the synthesized data…
Agreed, seems suspect. If they are really able to learn the population-level distribution then why even bother generating fake data. Just release that instead.