Does GPT-2 Know Your Phone Number? • Hacker News Board

Re: Does GPT-2 Know Your Phone Number?

#151

Post by why_only_15 » Mon, Dec 28, 2020, 6:05 AM UTC

est31 wrote: post #149

why_only_15 wrote: post #147

Earlier quoted context omitted.

This isn't about averaging over all your submissions, it's about averaging over the entire population (in this case, of iphone users, ~1,000,000,000). Data about the entire population, e.g. how many people click this button, or how many results do most people get from this page, or what kind of things do people take pictures of, i think are safe.

If you average over the entire population, it's privacy preserving indeed. But this averaging happens on the side of the server, not the client, so I can't really verify whether the server stores away the deltas of interesting/all users or it averages really all user deltas. I know apple has some means to anonymize users by using random changing IDs instead of the apple ID, but they can correlate it to an apple ID vi…

Ultimately, what can you really verify as a user? Apple is very different from bitcoin, or IPFS or matrix or the like, in that they have an existing reputation. Instead of trying to provide perfect guarantees, they make a real attempt at preserving privacy, and you just have to trust them on that. I personally trust them after having attempted experiments stymied by the privacy team, but i think in general there's reasonable evidence that apple really does try to be privacy-preserving.

For your example, sure it's possible for the server to do something you don't know. But it's the same people on the server doing the aggregation as on the client doing the obfuscation. If they really wanted they could easily e.g. set the seed for the noise to be based on a key which they have on the server, which would be very difficult to detect. I think you just have to trust that apple wouldn't let that happen.

Re: Does GPT-2 Know Your Phone Number?

#152

Post by andreareina » Mon, Dec 28, 2020, 6:29 AM UTC

minimaxir wrote: post #69

fouronnes3 wrote: post #4

Earlier quoted context omitted.

There is enough GPT discussion online now that the next language model will be trained on text that talks about itself. That's when it gets interesting.

I ran an experiment through GPT-3 with just that frame: https://github.com/minimaxir/gpt-3-experiments/tree/master/e...

What does the x_y signify? And what's going on with 0_0?? From my (very limited) understanding a fixed point/cycle like that should be vanishingly unlikely.

Re: Does GPT-2 Know Your Phone Number?

#153

Post by minimaxir » Mon, Dec 28, 2020, 6:35 AM UTC

andreareina wrote: post #152

minimaxir wrote: post #69

Earlier quoted context omitted.

I ran an experiment through GPT-3 with just that frame: https://github.com/minimaxir/gpt-3-experiments/tree/master/e...

What does the x_y signify? And what's going on with 0_0?? From my (very limited) understanding a fixed point/cycle like that should be vanishingly unlikely.

Those are the temperatures (0.0, 0.7, etc.) that control the craziness. 0.0 means the model makes the optimal guess for each token.

Unfortunately, both GPT-2 and GPT-3 have a tendency to enter loops. But it's not always bad: https://github.com/minimaxir/gpt-3-experiments/blob/master/e...

Re: Does GPT-2 Know Your Phone Number?

#154

Post by Ansil849 » Mon, Dec 28, 2020, 7:17 AM UTC

What is the point of the partial redaction in this blog post (and corresponding journal article) if with a simple web search you can find the unredacted PII of the individual given in the example?

Re: Does GPT-2 Know Your Phone Number?

#155

Post by [deleted] » Mon, Dec 28, 2020, 8:44 AM UTC

1vuio0pswjnm7 wrote: post #117

Zenst wrote: post #61

Friend asked me if Google knew his phone number and details, I said they do at a quantum level. If you don't look they may or may not have your details, yet if you look - you are giving the details to search for and then they would have them if they did not.

Like websites offering to search for a user's details in a data breach, or a padded hash of those details, in order to inform the user if their details have been leaked. https://en.wikipedia.org/wiki/Have_I_Been_Pwned%3A

[deleted]

Re: Does GPT-2 Know Your Phone Number?

#156

Post by visarga » Mon, Dec 28, 2020, 10:38 AM UTC

jakear wrote: post #62

visarga wrote: post #60

Earlier quoted context omitted.

Or, add a bloom filter loaded with the training data to the model.

Good way to get results that are almost but not quite verbatim copied, which are still restricted under copyright law.

Small steps

Re: Does GPT-2 Know Your Phone Number?

#157

Post by MauranKilom » Mon, Dec 28, 2020, 11:28 AM UTC

visarga wrote: post #54

taneq wrote: post #25

Earlier quoted context omitted.

Oooh, now that does raise some spooky time-delayed consciousness vibes.

GPT-3 knows about GPT-2 and can generate an article about itself.

...but GPT-3 will only ever describe itself by explaining the features and capabilities of GPT-2. That's usually a very easy giveaway.

Re: Does GPT-2 Know Your Phone Number?

#158

Post by barrkel » Mon, Dec 28, 2020, 3:29 PM UTC

tiborsaas wrote: post #37

barrkel wrote: post #30

Earlier quoted context omitted.

Ah yes. And if your name is included in an accusation of murder, well, I guess you shouldn't have published your name. This is infantile reverse justification from an end. GPT style corpuses are clearly problematic without a lot more smarts on figuring out what's appropriate and what's not.

Anyone can publish any nonsense online since the internet exists, how is this any different? Responsibility is on those who publish content online, not the tools.

If GPT-style tools are powering chat bots or augmenting content generation then it matters, because they may not be supervised. The tools don't have common sense, but they are powerful enough to be automated.

Re: Does GPT-2 Know Your Phone Number?

#159

Post by andreareina » Mon, Dec 28, 2020, 5:51 PM UTC

minimaxir wrote: post #153

andreareina wrote: post #152

Earlier quoted context omitted.

What does the x_y signify? And what's going on with 0_0?? From my (very limited) understanding a fixed point/cycle like that should be vanishingly unlikely.

Those are the temperatures (0.0, 0.7, etc.) that control the craziness. 0.0 means the model makes the optimal guess for each token. Unfortunately, both GPT-2 and GPT-3 have a tendency to enter loops. But it's not always bad: https://github.com/minimaxir/gpt-3-experiments/blob/master/e...

Thanks, didn't twig onto the fact that you linked a subtree of the whole repo. Weird that even with the nonzero temp the AskReddit prompt went a bit loopy.

> https://github.com/minimaxir/gpt-3-experiments/blob/master/e...

Oh my goodness that is absurd in the most delightful way. Thanks for sharing that.

Re: Does GPT-2 Know Your Phone Number?

#160

Post by taneq » Tue, Dec 29, 2020, 2:44 AM UTC

MauranKilom wrote: post #157

visarga wrote: post #54

Earlier quoted context omitted.

GPT-3 knows about GPT-2 and can generate an article about itself.

...but GPT-3 will only ever describe itself by explaining the features and capabilities of GPT-2. That's usually a very easy giveaway.

Wait, is GPT-3 the trained network or is it the architecture and methodology behind it? If they re-run the same code on a fresh web crawl would that then be called GPT-4? I'd have thought you could update GPT-3's... knowledge? Understanding? Whatever the hell it is? by doing this semi-regularly.