Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Does GPT-2 Know Your Phone Number?

bair.berkeley.edu

Post reply on HN
Print view View original

51–60 of 161 posts

Re: Does GPT-2 Know Your Phone Number?

#51

Post by jakear »

ben_w wrote: post #46
jakear wrote: post #36

Earlier quoted context omitted.

It’s my understanding that under our current scientific understanding of the mind, we cannot have “free will”. Where would it come from? All science knows is that neurons receive inputs from sensors, mixes them together (with some quantum randomness sprinkled in), and eventually actuates muscles to produce outputs, exactly as GPT. Thus “free will” must come from some sort of faith. Either it was given to us by “God”,…

It depends what you mean by “free will” — the phrase “free will” is as ill-defined as “common sense” and “consciousness”. I take the view that ∀ x ∈ , Has(x, humans) ≡ Has(x, AI)

I agree with your claim. I also believe many to see “free will”/consciousness/etc as that certain je ne sais quoi that separates the human experience from that of computers:

∀ x ∈ , Has(x, humans) ≡ ~Has(x, AI)

For these to both be true, becomes empty. Thus I argue we have none. Edit, or rather it’s meaningless to argue if we have any or not, as it cannot be defined.

Re: Does GPT-2 Know Your Phone Number?

#52

Post by aabhay »

colanderman wrote: post #28
newman8r wrote: post #3

Playing with the AI dungeon a while back (on the GPT-2 mode) I was presented with a tilapia recipe, titled "Kittencal's Broiled Tilapia" - it sounded bizarre so I decided to do a google search and I found that it was directly pulled from from https://www.recipezazz.com/recipe/broiled-parmesan-tilapia-7... - the user who posted it was 'Kittencal'

This is something that has repeatedly bothered me while playing with GPT-2. It retains too much "long-tail" knowledge that seems counterproductive for a generic language tool. I would think those one-off associations should not be present after training has "digested" them but there they are.

Hard to decide how much is too much.. the hard (and interesting) part of language is in the long tail

Re: Does GPT-2 Know Your Phone Number?

#53

Post by sp332 »

visarga wrote: post #44

I was expecting a bloom filter based solution to block verbatim reproduction of training data. They just need to hash the sensitive n-grams (hopefully, a small part of the whole dataset) and store one bit per hash. Alternatively, they could do something like GAN and have a 'discriminator' classify if a sample is natural or synthetic. Then, at inference, condition to be original. So, verbatim training data reproductio…

It's a language model. Asking it to do anything original is out of scope.

Re: Does GPT-2 Know Your Phone Number?

#54

Post by visarga »

taneq wrote: post #25
fouronnes3 wrote: post #4

Earlier quoted context omitted.

There is enough GPT discussion online now that the next language model will be trained on text that talks about itself. That's when it gets interesting.

Oooh, now that does raise some spooky time-delayed consciousness vibes.

GPT-3 knows about GPT-2 and can generate an article about itself.

Re: Does GPT-2 Know Your Phone Number?

#55

Post by visarga »

taneq wrote: post #26
dmm wrote: post #8

Earlier quoted context omitted.

So in terms of copyright, is GPT-2 a derived work of that recipe? Or generally are models derived works of their training data? It seems lots of people use training data from Flickr, like COCO, and then use the resulting model for commercial services.

It's as much a derived work as the user who posted the anecdote here. They were trained on the same data!

all of us are derived works

Re: Does GPT-2 Know Your Phone Number?

#56

Post by fastball »

px43 wrote: post #19

I've been saying this for years, but every post you make online, every unencrypted email, IM, text message, etc, will eventually end up getting sold off as training data for future machine learning projects. Every company stores this stuff for ages, and the value of candid conversation data just keeps increasing. Eventually these companies are either going to get hacked, get bought, or go bankrupt, and all the cleart…

End-to-end encryption is a possible solution to some of this, no?

Re: Does GPT-2 Know Your Phone Number?

#57

Post by schwede »

aboringusername wrote: post #50

I'm not even sure why people have phone numbers at all these days, the entire telephony system has been hijacked to hell (so much data collection from just owning a phone number). What you should be doing is replacing no less than yearly any numbers you have, also email addresses and anything within your control (changing physical address is much more difficult). You may even consider changing your legal name if that…

That sounds very impractical even if it’s a good solution.

Re: Does GPT-2 Know Your Phone Number?

#58

Post by visarga »

sp332 wrote: post #53
visarga wrote: post #44

I was expecting a bloom filter based solution to block verbatim reproduction of training data. They just need to hash the sensitive n-grams (hopefully, a small part of the whole dataset) and store one bit per hash. Alternatively, they could do something like GAN and have a 'discriminator' classify if a sample is natural or synthetic. Then, at inference, condition to be original. So, verbatim training data reproductio…

It's a language model. Asking it to do anything original is out of scope.

Almost all its outputs are original, based on ngram filtering. It can be original because at each token there is a stochastic step.

But if you mean by original to invent a whole new genre, or completely new esthetics, I agree, it's out of its scope. It is a great interpolator.

Re: Does GPT-2 Know Your Phone Number?

#59

Post by visarga »

grenoire wrote: post #2

Very rough outcome: "Moreover, if such a request were granted, must the model be retrained from scratch? The fact that models can memorize and misuse an individual’s personal information certainly makes the case for data deletion and retraining more compelling." Besides, how would one even know that their info was used in a training dataset, only if and when it's revealed in a generated excerpt?

They can try to prompt it. But I hope the GPT-3 authors have a hash list of ngrams in the training data to be able to avoid verbatim reproduction. I know they trained an offensive content detector model, they should also train a PII information detector to make sure it's being hashed.

Re: Does GPT-2 Know Your Phone Number?

#60

Post by visarga »

jakear wrote: post #38
lukeschlather wrote: post #24

Describing this as "memorizing" seems wrong. Humans often repeat things they heard earlier, and they will (honestly) swear up-and-down that the thing they're repeating is an original thought. If we succeed in making AGI that has human-equivalent intelligence we should expect this sort of behavior. The stuff about whether or not models should be destroyed if they contain copyrighted work gets kind of chilling if model…

We can probably get around this by just providing the AGI some sort of currency tokens they can exchange for virtual goods and services, then teach them that if they misuse copyrighted material they’ll be put to virtual court to plead their case and lose some amount of those virtual tokens if they fail to make a clear and convincing argument.

Or, add a bloom filter loaded with the training data to the model.
Post reply on HN