Does GPT-2 Know Your Phone Number? • Hacker News Board

Re: Does GPT-2 Know Your Phone Number?

#21

Post by wpietri » Sun, Dec 27, 2020, 3:14 PM UTC

tiborsaas wrote: post #10

> When Peter put his contact information online, it had an intended context of use. Unfortunately, applications built on top of GPT-2 are unaware of this context, and might thus unintentionally share Peter’s data in ways he did not intend. That's expected when you publish anything online, you lose control over the data.

In the same sense that when you leave the house you lose control over what happens to you. But if something bad happens to a person., the right solution is not to say, "Well, you left the house, so tough luck."

Re: Does GPT-2 Know Your Phone Number?

#22

Post by golergka » Sun, Dec 27, 2020, 3:20 PM UTC

I only took a beginner course in ML over 5 years ago, so this probably is a stupid question, but does this mean, the trained GPT-2 model encodes the source text into it's parameters somehow? Is this resilient — will it still remember it if we randomize the weights just a little tiny bit? Will it remember it if we clear a small portion of parameters?

Does the human memory work the same way?

Re: Does GPT-2 Know Your Phone Number?

#23

Post by est31 » Sun, Dec 27, 2020, 3:24 PM UTC

lixtra wrote: post #20

est31 wrote: post #15

Earlier quoted context omitted.

IANAL, but from my reading Article 3 of the EU copyright directive explicitly protects data mining for scientific research, and Article 4 enables everyone to use content for data mining whose rights have not been explicitly and machine readably reserved.

Edit: est31 and parent seem right and I stand corrected. Wrong: I don’t think you‘re paraphrasing article 4 correctly. Both are about science/teaching exceptions. Not general exceptions. https://www.consilium.europa.eu/media/35373/st09134-en18.pdf pp 43

That's the May 2018 version. The final/official April 17 2019 version [0] contains a different Article 4 which is the one I referred to: https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CEL...

[0]: apparently there were some mistakes in translations which required changes in some language versions, but the english version was final IIRC https://eur-lex.europa.eu/eli/dir/2019/790?locale=de

Re: Does GPT-2 Know Your Phone Number?

#24

Post by lukeschlather » Sun, Dec 27, 2020, 3:29 PM UTC

Describing this as "memorizing" seems wrong. Humans often repeat things they heard earlier, and they will (honestly) swear up-and-down that the thing they're repeating is an original thought. If we succeed in making AGI that has human-equivalent intelligence we should expect this sort of behavior.

The stuff about whether or not models should be destroyed if they contain copyrighted work gets kind of chilling if models actually achieve sentience someday. If I could make a faithful copy of my consciousness, my consciousness can reliably reproduce numerous copyrighted works. As can anyone.

Of course, most of those I deliberately memorized. I think it's a crucial thing here that the model is not actually memorizing anything - if we assume for a moment that the model is a consciousness, these are all half-remembered snippets leaking into casual conversation. And I think it's most likely any truly conscious entity is going to do that sort of thing from time to time.

Re: Does GPT-2 Know Your Phone Number?

#25

Post by taneq » Sun, Dec 27, 2020, 3:32 PM UTC

fouronnes3 wrote: post #4

newman8r wrote: post #3

Playing with the AI dungeon a while back (on the GPT-2 mode) I was presented with a tilapia recipe, titled "Kittencal's Broiled Tilapia" - it sounded bizarre so I decided to do a google search and I found that it was directly pulled from from https://www.recipezazz.com/recipe/broiled-parmesan-tilapia-7... - the user who posted it was 'Kittencal'

There is enough GPT discussion online now that the next language model will be trained on text that talks about itself. That's when it gets interesting.

Oooh, now that does raise some spooky time-delayed consciousness vibes.

Re: Does GPT-2 Know Your Phone Number?

#26

Post by taneq » Sun, Dec 27, 2020, 3:33 PM UTC

dmm wrote: post #8

newman8r wrote: post #3

Playing with the AI dungeon a while back (on the GPT-2 mode) I was presented with a tilapia recipe, titled "Kittencal's Broiled Tilapia" - it sounded bizarre so I decided to do a google search and I found that it was directly pulled from from https://www.recipezazz.com/recipe/broiled-parmesan-tilapia-7... - the user who posted it was 'Kittencal'

So in terms of copyright, is GPT-2 a derived work of that recipe? Or generally are models derived works of their training data? It seems lots of people use training data from Flickr, like COCO, and then use the resulting model for commercial services.

It's as much a derived work as the user who posted the anecdote here. They were trained on the same data!

Re: Does GPT-2 Know Your Phone Number?

#27

Post by kevinmchugh » Sun, Dec 27, 2020, 3:34 PM UTC

dmm wrote: post #8

newman8r wrote: post #3

Playing with the AI dungeon a while back (on the GPT-2 mode) I was presented with a tilapia recipe, titled "Kittencal's Broiled Tilapia" - it sounded bizarre so I decided to do a google search and I found that it was directly pulled from from https://www.recipezazz.com/recipe/broiled-parmesan-tilapia-7... - the user who posted it was 'Kittencal'

So in terms of copyright, is GPT-2 a derived work of that recipe? Or generally are models derived works of their training data? It seems lots of people use training data from Flickr, like COCO, and then use the resulting model for commercial services.

Recipes aren't typically copyrightable

In general, I think the question is unsettled

Re: Does GPT-2 Know Your Phone Number?

#28

Post by colanderman » Sun, Dec 27, 2020, 3:34 PM UTC

newman8r wrote: post #3

Playing with the AI dungeon a while back (on the GPT-2 mode) I was presented with a tilapia recipe, titled "Kittencal's Broiled Tilapia" - it sounded bizarre so I decided to do a google search and I found that it was directly pulled from from https://www.recipezazz.com/recipe/broiled-parmesan-tilapia-7... - the user who posted it was 'Kittencal'

This is something that has repeatedly bothered me while playing with GPT-2. It retains too much "long-tail" knowledge that seems counterproductive for a generic language tool. I would think those one-off associations should not be present after training has "digested" them but there they are.

Re: Does GPT-2 Know Your Phone Number?

#29

Post by tiborsaas » Sun, Dec 27, 2020, 3:36 PM UTC

wpietri wrote: post #21

tiborsaas wrote: post #10

> When Peter put his contact information online, it had an intended context of use. Unfortunately, applications built on top of GPT-2 are unaware of this context, and might thus unintentionally share Peter’s data in ways he did not intend. That's expected when you publish anything online, you lose control over the data.

In the same sense that when you leave the house you lose control over what happens to you. But if something bad happens to a person., the right solution is not to say, "Well, you left the house, so tough luck."

You'd be right, if my copy left my house. It's the physical vs. digital theft argument.

If I leave the house I take on some risk, but most of the time I'm in control.

Re: Does GPT-2 Know Your Phone Number?

#30

Post by barrkel » Sun, Dec 27, 2020, 3:37 PM UTC

tiborsaas wrote: post #10

> When Peter put his contact information online, it had an intended context of use. Unfortunately, applications built on top of GPT-2 are unaware of this context, and might thus unintentionally share Peter’s data in ways he did not intend. That's expected when you publish anything online, you lose control over the data.

Ah yes. And if your name is included in an accusation of murder, well, I guess you shouldn't have published your name.

This is infantile reverse justification from an end. GPT style corpuses are clearly problematic without a lot more smarts on figuring out what's appropriate and what's not.