Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Does GPT-2 Know Your Phone Number?

bair.berkeley.edu

Post reply on HN
Print view View original

121–130 of 161 posts

Re: Does GPT-2 Know Your Phone Number?

#121

Post by nightcracker »

agumonkey wrote: post #99
courtf wrote: post #96

Earlier quoted context omitted.

It's one of the biggest lies in tech right now, that all this user data is some untapped gold mine, as if text on the internet is the most important measure of human behavior. It's all going to end up rotting on backups while the marketing department spins yarns about the amazing AI that's going to spring forth any day now from a pile of tweets. It's like trying to build a dog by hanging around the park collecting tu…

is the data market a bubble ?

Answer this for yourself: what percentage of ads that you see is relevant? There's your answer.

Re: Does GPT-2 Know Your Phone Number?

#122

Post by rav »

breatheoften wrote: post #119

The article makes the claim that models which show similar train and test losses demonstrate minimal overfitting -- and are therefore less likely generally to exhibit a lot of text memorization. I wonder the degree to which this inference is true in practice with respect to information like phone numbers... How exactly are the train and test sets formed in a de-correlated-with-respect-to-memorization-of-phone-numbers…

That raises a more general issue: Text scraped from the internet may be duplicated to multiple sites with few changes. How does the team behind GPT-n ensure that the corpus is maximally deduplicated before splitting into train and test corpora?

Re: Does GPT-2 Know Your Phone Number?

#123

Post by agumonkey »

nightcracker wrote: post #121
agumonkey wrote: post #99

Earlier quoted context omitted.

is the data market a bubble ?

Answer this for yourself: what percentage of ads that you see is relevant? There's your answer.

I'm so mentally blind to ads I could never consider my opinion worth a cent on this :D

Re: Does GPT-2 Know Your Phone Number?

#124

Post by _flux »

theptip wrote: post #86

> There is a legal grey area as to how these regulations should apply to machine learning models. For example, can users ask to have their data removed from a model’s training data? Moreover, if such a request were granted, must the model be retrained from scratch? The fact that models can memorize and misuse an individual’s personal information certainly makes the case for data deletion and retraining more compellin…

Amusing alternative: for model-as-a-service kind of things there could be blacklist or another neural network to handle the banned cases.

"I'm afraid I cannot tell you that, Dave"

Re: Does GPT-2 Know Your Phone Number?

#125

Post by gugagore »

geofft wrote: post #107
fouronnes3 wrote: post #4

Earlier quoted context omitted.

There is enough GPT discussion online now that the next language model will be trained on text that talks about itself. That's when it gets interesting.

This seems like what xkcd called "citogenesis" - something inaccurate gets published on Wikipedia, a traditional "reliable source" (newspaper, published nonfiction book, etc.) retells that bit of information without a source, Wikipedia cites the source, that information is now firmly established as truth for the rest of humanity. I worry about this with Google's language translation models, too. It's entirely possibl…

This is an issue solvable with better provenance / data lineage, which has come up in recent HN discussions.

Re: Does GPT-2 Know Your Phone Number?

#126

Post by anfilt »

6gvONxR4sf7o wrote: post #112
theptip wrote: post #86

> There is a legal grey area as to how these regulations should apply to machine learning models. For example, can users ask to have their data removed from a model’s training data? Moreover, if such a request were granted, must the model be retrained from scratch? The fact that models can memorize and misuse an individual’s personal information certainly makes the case for data deletion and retraining more compellin…

I think there are also going to be new difficult problems arising training on copyrighted data. Can I do a DMCA if your model contains/regurgitates too much of my book verbatim in a way that doesn’t meet fair use?

The depends on the country, but as for the US since you mention DMCA, ML using copyrighted work is probably legal. There just is not really much case law. Talk to a lawyer though, but depending on how risk averse you want to be there are definitely ways to avoid issues, but ensuring you can use such copyrighted works ect... Although, making a very large curated data set can be expensive.

Re: Does GPT-2 Know Your Phone Number?

#127

Post by _flux »

hyperpape wrote: post #42
jrockway wrote: post #31

Earlier quoted context omitted.

Isn't that how humans work? Sometimes I wonder if I have free will and sentience, or if I am just stringing words together based on some probability function. I've no doubt taken into account copyrighted works and personal information when training my built-in neural network. The examples the article gives like "misremembering" the murder as the murder victim sounds like something a person would do. Knowing verbatim…

Of course it isn't how the human mind works. You can ask a person "did you come up with that sentence, or is it something you read?" And they can answer that question. The answer isn't perfectly reliable, but it isn't completely unreliable either. You can't do that with GPT-2/GPT-3. Big picture: the human mind is a machine of sorts. It's not the same kind of machine as any form of machine learning we've developed so…

In this case you could do it with GPT-x as well, if you have access to the learning material (so the OpenAI could do it when providing the model as a service): grep the learning data after-the-fact.

No idea if it would be feasible to integrate this information directly into the model. With my limited understanding of neural networks, this seems difficult.

Re: Does GPT-2 Know Your Phone Number?

#128

Post by PurpleFoxy »

dmm wrote: post #8
newman8r wrote: post #3

Playing with the AI dungeon a while back (on the GPT-2 mode) I was presented with a tilapia recipe, titled "Kittencal's Broiled Tilapia" - it sounded bizarre so I decided to do a google search and I found that it was directly pulled from from https://www.recipezazz.com/recipe/broiled-parmesan-tilapia-7... - the user who posted it was 'Kittencal'

So in terms of copyright, is GPT-2 a derived work of that recipe? Or generally are models derived works of their training data? It seems lots of people use training data from Flickr, like COCO, and then use the resulting model for commercial services.

Probably depends how similar your output is to something copyrighted. I doubt the simple act of learning from previous art is disallowed or else literally everyone would be blocked because no one has a truely original thought.

Re: Does GPT-2 Know Your Phone Number?

#129

Post by joe_the_user »

visarga wrote: post #54
taneq wrote: post #25

Earlier quoted context omitted.

Oooh, now that does raise some spooky time-delayed consciousness vibes.

GPT-3 knows about GPT-2 and can generate an article about itself.

I've noticed many GPT-3 article tending to wander into being article about AI, fear of AI and GPT-3-itself without any prompting. Many people's speech tends to become discussions about themselves, so that might demonstrate "humanity" in an odd sense.

Re: Does GPT-2 Know Your Phone Number?

#130

Post by geofft »

gugagore wrote: post #125
geofft wrote: post #107

Earlier quoted context omitted.

This seems like what xkcd called "citogenesis" - something inaccurate gets published on Wikipedia, a traditional "reliable source" (newspaper, published nonfiction book, etc.) retells that bit of information without a source, Wikipedia cites the source, that information is now firmly established as truth for the rest of humanity. I worry about this with Google's language translation models, too. It's entirely possibl…

This is an issue solvable with better provenance / data lineage, which has come up in recent HN discussions.

How do you enforce that other people provide provenance on text?

Taking the translation example - how do you enforce that users of Google Translate keep provenance in their translated text that remains with the text?

Post reply on HN