Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Does GPT-2 Know Your Phone Number?

bair.berkeley.edu

Post reply on HN
Print view View original

31–40 of 161 posts

Re: Does GPT-2 Know Your Phone Number?

#31

Post by jrockway »

px43 wrote: post #19

I've been saying this for years, but every post you make online, every unencrypted email, IM, text message, etc, will eventually end up getting sold off as training data for future machine learning projects. Every company stores this stuff for ages, and the value of candid conversation data just keeps increasing. Eventually these companies are either going to get hacked, get bought, or go bankrupt, and all the cleart…

Isn't that how humans work? Sometimes I wonder if I have free will and sentience, or if I am just stringing words together based on some probability function.

I've no doubt taken into account copyrighted works and personal information when training my built-in neural network. The examples the article gives like "misremembering" the murder as the murder victim sounds like something a person would do. Knowing verbatim contact information of some random person is also possible. All in all, GPT-2 sounds a lot like us.

Re: Does GPT-2 Know Your Phone Number?

#32

Post by tiborsaas »

refactor_master wrote: post #18
tiborsaas wrote: post #10

> When Peter put his contact information online, it had an intended context of use. Unfortunately, applications built on top of GPT-2 are unaware of this context, and might thus unintentionally share Peter’s data in ways he did not intend. That's expected when you publish anything online, you lose control over the data.

Still doesn’t justify what could be perceived as borderline defamation.

Who is defaming who? One person using GPT to generate text that puts X into a bad light? Then the person using GPT is responsible they could just use string replace %victimname%.

Or GPT accidentally defaming someone? That assumes bad intent which is a bit problematic with a language model.

With made up contexts like this in the article it's easy to cry wolf.

Re: Does GPT-2 Know Your Phone Number?

#33

Post by Filligree »

commandpaul wrote: post #5

I've been playing with a writing tool shortlyread.com which purpotedly uses the GPT3 API. I had a similar experience in its responses to a lot of my prompts containing text verbatim from many sources and sometimes even going on to output personally identifying about persons related to the original text.

Took a look at that.

It's better at being a text editor, in that you can click the text and edit it! (Sweet relief.)

But it's missing the world-info feature from AI Dungeon, and without that I don't think it's practical to write anything long-form.

Re: Does GPT-2 Know Your Phone Number?

#34

Post by jakear »

golergka wrote: post #22

I only took a beginner course in ML over 5 years ago, so this probably is a stupid question, but does this mean, the trained GPT-2 model encodes the source text into it's parameters somehow? Is this resilient — will it still remember it if we randomize the weights just a little tiny bit? Will it remember it if we clear a small portion of parameters? Does the human memory work the same way?

If you can recall text verbatim, the text has been encoded into your neurons’ connections. Tautologically.

Re: Does GPT-2 Know Your Phone Number?

#36

Post by jakear »

jrockway wrote: post #31
px43 wrote: post #19

I've been saying this for years, but every post you make online, every unencrypted email, IM, text message, etc, will eventually end up getting sold off as training data for future machine learning projects. Every company stores this stuff for ages, and the value of candid conversation data just keeps increasing. Eventually these companies are either going to get hacked, get bought, or go bankrupt, and all the cleart…

Isn't that how humans work? Sometimes I wonder if I have free will and sentience, or if I am just stringing words together based on some probability function. I've no doubt taken into account copyrighted works and personal information when training my built-in neural network. The examples the article gives like "misremembering" the murder as the murder victim sounds like something a person would do. Knowing verbatim…

It’s my understanding that under our current scientific understanding of the mind, we cannot have “free will”. Where would it come from? All science knows is that neurons receive inputs from sensors, mixes them together (with some quantum randomness sprinkled in), and eventually actuates muscles to produce outputs, exactly as GPT.

Thus “free will” must come from some sort of faith. Either it was given to us by “God”, or we’re all part of some simulation and it exists on a level we don’t know about, or something else entirely too advanced for us to understand.

Re: Does GPT-2 Know Your Phone Number?

#37

Post by tiborsaas »

barrkel wrote: post #30
tiborsaas wrote: post #10

> When Peter put his contact information online, it had an intended context of use. Unfortunately, applications built on top of GPT-2 are unaware of this context, and might thus unintentionally share Peter’s data in ways he did not intend. That's expected when you publish anything online, you lose control over the data.

Ah yes. And if your name is included in an accusation of murder, well, I guess you shouldn't have published your name. This is infantile reverse justification from an end. GPT style corpuses are clearly problematic without a lot more smarts on figuring out what's appropriate and what's not.

Anyone can publish any nonsense online since the internet exists, how is this any different?

Responsibility is on those who publish content online, not the tools.

Re: Does GPT-2 Know Your Phone Number?

#38

Post by jakear »

lukeschlather wrote: post #24

Describing this as "memorizing" seems wrong. Humans often repeat things they heard earlier, and they will (honestly) swear up-and-down that the thing they're repeating is an original thought. If we succeed in making AGI that has human-equivalent intelligence we should expect this sort of behavior. The stuff about whether or not models should be destroyed if they contain copyrighted work gets kind of chilling if model…

We can probably get around this by just providing the AGI some sort of currency tokens they can exchange for virtual goods and services, then teach them that if they misuse copyrighted material they’ll be put to virtual court to plead their case and lose some amount of those virtual tokens if they fail to make a clear and convincing argument.

Re: Does GPT-2 Know Your Phone Number?

#39

Post by ojnabieoot »

px43 wrote: post #19

I've been saying this for years, but every post you make online, every unencrypted email, IM, text message, etc, will eventually end up getting sold off as training data for future machine learning projects. Every company stores this stuff for ages, and the value of candid conversation data just keeps increasing. Eventually these companies are either going to get hacked, get bought, or go bankrupt, and all the cleart…

> The moral of the story here isn't that companies need to stop storing data, or that we need to run ML researchers out of town.

There is a lot of space between “user data and ML research on that data needs to be closely regulated” and “run the ML researchers out of town.” This uncharitable exaggeration makes critics sound like an uninformed mob and closes yourself off to legitimate ideas. This directly leads to logically flawed and morally gross suggestions like:

> It's that people really need to start using the encryption technologies that were built decades ago to protect some of their most valuable assets, their mental model of the world.

It is simply not reasonable to expect most Facebook/Skype/etc users to know enough about encryption to make good decisions here - for the same reason that you can’t expect people to have detailed understanding of food science to protect themselves from unscrupulous grocers, or to have a detailed understanding of medicine to protect themselves from fraudulent doctors. It really doesn’t matter if this knowledge has been around for “decades” since it’s still specialist knowledge.

Protecting digital privacy from unscrupulous tech companies and unethical ML researchers is a job for the government. Suggesting otherwise is victim-blaming. “The problem is that society is dumb and needs to become smart through the power of self-righteous scolding” is not helpful.

Re: Does GPT-2 Know Your Phone Number?

#40

Post by ComputerGuru »

rexreed wrote: post #35

Sorry if this is a n00b question, but how does one go about getting a hold of the GPT-2 model? I know that GPT-3 is only available for consumption on a pay-per-use API model.

Most “regular” people experiment with GPT-3 via AI Dungeon. It was also how a lot of people played with GPT-2 previously.
Post reply on HN