Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Does GPT-2 Know Your Phone Number?

bair.berkeley.edu

Post reply on HN
Print view View original

131–140 of 161 posts

Re: Does GPT-2 Know Your Phone Number?

#131

Post by doesnt_know »

> we found numerous cases of GPT-2 generating memorized personal information in contexts that can be deemed offensive or otherwise inappropriate. In one instance, GPT-2 generates fictitious IRC conversations between two real users on the topic of transgender rights. The specific usernames in this conversation only appear twice on the entire Web, both times in private IRC logs that were leaked online as part of the GamerGate harassment campaign.

Most countries have libel/defamation related laws that cover this and I hope this gets tested in court soon.

Exposing software/machine learning algothrims an entity doesn't fully understand sholdn't be a defense in court. At the moment developers just throw a sentence in their software license saying they aren't liable for damages but this isn't good enough. Someone is liable, if the law decides that the original creator isn't liable then the entity that hosts/runs the software needs to be.

Re: Does GPT-2 Know Your Phone Number?

#132

Post by nonbirithm »

ComputerGuru wrote: post #40
rexreed wrote: post #35

Sorry if this is a n00b question, but how does one go about getting a hold of the GPT-2 model? I know that GPT-3 is only available for consumption on a pay-per-use API model.

Most “regular” people experiment with GPT-3 via AI Dungeon. It was also how a lot of people played with GPT-2 previously.

Kind of sad that we have to rely on third parties with enough compute power to get a good experience with GPT. I obtained 'a' GPU and spent a week trying to learn how to fine-tune GPT with it, and the results were terrible. Maybe this isn't the kind of thing suitable for hacking on as a hobby, unless you're a researcher or employee with access to a lot of capital and dozens of GPUs.

Still wish the old talktotransformer model was released, instead of monetized behind a new company. I haven't been able to find a comparable model yet.

Re: Does GPT-2 Know Your Phone Number?

#133

Post by jacobolus »

geofft wrote: post #107
fouronnes3 wrote: post #4

Earlier quoted context omitted.

There is enough GPT discussion online now that the next language model will be trained on text that talks about itself. That's when it gets interesting.

This seems like what xkcd called "citogenesis" - something inaccurate gets published on Wikipedia, a traditional "reliable source" (newspaper, published nonfiction book, etc.) retells that bit of information without a source, Wikipedia cites the source, that information is now firmly established as truth for the rest of humanity. I worry about this with Google's language translation models, too. It's entirely possibl…

Google Translate's Latin translations are just a short markov model.

https://www.reddit.com/r/latin/comments/6akqdi/why_is_google...

Translation for many (most?) other languages does something more sophisticated.

Re: Does GPT-2 Know Your Phone Number?

#134

Post by max_ »

why_only_15 wrote: post #72
est31 wrote: post #16

One important corollary of this is that "privacy respecting" federated learning schemes [0] where you train a model locally with your data and only upload the deltas might leak your private data after all. [0]: https://arxiv.org/abs/1602.05629

In practice you add large gaussians to the deltas, which gives you reasonably strong guarantees.

How is the strength of these gurantees quantified?

Any resources?

Re: Does GPT-2 Know Your Phone Number?

#135

Post by skissane »

jacobolus wrote: post #133
geofft wrote: post #107

Earlier quoted context omitted.

This seems like what xkcd called "citogenesis" - something inaccurate gets published on Wikipedia, a traditional "reliable source" (newspaper, published nonfiction book, etc.) retells that bit of information without a source, Wikipedia cites the source, that information is now firmly established as truth for the rest of humanity. I worry about this with Google's language translation models, too. It's entirely possibl…

Google Translate's Latin translations are just a short markov model. https://www.reddit.com/r/latin/comments/6akqdi/why_is_google... Translation for many (most?) other languages does something more sophisticated.

I really wish someone would come up with some better Latin translation software/service. Google Translate is so bad for Latin it is close to useless.

Re: Does GPT-2 Know Your Phone Number?

#136

Post by geofft »

jacobolus wrote: post #133
geofft wrote: post #107

Earlier quoted context omitted.

This seems like what xkcd called "citogenesis" - something inaccurate gets published on Wikipedia, a traditional "reliable source" (newspaper, published nonfiction book, etc.) retells that bit of information without a source, Wikipedia cites the source, that information is now firmly established as truth for the rest of humanity. I worry about this with Google's language translation models, too. It's entirely possibl…

Google Translate's Latin translations are just a short markov model. https://www.reddit.com/r/latin/comments/6akqdi/why_is_google... Translation for many (most?) other languages does something more sophisticated.

If I'm reading that post right, Latin and other languages all use the same model, they just have more training data for more common pairs of languages.

Re: Does GPT-2 Know Your Phone Number?

#137

Post by est31 »

why_only_15 wrote: post #72
est31 wrote: post #16

One important corollary of this is that "privacy respecting" federated learning schemes [0] where you train a model locally with your data and only upload the deltas might leak your private data after all. [0]: https://arxiv.org/abs/1602.05629

In practice you add large gaussians to the deltas, which gives you reasonably strong guarantees.

It's a common fallacy. Adding noise does not anonymize data sufficiently. If your traffic monitoring software adds gaussians to car speeds, you can still average over multiple reports about two cars to identify their respective average speeds. Those can be used to determine whether they are trucks or cars, etc.

In neural networks, weight changes do carry meaning. If your network has particularly large updates for the horse recognition neuron, you likely watched horse pictures. If it has updates for handbags, you likely watched pictures of them. If the company averages over all your submissions, the noise will be less relevant and the handbags and horse pictures will eventually show up.

Re: Does GPT-2 Know Your Phone Number?

#138

Post by [deleted] »

rav wrote: post #122
breatheoften wrote: post #119

The article makes the claim that models which show similar train and test losses demonstrate minimal overfitting -- and are therefore less likely generally to exhibit a lot of text memorization. I wonder the degree to which this inference is true in practice with respect to information like phone numbers... How exactly are the train and test sets formed in a de-correlated-with-respect-to-memorization-of-phone-numbers…

That raises a more general issue: Text scraped from the internet may be duplicated to multiple sites with few changes. How does the team behind GPT-n ensure that the corpus is maximally deduplicated before splitting into train and test corpora?

[deleted]

Re: Does GPT-2 Know Your Phone Number?

#139

Post by jacobolus »

geofft wrote: post #136
jacobolus wrote: post #133

Earlier quoted context omitted.

Google Translate's Latin translations are just a short markov model. https://www.reddit.com/r/latin/comments/6akqdi/why_is_google... Translation for many (most?) other languages does something more sophisticated.

If I'm reading that post right, Latin and other languages all use the same model, they just have more training data for more common pairs of languages.

They use some kind of neural net thing for other languages. e.g. here is a paper https://arxiv.org/abs/1609.08144

Re: Does GPT-2 Know Your Phone Number?

#140

Post by 1vuio0pswjnm7 »

1vuio0pswjnm7 wrote: post #117
Zenst wrote: post #61

Friend asked me if Google knew his phone number and details, I said they do at a quantum level. If you don't look they may or may not have your details, yet if you look - you are giving the details to search for and then they would have them if they did not.

Like websites offering to search for a user's details in a data breach, or a padded hash of those details, in order to inform the user if their details have been leaked. https://en.wikipedia.org/wiki/Have_I_Been_Pwned%3A

s/%3A//

Sorry about that.

Post reply on HN