Live data from Hacker News

Does GPT-2 Know Your Phone Number?

bair.berkeley.edu

111–120 of 161 posts

Re: Does GPT-2 Know Your Phone Number?

#111

It’s always fascinating to apply data/results like from this paper to help evaluate the hypothesis that machine learning/AI is mostly just a rough “lookup table” or memorization.

Anything trained by gradient descent is in some sense a weighted combination lookup table (see domingos’s recent paper on everything being kernel machines, I’m on my phone so won’t try to find it). The crucial bit is finding an appropriate weighting. If you look up the nearest training data point and just use it to predict, you assume a notion of “nearest.” Getting that right is the trick.

So even if it all is mathematically equivalent to approximate lookup and approximate memorization, that doesn’t mean it’s “just” that.

Re: Does GPT-2 Know Your Phone Number?

#112
post #86

> There is a legal grey area as to how these regulations should apply to machine learning models. For example, can users ask to have their data removed from a model’s training data? Moreover, if such a request were granted, must the model be retrained from scratch? The fact that models can memorize and misuse an individual’s personal information certainly makes the case for data deletion and retraining more compellin…

I think there are also going to be new difficult problems arising training on copyrighted data. Can I do a DMCA if your model contains/regurgitates too much of my book verbatim in a way that doesn’t meet fair use?

Re: Does GPT-2 Know Your Phone Number?

#113

It’s always fascinating to apply data/results like from this paper to help evaluate the hypothesis that machine learning/AI is mostly just a rough “lookup table” or memorization.

It kind of is. The first self-learning algorithm I was taught adjusted a single variable in order to aproximate a linear function. Now, being continuous and unbounded, it is mathematically impossible to memoize such a function, however you can memoize the function calls as they happen. The results would be indistinguishable.

However real machine learning tries to approximate functions in n-dimensions and that is really, really, really hard to do. Currently no one really has a lookup table. There are some inputs where the error level is acceptably low, and others where the model just isn't optimized enough and the errors are ridiculous. The only question that remains to be answered, is whether any of these high-dimensional functions can actually be found by machine learning, or if we are just stuck with these endless approximations. Also I suppose you could ask if any such functions actually exist; maybe certain phenomena are just pure chaos.

Re: Does GPT-2 Know Your Phone Number?

#114
post #91

Earlier quoted context omitted.

Even in the science fiction version of this, we're still talking about a copy. It's not you, you can't leave your body or be uploaded. It's just a digital copy that looks like you.

You can destroy the original. This would make a good SF story. Probably has already :-)

> Probably has already

It's a variation on the transporter problem from Star Trek - https://www.youtube.com/watch?v=nQHBAdShgYI

Re: Does GPT-2 Know Your Phone Number?

#115
post #3

Playing with the AI dungeon a while back (on the GPT-2 mode) I was presented with a tilapia recipe, titled "Kittencal's Broiled Tilapia" - it sounded bizarre so I decided to do a google search and I found that it was directly pulled from from https://www.recipezazz.com/recipe/broiled-parmesan-tilapia-7... - the user who posted it was 'Kittencal'

There is enough GPT discussion online now that the next language model will be trained on text that talks about itself. That's when it gets interesting.

This has been a thing for over a year now. https://www.reddit.com/user/subsimgpt2metaGPT2/?sort=top

Re: Does GPT-2 Know Your Phone Number?

#116

It’s always fascinating to apply data/results like from this paper to help evaluate the hypothesis that machine learning/AI is mostly just a rough “lookup table” or memorization.

Anything trained by gradient descent is in some sense a weighted combination lookup table (see domingos’s recent paper on everything being kernel machines, I’m on my phone so won’t try to find it). The crucial bit is finding an appropriate weighting. If you look up the nearest training data point and just use it to predict, you assume a notion of “nearest.” Getting that right is the trick. So even if it all is mathem…

Thanks for the reference. Here it is:

“We show, however, that deep networks learned by the standard gradient descent algorithm are in fact mathematically approximately equivalent to kernel machines, a learning method that simply memorizes the data and uses it directly for prediction via a similarity function (the kernel)“

https://arxiv.org/abs/2012.00152

Re: Does GPT-2 Know Your Phone Number?

#117
post #61

Friend asked me if Google knew his phone number and details, I said they do at a quantum level. If you don't look they may or may not have your details, yet if you look - you are giving the details to search for and then they would have them if they did not.

Like websites offering to search for a user's details in a data breach, or a padded hash of those details, in order to inform the user if their details have been leaked. https://en.wikipedia.org/wiki/Have_I_Been_Pwned%3A

Re: Does GPT-2 Know Your Phone Number?

#118
post #8
post #3

Playing with the AI dungeon a while back (on the GPT-2 mode) I was presented with a tilapia recipe, titled "Kittencal's Broiled Tilapia" - it sounded bizarre so I decided to do a google search and I found that it was directly pulled from from https://www.recipezazz.com/recipe/broiled-parmesan-tilapia-7... - the user who posted it was 'Kittencal'

So in terms of copyright, is GPT-2 a derived work of that recipe? Or generally are models derived works of their training data? It seems lots of people use training data from Flickr, like COCO, and then use the resulting model for commercial services.

The current legal understanding is that according to the current copyright laws the ML models trained on some data are not considered derived works of that data, so if the data was obtained legally and without other restrictions (e.g. if the researchers sign a contract with the data owner that gives them access to data but agree to certain conditions) then the authors of the data do not have any claim on some rights to the resulting model.

One aspect that causes this is that historically statistical models calculated from large volumes of text (which is a notion that predates computers, e.g. frequency dictionaries and the whole [sub]field of quantitative corpus linguistics) have been considered facts about that corpus of text and thus not copyrightable at all or (depending on jurisdiction) entitled to different set of protections/limitations assigned to compilations of facts, which give some rights to the people who compiled the facts but no rights to the source of these facts (since facts as such aren't entitled to protection by copyright law).

This also applies to many forms of analysis of audiovisual data, where the copyrights of the source works do not transfer to the results of the statistical or qualitative analysis and can't limit their creation, distribution or sale.

The appropriate analogy to a commercial book or movie is not a translation, but some analysis of it - e.g. a thorough literary review and critique of some book or movie is a separate work with its own copyright, and the original author has no claim on it despite the fact that is (obviously) based on the contents of the work and describes it in great detail. Including verbatim fragments of the work is limited (fair use allows some inclusions but not all), but all the other details are not.

The whole notion of copyrightability of ML model weight files is interesting and IMHO not settled. You could argue that there is some creative expression in forming the model (which would support it being copyrightable) or you could argue that it's a mechanistic result of the application of some algorithm and settings (which have the creative part, and are copyrightable on their own), and so the output can't be copyrightable, no matter how much work (human or machine), time and cost it took - at least in USA copyright law doctrine (e.g. Feist Publications v. Rural Telephone Service) is that mere "sweat of the brow" (no matter how much) does not entitle a work to copyright protection; it requires application of human creativity to create an original work, and automated processes can't satisfy that requirement.

And crucially, if some output is not copyrightable in the first place, it can't be considered a derived work according to copyright law i.e. the exclusive right of authors to create derived works (or grant permission for others to do so) does not apply.

Another analogy might be a simple n-gram model (i.e. counts of bigrams - word pairs, trigrams, etc) which is quite clearly a mechanistic noncreative collection of facts about a dataset, and is also able to "answer" questions such as what is someone's telephone number if that was in the source data.

Re: Does GPT-2 Know Your Phone Number?

#119
The article makes the claim that models which show similar train and test losses demonstrate minimal overfitting -- and are therefore less likely generally to exhibit a lot of text memorization.

I wonder the degree to which this inference is true in practice with respect to information like phone numbers... How exactly are the train and test sets formed in a de-correlated-with-respect-to-memorization-of-phone-numbers manner for models of GPT class that are trained on corpus's the size of the internet?

If a particular person's phone number occurs 1000 times in the corpus prior to being split into train/test sets, what are the chances that the number only appears in either the train or test set but not both?

Post reply on HN