Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Democratization in the Era of GPT-3

thegradient.pub

Post reply on HN
Print view View original

81–90 of 90 posts

Re: AI Democratization in the Era of GPT-3

#81

Post by searchableguy »

natch wrote: post #64
searchableguy wrote: post #11

Earlier quoted context omitted.

Shouldn't I own part of gpt3 because it's trained on my data and likely spitting out what I have commented somewhere on the internet? I think ml models should be public unless the data itself isn't.

It makes its own new arrangements of words. It’s not just clips of what it was trained with pasted together.

That's why I gave an example of translation. There are new sequences as well as words but the original owner still own the copyright based on the observed meaning or similarities. If you try searching some of the outputs from gpt, you will come across original source in many cases which are slightly or moderately different.

Re: AI Democratization in the Era of GPT-3

#82

Post by searchableguy »

FeepingCreature wrote: post #59
searchableguy wrote: post #11

Earlier quoted context omitted.

Shouldn't I own part of gpt3 because it's trained on my data and likely spitting out what I have commented somewhere on the internet? I think ml models should be public unless the data itself isn't.

Should you own part of my brain if I read your comment?

No. As per my layman understanding of copyright, I will have to prove at least:

1. Artistic/creative expression in my comments.

2. Direct commerical interest. If you are not benefiting from my comment directly, it can be considered fair use and hard to prove otherwise.

In gpt3's case, it outputs artistic expression in the form of poetry/stories and direct commerical interest from data used by common crawl and their api service.

You can argue about derivatives and so on. It's not clear cut in any way but your analogy isn't a good one. We can also give some leeway to humans compared to machines as latter is more deterministic. We know that gpt 3 was trained on which data. We can't say that for sure about you.

Re: AI Democratization in the Era of GPT-3

#83

Post by searchableguy »

PeterisP wrote: post #61
searchableguy wrote: post #37

Earlier quoted context omitted.

Curious about the downvotes. If you remove compute and code, what remains is data that isn't owned by a single entity in the case of gpt3. You can sell both compute and code which is what you own but should you be able to sell data? You can't translate a commerical book without paying the copyright holder. You own the copyright for the translation but everything else still remains that of the original author. Why wou…

The current legal understanding is that according to the current copyright laws the ML models trained on some data are not considered derivative works of that data, so if the data was obtained legally and without other restrictions (e.g. if the researchers sign a contract with the data owner that gives them access to data but agree to certain conditions) then the authors of the data do not have any claim on some righ…

Thanks for your answer. I appreciate it.

Re: AI Democratization in the Era of GPT-3

#84

Post by natch »

searchableguy wrote: post #81
natch wrote: post #64

Earlier quoted context omitted.

It makes its own new arrangements of words. It’s not just clips of what it was trained with pasted together.

That's why I gave an example of translation. There are new sequences as well as words but the original owner still own the copyright based on the observed meaning or similarities. If you try searching some of the outputs from gpt, you will come across original source in many cases which are slightly or moderately different.

This is an area open to interpretation but I wouldn’t go so far as to say someone who used a phrase before owns the copyright to a new work that happens to contain that phrase. But yes they certainly own the copyright to their original entire work in its original form. If you’re talking about excerpts longer than a phrase it does become more murky for sure.

Re: AI Democratization in the Era of GPT-3

#85

Post by FeepingCreature »

searchableguy wrote: post #82
FeepingCreature wrote: post #59

Earlier quoted context omitted.

Should you own part of my brain if I read your comment?

No. As per my layman understanding of copyright, I will have to prove at least: 1. Artistic/creative expression in my comments. 2. Direct commerical interest. If you are not benefiting from my comment directly, it can be considered fair use and hard to prove otherwise. In gpt3's case, it outputs artistic expression in the form of poetry/stories and direct commerical interest from data used by common crawl and their a…

> In gpt3's case, it outputs artistic expression in the form of poetry/stories and direct commerical interest from data used by common crawl and their api service.

Okay, so you don't own part of my brain, but StackOverflow would? Because I definitely commercially use information I've read there.

And I mean, I know English because of all the English material I've read. If I write poetry, I'm working off training data I've read online. It would be borderline impossible to be an artist under these rules.

> We know that gpt 3 was trained on which data.

We don't know what data GPT uses for a given answer. We don't know what data I use for a given answer. But we know what data I read, and we know what data GPT read.

Far as I can tell, the brain analogy is 1:1 here.

Re: AI Democratization in the Era of GPT-3

#86

Post by searchableguy »

FeepingCreature wrote: post #85
searchableguy wrote: post #82

Earlier quoted context omitted.

No. As per my layman understanding of copyright, I will have to prove at least: 1. Artistic/creative expression in my comments. 2. Direct commerical interest. If you are not benefiting from my comment directly, it can be considered fair use and hard to prove otherwise. In gpt3's case, it outputs artistic expression in the form of poetry/stories and direct commerical interest from data used by common crawl and their a…

> In gpt3's case, it outputs artistic expression in the form of poetry/stories and direct commerical interest from data used by common crawl and their api service. Okay, so you don't own part of my brain, but StackOverflow would? Because I definitely commercially use information I've read there. And I mean, I know English because of all the English material I've read. If I write poetry, I'm working off training data…

> Okay, so you don't own part of my brain, but StackOverflow would? Because I definitely commercially use information I've read there.

They have a well established license you need to follow to use the answers posted on their site and what freedom it provides you.

Using stackoverflow answer to add little bit of code in your codebase is different than reselling stackoverflow answers behind a pay wall.

> But we know what data I read, and we know what data GPT read

No unless you are under surveillance 24/7.

And as I already said, machines are different than people. Copyright laws acknowledge that so human situation wouldn't be comparable. You aren't mass producing creativity and selling it.

Re: AI Democratization in the Era of GPT-3

#87

Post by FeepingCreature »

searchableguy wrote: post #86
FeepingCreature wrote: post #85

Earlier quoted context omitted.

> In gpt3's case, it outputs artistic expression in the form of poetry/stories and direct commerical interest from data used by common crawl and their api service. Okay, so you don't own part of my brain, but StackOverflow would? Because I definitely commercially use information I've read there. And I mean, I know English because of all the English material I've read. If I write poetry, I'm working off training data…

> Okay, so you don't own part of my brain, but StackOverflow would? Because I definitely commercially use information I've read there. They have a well established license you need to follow to use the answers posted on their site and what freedom it provides you. Using stackoverflow answer to add little bit of code in your codebase is different than reselling stackoverflow answers behind a pay wall. > But we know wh…

> Using stackoverflow answer to add little bit of code in your codebase is different than reselling stackoverflow answers behind a pay wall.

Working for a company using information I learnt on SO is exactly equivalent to reselling SO understanding behind a pay wall.

> No unless you are under surveillance 24/7.

So if I'm under surveillance, that means I'm affected by copyright?

> And as I already said, machines are different than people.

I disagree.

> You aren't mass producing creativity and selling it.

As a software developer: yes I am!

Re: AI Democratization in the Era of GPT-3

#88

Post by searchableguy »

FeepingCreature wrote: post #87
searchableguy wrote: post #86

Earlier quoted context omitted.

> Okay, so you don't own part of my brain, but StackOverflow would? Because I definitely commercially use information I've read there. They have a well established license you need to follow to use the answers posted on their site and what freedom it provides you. Using stackoverflow answer to add little bit of code in your codebase is different than reselling stackoverflow answers behind a pay wall. > But we know wh…

> Using stackoverflow answer to add little bit of code in your codebase is different than reselling stackoverflow answers behind a pay wall. Working for a company using information I learnt on SO is exactly equivalent to reselling SO understanding behind a pay wall. > No unless you are under surveillance 24/7. So if I'm under surveillance, that means I'm affected by copyright? > And as I already said, machines are di…

Section 306 and 313.2 which states that content purely created by machines will not be eligible for protection. Content that is randomly generated will not be eligible either.

https://www.copyright.gov/comp3/

Also see: https://stackoverflow.com/help/licensing

> As a software developer: yes I am!

No. Another reason why the copyright protection not applying to purely automatic processes make sense. A machine could theoretically generate every potential expression understandable by the compiler or interpreter. You can't. You aren't capable of the same mass production. You will die at some point and your work will be released for public but same won't happen to machines.

> So if I'm under surveillance, that means I'm affected by copyright?

No. It just makes it easier to prove original sources.

The reason why I proposed my initial argument is simply for accountability and right to information.

1. Unlike stackoverflow which every participant agrees on a license when they signup, common crawl is different. I believe everyone has right to the weights derived from their data unless they explicitly sell them or give permission.

2. When gpt3 is used for interviews and other filtration: it is important for anyone to be able discern it.

Re: AI Democratization in the Era of GPT-3

#89

Post by FeepingCreature »

searchableguy wrote: post #88
FeepingCreature wrote: post #87

Earlier quoted context omitted.

> Using stackoverflow answer to add little bit of code in your codebase is different than reselling stackoverflow answers behind a pay wall. Working for a company using information I learnt on SO is exactly equivalent to reselling SO understanding behind a pay wall. > No unless you are under surveillance 24/7. So if I'm under surveillance, that means I'm affected by copyright? > And as I already said, machines are di…

Section 306 and 313.2 which states that content purely created by machines will not be eligible for protection. Content that is randomly generated will not be eligible either. https://www.copyright.gov/comp3/ Also see: https://stackoverflow.com/help/licensing > As a software developer: yes I am! No. Another reason why the copyright protection not applying to purely automatic processes make sense. A machine could theo…

> No. Another reason why the copyright protection not applying to purely automatic processes make sense. A machine could theoretically generate every potential expression understandable by the compiler or interpreter. You can't.

Sure I can, it'll just take a while.

> You will die at some point and your work will be released for public but same won't happen to machines.

I'm not arguing machine produced work should be copyright protected. This is about quite the opposite, about how copyright protection relates to the process of learning as exercised by machines vs. humans.

> 1. Unlike stackoverflow which every participant agrees on a license when they signup, common crawl is different. I believe everyone has right to the weights derived from their data unless they explicitly sell them or give permission.

The big disagreement here is that I think posting anything on the internet gives an implicit license for "brainlike consumption" - reading, understanding and learning. GPT isn't doing anything while training that I'm not doing while reading. It would be impossible for a human to read the text at all without engaging in these processes, and so it must likewise be permissible for GPT.

Consider the futuristic scenario of a brain implant extending your memory for Alzheimer's patients. GPT is not so dissimilar from such an implant operating in standalone mode. I don't see how screwing a brain onto it should change the ethical algebra to make it more permissible.

Re: AI Democratization in the Era of GPT-3

#90

Post by searchableguy »

FeepingCreature wrote: post #89
searchableguy wrote: post #88

Earlier quoted context omitted.

Section 306 and 313.2 which states that content purely created by machines will not be eligible for protection. Content that is randomly generated will not be eligible either. https://www.copyright.gov/comp3/ Also see: https://stackoverflow.com/help/licensing > As a software developer: yes I am! No. Another reason why the copyright protection not applying to purely automatic processes make sense. A machine could theo…

> No. Another reason why the copyright protection not applying to purely automatic processes make sense. A machine could theoretically generate every potential expression understandable by the compiler or interpreter. You can't. Sure I can, it'll just take a while. > You will die at some point and your work will be released for public but same won't happen to machines. I'm not arguing machine produced work should be…

> GPT isn't doing anything while training that I'm not doing while reading.

That's another disagreement. I consider the scale and classification (machine?) to be an important distinction.

I would like to think more about this. I probably won't change my initial stance on requiring ml models to be public if they are used commerically even if point 1 shouldn't apply. However, whether they get IP protection of any sort and who owns it depends on sorting out other points I have considered after this.

Post reply on HN