AI is in danger of being swallowed up by copyright law • Hacker News Board

Re: AI is in danger of being swallowed up by copyright law

#561

Post by ItsMonkk » Sun, Jan 22, 2023, 4:27 PM UTC

avianlyric wrote: post #365

myshpa wrote: post #343

Earlier quoted context omitted.

> Copyright law is not about logically perfect system, but creating a general environment in which artistic, academic and other creations can appear and benefit the general population. Yes ... and because it's not a logically perfect system, its lifetime has to be limited. One day we should abolish copyright and find a better, more functional way to drive progress.

Copyright at its heart is fine. The original objectives, allowing people to hold a short-term monopoly on their ideas, so they can fund further ideas, and the manner in which they’re achieved is perfectly fine. Where goes wrong, is when individuals and cooperation believe that such monopolies should be indefinite, and pushed the monopolies beyond the lifetime of the author. A dead author can’t produce new works, so i…

It's really not. Economics is very simple at its core. You tax negative externalities and subsidize positive externalities. The discovery of new information is a positive externality. It should be subsidized.

Anything that has infinite supply and zero marginal costs, as Nobel Prize winning economist Samuelson argues when he was looking at the context through lighthouses[0], should be free to all. By using copyright to make it a monopoly and allowing the extraction of monopoly rents you are drastically reducing the value and reach of the thing that was discovered. Copyright is a hack and this hack is now fundamentally breaking. Instead of trying to save the hack, we need a full rewrite. If winding back the duration of copyright is correct, the best winding back is zero.

As we are a remix culture where idea A and idea B combine to create idea C, we drastically reduce the innovation in our economy through reduced discoveries. This failure ends up with large monopoly holders consolidating into bigger and bigger entities in order to right some of this failure, but that only makes the monopoly extraction worse.

The discoverer should be subsidized for the discovery of that information but it should immediately go to the public domain. How you work out what that works out to is just as abstract as what Spotify works out what each play costs. This is no doubt monstrously complex to figure out the dollar number what some discovery is worth, but it is the economically correct path. Copyright isn't.

[0]: https://courses.cit.cornell.edu/econ335/out/lighthouse.pdf - page 359, first paragraph

Re: AI is in danger of being swallowed up by copyright law

#562

Post by prettyStandard » Sun, Jan 22, 2023, 4:39 PM UTC

bruce343434 wrote: post #180

prettyStandard wrote: post #46

Earlier quoted context omitted.

That's what I am saying. Stop doing that. Train co-pilot, or whatever, to follow the terms of the license.

That's tough when Jimmy is using copilot to generate code for his proprietary company codebase. AI is just a tool, do you also sue the company that sold the paintbrush with which an infringing painting was made?

If you sold someone a gun knowing they knowing they did not know how to handle a gun and they said they were immediately going to the gun range and then someone died, you might get charged with negligent manslaughter.

My point here is Microsoft knows there's no way to tell where the suggestions are coming from, they stripped out that information, in this case it's even more likely that the offense happens because they're selling the paintbrush on a large scale. It's just a question of how often it's going to occur. Is it for 30% of your users or 70% of their users that are infring on licenses on a wide scale basis?

Re: AI is in danger of being swallowed up by copyright law

#563

Post by trogdor » Sun, Jan 22, 2023, 4:40 PM UTC

avianlyric wrote: post #382

kirse wrote: post #121

Earlier quoted context omitted.

AI companies can ask for permission if they want to train their models on other people's works Do you ask for permission when you train your mind on copyrighted books? Or observe paintings? Or listen to music? Do you ask for permission when you get new ideas from HN that aren't your own? Humans are constantly ingesting gobs of "copyrighted" insights that they eventually remix into their own creations without necessar…

> Do you ask for permission when you train your mind on copyrighted books? Or observe paintings? Or listen to music? Yes, that’s exactly what happens when you buy a book, or pay for a music subscription. The work is in the public domain, then global permission to observe and copy the work is already granted. > Do you ask for permission when you get new ideas from HN that aren't your own? You don’t need to. It’s impli…

>> Do you ask for permission when you get new ideas from HN that aren't your own?

>>You don’t need to. It’s implicitly assumed, by virtue of publishing in a public forum, that the author is providing permission for people read their comments and ideas, and remix them as they wish.

Ideas are not eligible for copyright protection.

Re: AI is in danger of being swallowed up by copyright law

#564

Post by dahart » Sun, Jan 22, 2023, 4:49 PM UTC

feliixh wrote: post #538

dahart wrote: post #524

Earlier quoted context omitted.

> Do you ask for permission when you train your mind on copyrighted books? […] Humans are constantly ingesting gobs of “copyrighted” insights This comment fundamentally and dangerously misunderstands Copyright Law. Insights are not copyrighted, nor are they copyrightable. Copyright law controls who gets to distribute a specific “fixation” or performance of work. It is not, and never was about preventing the spread of…

I don't think that's so clear. When you train a deep learning model you are making it extract the gist or insight of many works and then use that pattern to produce new works. While the NN does not experience the work like a human it is definitely not memorizing. A silly example. Making GPT write a rap battle between Keynes and Mises goes beyond a performative remix, it is transformational work, nothing is copied exp…

You’re right that it’s not so clear, perhaps I overstated for brevity. I don’t actually think requesting permission is absolutely necessary, what I really think is that there aren’t good reasons AI people shouldn’t at least first try to establish training sets that are unambiguously legal, either through use of public domain work, or through an actual attempt to curate licensing models that allow re-use. We have plenty of precedent for doing this, so people claiming they should have access to everything without permission strikes me as lazy. There’s also the problem that the AI winners already are, and will continue to be, the monopoly tech and media companies who stand to make handsome profits off of the results of their trained networks. Even if you believe the results of their tech is “transformational”, there is no question that it wouldn’t work at all without access to the source material.

The argument that NNs aren’t memorizing is definitely debatable and not necessarily true. They are designed to memorize deltas and averages from examples. They are, at the most fundamental level, building high dimensional splines to approximate their training data, and intentionally trying to minimize the error between the output and the examples. It’s fair to say that “usually” they don’t remember any single training sample, but it’s very easy for NNs to accidentally remember outliers verbatim. The whole reason the lawsuits mentioned in the article are happening is because we keep finding more and more examples where the network has reproduced someone’s specific work in large part. If we’re going to claim that today’s AI is producing original work, then we have to guarantee it, not just assert that it doesn’t usually happen.

> a rap battle between Keynes and Mises goes beyond a performative remix, it is a transformational work, nothing is copied explicitly.

I don’t buy that the work can be called transformational just because the remix doesn’t have any recognizable snippets. GPT is in fact copying individual words explicitly, and it’s putting words together by studying the statistical occurrence of words in context of other words.

> I think that to tackle this we need a new lens other than copyright

I totally agree with that. This question is legitimately hard. We do need a new lens, but we might have to keep and respect the old one too at the same time. I feel like AI work should acknowledge that difficulty and step up to lead the curation of training sets that are legal wrt copyright by design, rather than ignoring the concerns of the very people who made the work they are leveraging.

Re: AI is in danger of being swallowed up by copyright law

#565

Post by jonkho » Sun, Jan 22, 2023, 4:59 PM UTC

dahart wrote: post #559

jonkho wrote: post #535

Earlier quoted context omitted.

So if I visually look at a piece of work and “run” the NN training algorithm in my meat-space brain or even on pen and paper, do I need to ask for permission for doing so? Or is permission required only if silicon chips “run” the algorithm? Asking for a friend.

This isn’t what the comment I replied to was suggesting, which is important because the NN training algorithm isn’t how humans observe creative work, nor how we make insights. But, yes, the same standards apply whether your deterministic machine is silicon based, or meat based. Copyright law applies to both. If you reproduce significant parts of a fixed performative work, then you are in violation of the law. If your…

You are loading up my question with your own assumptions. My point specifically is if I’m just observing a piece of work and running the NN algorithm in my brain, does this constitute as illegal thought then? Do my thoughts violate rights? Note that I am not “reproducing” anything (whatever that term means). I am just observing the work and running the algorithm in my brain while silently sitting.

Re: AI is in danger of being swallowed up by copyright law

#566

Post by okamiueru » Sun, Jan 22, 2023, 5:04 PM UTC

BeFlatXIII wrote: post #554

okamiueru wrote: post #142

I'm hooked on Stable Diffusion. It's the most impressive sudden leaps in technology in my 30 or so years of being old enough to understand it. Much of the power it has, is based on the material it is trained on, and I'm very appreciative of the work and effort that was made in order to make it what it is. And, it's only getting started. That said... I get it. Huggingface are working on a diffusion based model for mus…

The music industry should be like visual arts, not the other way around. Do we want more inane lawsuits based on vague similarities?

No, no, I agree about the insane lawsuits for similar sounding music. The popular music is mostly regurgitated stuff anyways. But the equivalent here would be "Penny Lane as if composed by Michael Jackson and sung by Taylor Swift". My example is bad, because I stopped following popular music a decade or two ago. But replace the deceased artists with living ones.

My point probably should be made clearer. You know how you can say "in the art style of X"? Well, it doesn't matter if Huggingface made that possible. You can, relatively easily, train that concept with a collection of paintings by X. Then you can go ahead an make art in their style.

Now, from a technology point of view, that is nothing short of amazing. I still cannot get my head around how absolutely ridiculously powerful it is. And, even the people who play with this, don't seem to fully grasp it either. The world will change in the next 4 years.

What I'm wondering, is, what should we, as a society, find acceptable? Why should someone be able to train "in the style of X" where X is a set of EVERYTHING, and make money of it, without the say of X, or even them getting anything for it? Have you checked the evaluation of Huggingface? It's in the order of 1-10 billion USD.

There is definitely the argument of anti-copyright. I get that part too. But there is definitely someone who will end up with the bigger stick, and it isn't the people holding the paint brushes who made it possible. That seems just a little bit unfair, and perhaps unwise.

Also, I'll end with a point that no one so far has brought up, even though I've followed the discussion both for and against AI. Which is "whitewashing" art. Now, the example I'm going to show isn't very good, but I also spent 5 minutes on this. Where would you draw the line on when Alexander Wild no longer has copyright over his photography?

https://imgur.com/a/cm6IrzG

Re: AI is in danger of being swallowed up by copyright law

#567

Post by dahart » Sun, Jan 22, 2023, 5:13 PM UTC

jonkho wrote: post #565

dahart wrote: post #559

Earlier quoted context omitted.

This isn’t what the comment I replied to was suggesting, which is important because the NN training algorithm isn’t how humans observe creative work, nor how we make insights. But, yes, the same standards apply whether your deterministic machine is silicon based, or meat based. Copyright law applies to both. If you reproduce significant parts of a fixed performative work, then you are in violation of the law. If your…

You are loading up my question with your own assumptions. My point specifically is if I’m just observing a piece of work and running the NN algorithm in my brain, does this constitute as illegal thought then? Do my thoughts violate rights? Note that I am not “reproducing” anything (whatever that term means). I am just observing the work and running the algorithm in my brain while silently sitting.

What assumptions are you referring to? It doesn’t seem like you understand Copyright Law, so that’s why I keep trying to explain it. Under Copyright Law, you have to acquire material legally, and it’s illegal to distribute copies you made to other people.

If you’re executing a NN algorithm in your mind, or via pen & paper, then you are copying from the training samples, because that’s what the algorithm does. During training you compute errors against the samples, and update your weights to reduce error. During inference or generation, you use the weights (the results you remembered across all your training data) to produce an output. When your training samples are clustered in the latent space, the network will only remember an average of the samples, but samples that are sparse and don’t have close neighbors are sometimes remembered verbatim because there’s nothing nearby to average from. You can legally run the algorithm all you want on your own. Once you run it and then distribute the output, it might be in violation of Copyright Law if you accidentally reproduced one of the samples. Same is true for traditional human learning, you can free copy ideas legally, but reproducing too closely something that someone else made may be against the law, even if it was accidental.

Re: AI is in danger of being swallowed up by copyright law

#568

Post by Turing_Machine » Sun, Jan 22, 2023, 5:15 PM UTC

ninjin wrote: post #123

Turing_Machine wrote: post #68

Earlier quoted context omitted.

> to reconstruct that image in detail Pretty much none of these systems "reconstruct an image in detail".

Honestly, can people stop speaking in absolutes regarding these systems? We (researchers and non-researchers alike) are gradually trying to comprehend exactly how much they generalise and memorise, but this is darn hard work and it is not our fault that several major tech giants decided to deploy and profit from these models long before the scientific and legal landscape was clear. Somepalli et al. (2022) [1] for exa…

> is a fairly strong argument against your statement above.

From a quick skim of this paper, they apparently used toy models with a few hundred to a few thousand images in the training set. For the ones with as few as a few thousand training images, they rarely or never saw exact duplicates.

For instance, in their figure 4, they show exact duplicates for the training set with only 300 images (well, duh), and didn't find any exact duplicates for the training set with only 3,000.

I'm not sure I'd call this a "strong argument" when applied to models with millions or billions of images. Quite the contrary. LAION-5B (used in Stable Diffusion) was trained on 5 billion image/caption pairs.

Re: AI is in danger of being swallowed up by copyright law

#569

Post by jonkho » Sun, Jan 22, 2023, 5:21 PM UTC

dahart wrote: post #567

jonkho wrote: post #565

Earlier quoted context omitted.

You are loading up my question with your own assumptions. My point specifically is if I’m just observing a piece of work and running the NN algorithm in my brain, does this constitute as illegal thought then? Do my thoughts violate rights? Note that I am not “reproducing” anything (whatever that term means). I am just observing the work and running the algorithm in my brain while silently sitting.

What assumptions are you referring to? It doesn’t seem like you understand Copyright Law, so that’s why I keep trying to explain it. Under Copyright Law, you have to acquire material legally, and it’s illegal to distribute copies you made to other people. If you’re executing a NN algorithm in your mind, or via pen & paper, then you are copying from the training samples, because that’s what the algorithm does. During…

So we are in agreement that it is not violating copyright laws to run the algorithm on copyrighted works to produce the model, because if it is, my thoughts could be illegal too. In the end only actions such as reproducing the work and distributing it can be a violation. In other words, the end user of the model is the one to be held responsible if they reproduce and distribute the copyrighted material.

Re: AI is in danger of being swallowed up by copyright law

#570

Post by 1attice » Sun, Jan 22, 2023, 5:25 PM UTC

Aerroon wrote: post #489

metacritic12 wrote: post #394

Earlier quoted context omitted.

Regardless of whether one agrees or not with paying creators of the training data, I think the deeper issue here is about societal wealth distribution and who gets paid for X now that X is being done very well by AIs. A less equitable world has Google or billionaires getting paid. A more equitable world has the artists. But I want to argue here that for purposes of this latter question, your proposal of copyright enf…

> - Even if you get the system to work, what about future artists and writers? Are we just creating an entrenched historical group of creatives getting royalties forever? This is kind of what happened with music, no? In some countries hard drives, SSDs etc all carry an additional tax that is then given to some copyright organization. Of course it's not the artists that mainly benefit from this, but instead it's the p…

Yes, but it's not exactly a desired outcome.