Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI: Copy, Steal, Paste

computerworld.com

Post reply on HN
Print view View original

61–70 of 79 posts

Re: OpenAI: Copy, Steal, Paste

#61

Post by candiddevmike »

theamk wrote: post #40
Casteil wrote: post #26

Earlier quoted context omitted.

In a just world, royalties would be paid to those whose content AIs were trained on. In reality, the most we'll likely see is a few people who successfully sue for their small piece of the pie, while the big companies carry on like nothing happened - further accelerating the transfer of wealth right to the top.

In a just world, publishers would be able to opt-out of AI training, and AI companies would have to respect this. Then, if they still want the material, they can approach publishers and buy the content. (Smaller publishers might make a "content pool" so they don't have to negotiate individually) (This is how current search engines work, btw.. and that's why no one complains about them crawling content)

I'd go even further and say all content is by default "opted-out" from training unless explicitly allowed.

Re: OpenAI: Copy, Steal, Paste

#62

Post by corethree »

logiduck wrote: post #36
corethree wrote: post #14

Technically it's not a copy. And nothing was stolen. It's a best fit curve among a series of datapoints. The datapoint is a copy, if the best fit curve never touches the datapoint then it's technically not a copy. The difficult part is the technicality here is legal and ethical from any standpoint. The high level ramifications are a bit unfair in the sense that yes the data is being used to create an AI that can repl…

Copyright laws do not support your argument. There have been many cases in music where the offending song was forced to pay because it was "close enough" to the curve but not touching it.

True, but most points on the curve aren't close to Any data point. That means most of the output of ML is completely original. Let's use a simplified example of a straight line between two datapoints. Example:

   Point A ------|----------------------------------------|------Point B
For a line segment (above) between two points, most of the line does not approach the vicinity of Either point (the boundary of closeness for points A and B is demarcated figuratively with a pipe "|" if the line segment is between the pipe and the point it is too close, if it is not then it is an original work). This intuition still applies even if the line only moves close to the point and does not touch either point. Basically the output of ML is by majority not even close to a copy as most of the curve is far from any point.

The only way for most of the line to be a copy is if the data cluster is so close and similar that the data itself is mostly a chain of "similar" copies. Not sure if you're catching my meaning here. Example:

   Point A --|--|-- Point B
Above A and B cross their own thresholds and are essentially "close enough" copied data points. The left Pipe is the threshold for B and the right pipe is the threshold for A. As a result the entire line between the two points must be a an illegal "close enough" copy as well.

If such data is used it means existing data is in violation of copywrite law already. The logical implication is this:

For most of the results of ML to be a technical "close enough" copy of the datapoints, you must also admit that most of your data contained "close enough" copies as well.

As a side note this kind of thing can be useful for defining a quantitative measure of what close enough even means as we can certainly define a numeric threshold between close enough and not close enough for copywrite law.

Re: OpenAI: Copy, Steal, Paste

#64

Post by denebula »

cebert wrote: post #23
ipython wrote: post #2

Can't agree more. These AI systems have been built on the back of free labor for decades. We will look back on this and wonder how we essentially subsidized these behemoth corporations, then allowed them to extract subscription fees back from the very people it took the labor from. Don't get me wrong, there are genuine innovations in AI and ML. But, on the same token, you can't have ChatGPT without content.

> These AI systems have been built on the back of free labor for decades. Do you repay publishers for information you’ve summarized or new insights you’ve gained after consuming their content? How is it different when AI does the same thing?

False equivalence. You stated how its different in the premise on top of that

Re: OpenAI: Copy, Steal, Paste

#66

Post by yokem55 »

theamk wrote: post #46
yokem55 wrote: post #44

Earlier quoted context omitted.

> I think that from a legal standpoint, the actual technical means by which something was accomplished doesn't matter if the process as a whole is effectively copyright infringement. Which is why when the user of the model prompts for something infringing, and is successful at getting close to verbatim output (because the prompt was too constraining, becuase the work is overrepresented in the training) it is that par…

TensorFlow is also perfectly legal to develop and distribute, and no one contests this. People object to specific artifact, "model weights", which were produced using copyrighted works at the input, and can be used to reproduce those same copyrighted works back. In bittorrent analogy, people want to shut down specific pirate trackers and the pirate bay website.

From the above EFF article:

> First, a derivative work still has to be “substantially similar” to the original in order to be infringing. If the original is transformed or abridged or adapted to such an extent that this is no longer true, then it’s not a derivative work. A 10-line summary of a 15,000-line epic isn’t a derivative work, and neither are most summaries of books that people make in order to describe those copyrighted works to others.

The statistics generated about the works entered as input, do not resemble the original works. Nor can those statistics on their own reproduce the original work. At most they are brief mathematical summaries of the work. And it's only after combining those stats with the stats of billions of other works (which is its own creative process to determine the best statistical methodologies to achieve that combination) that anything intelligble can be produced in the output stage.

Re: OpenAI: Copy, Steal, Paste

#68

Post by Scion9066 »

theamk wrote: post #46
yokem55 wrote: post #44

Earlier quoted context omitted.

> I think that from a legal standpoint, the actual technical means by which something was accomplished doesn't matter if the process as a whole is effectively copyright infringement. Which is why when the user of the model prompts for something infringing, and is successful at getting close to verbatim output (because the prompt was too constraining, becuase the work is overrepresented in the training) it is that par…

TensorFlow is also perfectly legal to develop and distribute, and no one contests this. People object to specific artifact, "model weights", which were produced using copyrighted works at the input, and can be used to reproduce those same copyrighted works back. In bittorrent analogy, people want to shut down specific pirate trackers and the pirate bay website.

One thing that I think people forget about is that the prompt used when "reproduc[ing] those same copyrighted works" is also a part of why it spits out similar things. It's not just the model doing it. A traditional artist can be prompted to recreate a copyrighted work in much the same way with the right prompts.

Re: OpenAI: Copy, Steal, Paste

#69

Post by jchw »

yokem55 wrote: post #66
theamk wrote: post #46

Earlier quoted context omitted.

TensorFlow is also perfectly legal to develop and distribute, and no one contests this. People object to specific artifact, "model weights", which were produced using copyrighted works at the input, and can be used to reproduce those same copyrighted works back. In bittorrent analogy, people want to shut down specific pirate trackers and the pirate bay website.

From the above EFF article: > First, a derivative work still has to be “substantially similar” to the original in order to be infringing. If the original is transformed or abridged or adapted to such an extent that this is no longer true, then it’s not a derivative work. A 10-line summary of a 15,000-line epic isn’t a derivative work, and neither are most summaries of books that people make in order to describe those…

I think the case for Stable Diffusion in general is not too bad, however EFF tempers their optimism when it comes to cases where the model may actually memorize the inputs:

> To sum up: a diffusion model can, in rare circumstances, generate images that resemble elements of the training data. De-duplication can substantially reduce the risk of this occurring. But the strongest copyright suit against a diffusion-based AI art generator would likely be one brought by the holder of the copyright in an image that subsequently was actually reproduced this way.

EFF's position seems to be (to which I personally agree, FWIW) that Stable Diffusion almost certainly does not run afoul of at least the vast majority of copyright holders of data it was trained from.

> The statistics generated about the works entered as input, do not resemble the original works. Nor can those statistics on their own reproduce the original work. At most they are brief mathematical summaries of the work.

Of course, this needs a lot of qualification. Compression and intelligence are generally considered to be related, and indeed, compression also works on statistical analysis (like entropy coding a la Huffman, or frequency analysis via Fourier transforms). Granted, compression algorithms are designed to reproduce their input verbatim--it's the entire point. But I think ML weights may exist somewhere "in the middle" so to speak; depending on the model architecture and how it's trained, it may be more or less literally like compression. Vastly overfit models are very much like compression, whereas large generalized models like Stable Diffusion are pretty far away and yes, generally can't reproduce inputs verbatim. (However: I suspect many LoRA models are quite overfit and may not be in the same boat.)

However, that's just for image generation. I feel like LLMs and text generation are an entirely different ballgame, and given that we can't actually inspect the model weights in the case of GPT4, the best we can really do to surmise what's going on is to see how badly it seems to overfit its training data.

I am unconvinced that this matter is settled as a whole, although I do think the EFF article presents a good overview of the case regarding Stable Diffusion and it does coincide pretty closely with what I actually believe. But this article is about large language models, which may legitimately be a completely different ball game.

Re: OpenAI: Copy, Steal, Paste

#70

Post by JoshuaRogers »

Kiro wrote: post #16
jokethrowaway wrote: post #4

Hard disagree. The whole concept of copyright is unnatural and technology evolution is showing its limits. In the same way a writer reads books and use that knowledge to create more, an ai model can do the same.

Imagine getting downvoted for this on a forum called Hacker News. Embarrassing that we have become copyright zealots.

That might be misunderstanding the role of copyright as it applies to the worlds we live in.

Many here see copyright as a critical tool necessary for the existence of copy-left licenses, licenses that we use to make sure that what we write stays open in a way that others can benefit from.

We rely on copyright to ensure that our code stays “Free as in speech” not simply “Free as in beer”. Simply accepting no form of creator control is at odds with our desire to make sure that the things we produce stay open for others to use since there is nearly no incentive for the next person to keep our code open.

Post reply on HN