Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Judge said Meta illegally used books to build its AI

wired.com

Post reply on HN
Print view View original

211–220 of 352 posts

Re: Judge said Meta illegally used books to build its AI

#211

Post by Workaccount2 »

crystal_revenge wrote: post #202
meta_ai_x wrote: post #152

Earlier quoted context omitted.

completely different scenarios. A pirated movie is marketed/sold as a copy of something, which is not fair use. An LLM just remembers/get inspired by what it consumes

> An LLM just remembers/get inspired by what it consumes As mentioned in another comment, LLMs (and most popular machine learning algorithms) can be viewed, correctly, as compression algorithms which leverage lossy encoding + interpolation to force a kind of generalization. Your argument is that a video wouldn't count as pirated if the compression used for the pirated copy was lossy (or at least sufficiently lossy).…

They are in no way compression algorithms. They can be modeled like that in the same way you can model humans as lossy compression algorithms.

You would never use a human to backup your financial reports, but the human might be able to give a good overview. You would never use an LLM to backup your financial reports, but they might be able to give a good overview.

AI training data is disposable. There is nothing that could be called a compression algorithm that disposes all of the data you put into it. AI uses training data as examples of what the next token in a token sequence is. The examples are disposable reference points, not the model itself. That's how you get image models that are 20GB in size despite training on 20PB of data. It's 20PB of examples used to form the shape of a 20GB model. You could show it 5GB of training data or 500EB of training data and it would still be 20GB - because it is not a compression algo, it's a 20GB shape formed by external data.

Re: Judge said Meta illegally used books to build its AI

#212

Post by stevenAthompson »

leovander wrote: post #177
TheOtherHobbes wrote: post #122

Earlier quoted context omitted.

No transformation is needed. The point here is that book files have to be copied before they can be used for training. Copyright texts typically say something like "No unauthorised copying or transmission in any form (physical, electronic, etc.)" Individuals who torrented music and video files have been bankrupted for doing exactly this. The same laws should apply when a corporation downloads torrent files. What happ…

> have been bankrupted for doing exactly this. Only if they seeded the data and some other entity downloaded it, i.e. they hosted the data. In a previous article I believe it was called out that Meta was being a leecher (not seeding back what they downloaded). It's the hosting that gets you, not the act of downloading it.

I would like to expand on this, since it seems to be a common misunderstanding. Lets imagine a hypothetical situation where one friend loans a book to another, who then makes a copy of it.

The lender owns the book, and it is within his rights to loan it to whoever he wants. That is legal. Making this illegal would end libraries.

The borrower is well within his rights to accept the book, and as the current owner he is even allowed to make a copy of the book (see the famous TIVO case). Making this illegal would end backups and format/time shifting.

When the borrower returns the book, he keeps the copy. Oh no! Surely he must now become a criminal? Nope. Possessing an unauthorized copy is also not illegal, despite what many copyright holders would like you to believe. Making this illegal would also criminalize a lot of legitimate format/time shifting, again see the famous TIVO case.

If the borrower were to loan his homemade copy to someone else THEN it would finally become illegal.

Nothing about AI changes any of this.

Re: Judge said Meta illegally used books to build its AI

#213

Post by throwaway290 »

stevenAthompson wrote: post #179
xyzzy_plugh wrote: post #81

Earlier quoted context omitted.

Why does it have to be verbatim? Seriously, this I don't understand. If I produce a terrible shakycam recording of a film while sitting in a movie theater, it's not a verbatim copy, nor is it even necessarily representative of the original work -- muddied audio, audience sounds, cropped screen, backs of heads -- and yet it would be considered copyright infringement? How many times does one need to compress the JPEG b…

The purpose of copyright is it progress the arts and sciences. Not to guarantee profit. Guaranteeing profit is just the way we encourage people to progress the arts and sciences. That is why so called derivative works are allowed (and even encouraged). If copyrighted material is ingested, modified or enhanced to add value, and then regurgitated that is legal, whereas copying it without adding value is not legal. If d…

Derivative works are not generally allowed in many jurisdictions. Try releasing a cover song without clearing it first etc. Even using a recognizable sample will bite you

Derivative works are tolerated in some cases like some manga or fanfics but it is a gray area and whenever the author or publisher wants to pursue it it is their full right to do it. Many do pursue it

(You can get inspired by something, and this is where some arguments can happen if you get inspired too mmmm literally, but no one will say with a straight face that inspiration is a thing that happens to software)

Re: Judge said Meta illegally used books to build its AI

#214

Post by spwa4 »

triceratops wrote: post #183
Workaccount2 wrote: post #175

Earlier quoted context omitted.

That's not really true. Models train (in greatly simplified way) by being shown an excerpt and being told to guess the next token from the excerpt. They push around their weights until the token they output matches the next token in the excerpt. Then the excerpt is no longer needed. You can think of it like the article is loaded, the LLM plays this token guessing game through it, then the article is discarded. On the…

> by being shown an excerpt [of copyrighted material] How is this done? Are bits not written into RAM or disk? Are they not sent between machines in a training cluster? That's copying. > it is seemingly not far removed from how humans consume content Except that humans don't make full copies to RAM, or disk or paper.

> Except that humans don't make full copies to RAM, or disk or paper.

Images on your retina form exact copies.

They are scanned and translated into impulses that are then sent to a first set of "neural columns" - that's an exact copy.

This is then connected to the visual cortex by the two most high bandwidth links in the human body ("the optical nerve", there's 2 of them of course, always wondered why everybody insists on using the singular). Why would you have that high bandwidth link unless to create verbatim copies.

The way those columns are structured also very strongly suggests they make carbon copies, which they then make available on the "brain bridge" (which is probably at least vaguely similar to the "attention matrix" of a transformer). If it does work like that, that's also a verbatim copy.

The only way "humans don't make full copies to RAM" is that humans don't have separate RAM. The processing power is colocated with the processing, even on a microscopic level. You know, what everybody knows is the best way of doing things even in silicon, it's just incredibly impractical if you can't rebuild your circuit every time there's a slight change to the instructions your "computer" carries out (the brain is not a "Von Neumann architecture", except it kind of is when it regrows connections. But in the short term it isn't)

Re: Judge said Meta illegally used books to build its AI

#215

Post by __loam »

stevenAthompson wrote: post #179
xyzzy_plugh wrote: post #81

Earlier quoted context omitted.

Why does it have to be verbatim? Seriously, this I don't understand. If I produce a terrible shakycam recording of a film while sitting in a movie theater, it's not a verbatim copy, nor is it even necessarily representative of the original work -- muddied audio, audience sounds, cropped screen, backs of heads -- and yet it would be considered copyright infringement? How many times does one need to compress the JPEG b…

The purpose of copyright is it progress the arts and sciences. Not to guarantee profit. Guaranteeing profit is just the way we encourage people to progress the arts and sciences. That is why so called derivative works are allowed (and even encouraged). If copyrighted material is ingested, modified or enhanced to add value, and then regurgitated that is legal, whereas copying it without adding value is not legal. If d…

You do not understand fair use lol

Re: Judge said Meta illegally used books to build its AI

#216

Post by 3036e4 »

throwawaymaths wrote: post #195
triceratops wrote: post #108

Earlier quoted context omitted.

So Meta borrowed every book from a library and paid to obtain all of the movies and TV shows? They kept only one copy of every book at any time on their system? Because I'm certainly not allowed to photocopy a library book in its entirety. And I guarantee you a Netflix subscription doesn't allow me to keep a copy of a movie on my hard drive and use it for training man or machine.

> Because I'm certainly not allowed to photocopy a library book in its entirety. IANAL but that probably falls under fair use? You'll get in trouble if you photocopy the work and sell access to it.

Depends on where you live. In Sweden you can make a few copies of almost anything without violating copyright. There are a few exceptions. Copying entire books was added as an exception in 2005. You can still copy parts of a book. How large parts? I don't know, but I once asked for a copy from a library and they said that a few chapters was fine, so maybe that much (I am not a lawyer).

Re: Judge said Meta illegally used books to build its AI

#217

Post by amelius »

xyzzy_plugh wrote: post #81
thethimble wrote: post #69

Earlier quoted context omitted.

“Copy” is ambiguous here. Of course data is copied during training. That said, OP is referring to whether the resulting model is able to produce verbatim copies of the data.

Why does it have to be verbatim? Seriously, this I don't understand. If I produce a terrible shakycam recording of a film while sitting in a movie theater, it's not a verbatim copy, nor is it even necessarily representative of the original work -- muddied audio, audience sounds, cropped screen, backs of heads -- and yet it would be considered copyright infringement? How many times does one need to compress the JPEG b…

> I'm legitimately curious what the test is here.

The test is if a judge says it is fair use, nothing else.

The judge will take into account the human factor in this matter, e.g. things like who did the actual work, and who just used an algorithm (which is not the hard part anymore, code can be obtained on the internet for free). And we all know that DL is nowhere without huge amounts data.

Re: Judge said Meta illegally used books to build its AI

#218

Post by moregrist »

stevenAthompson wrote: post #179
xyzzy_plugh wrote: post #81

Earlier quoted context omitted.

Why does it have to be verbatim? Seriously, this I don't understand. If I produce a terrible shakycam recording of a film while sitting in a movie theater, it's not a verbatim copy, nor is it even necessarily representative of the original work -- muddied audio, audience sounds, cropped screen, backs of heads -- and yet it would be considered copyright infringement? How many times does one need to compress the JPEG b…

The purpose of copyright is it progress the arts and sciences. Not to guarantee profit. Guaranteeing profit is just the way we encourage people to progress the arts and sciences. That is why so called derivative works are allowed (and even encouraged). If copyrighted material is ingested, modified or enhanced to add value, and then regurgitated that is legal, whereas copying it without adding value is not legal. If d…

> That is why so called derivative works are allowed (and even encouraged).

Derivative works are not given a free pass from the normal constraints of copyright. You cannot legally publish books in the universe of A Song of Ice and Fire without permission from the author (and often publisher), calling them “derivative works.”

It’s why fan fiction is such a gray area for copyright and why some publishers have historically squashed it hard.

The exceptions for this are typically fair use, which requires multi-factor analysis by the judiciary and is typically decided on a case-by-case basis.

Re: Judge said Meta illegally used books to build its AI

#219

Post by crystal_revenge »

Workaccount2 wrote: post #211
crystal_revenge wrote: post #202

Earlier quoted context omitted.

> An LLM just remembers/get inspired by what it consumes As mentioned in another comment, LLMs (and most popular machine learning algorithms) can be viewed, correctly, as compression algorithms which leverage lossy encoding + interpolation to force a kind of generalization. Your argument is that a video wouldn't count as pirated if the compression used for the pirated copy was lossy (or at least sufficiently lossy).…

They are in no way compression algorithms. They can be modeled like that in the same way you can model humans as lossy compression algorithms. You would never use a human to backup your financial reports, but the human might be able to give a good overview. You would never use an LLM to backup your financial reports, but they might be able to give a good overview. AI training data is disposable. There is nothing that…

> They are in no way compression algorithms.

I'm sorry, but this a fundamentally incorrect view of machine learning (including, but not limited to transformers).

From an information theoretic perspective the two are essentially identical with the exception that standard compression algorithms do not have a proper "loss" function other than just trying to minimize reconstruction loss with the resulting compression size.

Here's a link to the section on the Wikipedia for more information if you'd like [0]. MacKay's Information Theory, Inference and Learning Algorithms is the standard full text treatment of this topic [1]. Ted Chiang's article "ChatGPT is a Blurry JPEG of the web" is pretty good "pop sci" exploration of this topic if you don't want to get too into the mathematics [2].

0. https://en.wikipedia.org/wiki/Data_compression#Machine_learn...

1. https://www.inference.org.uk/itprnn/book.pdf

2. https://www.newyorker.com/tech/annals-of-technology/chatgpt-...

Re: Judge said Meta illegally used books to build its AI

#220

Post by codedokode »

It's typical double standards policy: Google and Github remove links to pirated material (and pirated material itself) so that ordinary folks cannot download it for free, but when Zuckerberg downloads gigabytes of pirated material without paying, it's ok. The legal system doesn't want to put an ordinary folk and Zuckerberg at the same level.

Also I read that ordinary folks have been arrested for filming in the cinema even if they did not redistribute the video (due to being arrested). Again, it is unfair why they get arrested and Zuckerberg doesn't.

Post reply on HN