Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

understandingai.org

Post reply on HN
Print view View original

131–140 of 326 posts

Re: Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

#131

Post by psychoslave »

OtherShrezzing wrote: post #68
paxys wrote: post #64

As an experiment I searched Google for "harry potter and the sorcerer's stone text": - the first result is a pdf of the full book - the second result is a txt of the full book - the third result is a pdf of the complete harry potter collection - the fourth result is a txt of the full book (hosted on github funny enough) Further down there are similar copies from the internet archive and dozens of other sites. All in…

I think the argument is less about piracy and more that the model(s output) is a derivative work of Harry Potter, and the rights holder should be paid accordingly when it’s reproduced.

The main issue on an economical point of view is that copyright is not the framework we need for social justice and everyone florishing by enjoying pre-existing treasures of human heritage and fairly contributing back.

There is no morale and justice ground to leverage on when the system is designed to create wealth bottleneck toward a few recipients.

Harry Potter is a great piece of artistic work, and it's nice that her author could make her way out of a precarious position. But not having anyone in such a situation in the first place would be what a great society should strive to produce.

Rowling already received more than all she needs to thrive I guess. I'm confident that there are plenty of other talented authors out there that will never have such a broad avenue of attention grabbing, which is okay. But that they are stuck in terrible economical situations is not okay.

The copyright loto, or the startup loto are not that much different than the standard loto, they just put so much pression on the player that they get stuck in the narrative that merit for hard efforts is the key component for the gained wealth.

Re: Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

#132

Post by nashashmi »

abtinf wrote: post #90
paxys wrote: post #64

As an experiment I searched Google for "harry potter and the sorcerer's stone text": - the first result is a pdf of the full book - the second result is a txt of the full book - the third result is a pdf of the complete harry potter collection - the fourth result is a txt of the full book (hosted on github funny enough) Further down there are similar copies from the internet archive and dozens of other sites. All in…

You really don't see the difference between Google indexing the content of third parties and directly hosting/distributing the content itself?

The way I see it is that an LLM took search results and outputted that info directly. Besides, I think that if an LLM was able to reproduce 42%, assuming that it is not continuous, I would say that is fair use.

Re: Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

#133

Post by bakugo »

imgabe wrote: post #109
abtinf wrote: post #104

Earlier quoted context omitted.

Of course it is. It's just a form of compression. If I train an autoencoder on an image, and distribute the weights, that would obviously be the same as distributing the content. Just because the content is commingled with lots of other content doesn't make it disappear. Besides, where did the sections of text from the input works that show up in the output text come from? Divine inspiration? God whispering to the ma…

[flagged]

> Have you ever repeated a line from your favorite movie or TV show? Memorized a poem? Guess the rights holders better sue you for stealing their content by encoding it in your wetware neural network.

I see this absolute non-argument regurgitated ad infinitum in every single discussion on this topic, and at this point I can't help but wonder: doesn't it say more about the person who says it than anything else?

Do you really consider your own human speech no different than that of a computer algorithm doing a bunch of matrix operations and outputting numbers that then get turned into text? Do you truly believe ChatGPT deserves the same rights to freedom of speech as you do?

Re: Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

#135

Post by briffid »

Quotation is fair use in all sensible copyright system. An LLM will mostly be able to quote anything, and should be. Quotation is not derived work. LLMs are not stealing copyrighted work. They just show that Harry Potter is in English and a mostly logical story. If someone is stabbed, they will die in most stories, that's not copyrightable. If you have an engine that knows everything, it will be able to quote everything.

Re: Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

#136

Post by briffid »

sumeno wrote: post #101
ninetyninenine wrote: post #66

Earlier quoted context omitted.

> that exists in the LLM, is that you can't make copies and distribute your brain to billions of people. I can record myself reciting the full Harry Potter book then distribute it on YouTube. Could do the exact same thing with an LLM. The potential for distribution exists in both cases. Why is one illegal and the other not?

> I can record myself reciting the full Harry Potter book then distribute it on YouTube Not legally you can't. Both of your examples are copyright violations

Recording yourself is not a violation, only publishing on Youtube. Content generated with LLMs are not a violation. Publishing the content you generated might be.

Re: Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

#137

Post by imgabe »

bakugo wrote: post #133
imgabe wrote: post #109

Earlier quoted context omitted.

[flagged]

> Have you ever repeated a line from your favorite movie or TV show? Memorized a poem? Guess the rights holders better sue you for stealing their content by encoding it in your wetware neural network. I see this absolute non-argument regurgitated ad infinitum in every single discussion on this topic, and at this point I can't help but wonder: doesn't it say more about the person who says it than anything else? Do you…

Who said anything about freedom of speech? Nobody is claiming the LLM has free speech rights, which don't even apply to infringing copyright anyway. Freedom of speech doesn't give me the right to make copies of copyrighted works.

The question is whether the model weights constitute of copy of the work. I contend that they do not, or they did, than so do the analogous weights (reinforced neural pathways) in your brain, which is clearly absurd and is intended to demonstrate the absurdity of considering a probabilistic weighting that produces similar text to be a copy.

Re: Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

#138

Post by aschobel »

abtinf wrote: post #104
imgabe wrote: post #99

Earlier quoted context omitted.

Hosting model weights is not hosting / distributing the content.

Of course it is. It's just a form of compression. If I train an autoencoder on an image, and distribute the weights, that would obviously be the same as distributing the content. Just because the content is commingled with lots of other content doesn't make it disappear. Besides, where did the sections of text from the input works that show up in the output text come from? Divine inspiration? God whispering to the ma…

Indeed! It is a form of massive lossy compression.

> Llama 3 70B was trained on 15 trillion tokens

That's roughly a 200x "compression" ration; compared to 3-7x for tradtional lossless text compression like bzip and friends.

LLM don't just compress, they generalize. If they could only recite Harry Potter perfectly but couldn’t write code or explain math, they wouldn’t be very useful.

Re: Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

#139

Post by timeon »

paxys wrote: post #64

As an experiment I searched Google for "harry potter and the sorcerer's stone text": - the first result is a pdf of the full book - the second result is a txt of the full book - the third result is a pdf of the complete harry potter collection - the fourth result is a txt of the full book (hosted on github funny enough) Further down there are similar copies from the internet archive and dozens of other sites. All in…

Is this whataboutism?

Anyway, it is not the same. While one points you to pirated source on specific request, other use it to creating other content not just on direct request. As it was part of training data. Nihilists would then point out that 'people do the same' but they don't as we do not have same capabilities of processing the content.

Post reply on HN