121–130 of 326 posts
Many people could also produce text snippets from memory. I dispute that reading a book is a copyright violation. Copying and distributing a book, yes, but just reading it - no. If the book was obtained legitimately, letting an LLM read it is not an issue.
Earlier quoted context omitted.
Of course it is. It's just a form of compression. If I train an autoencoder on an image, and distribute the weights, that would obviously be the same as distributing the content. Just because the content is commingled with lots of other content doesn't make it disappear. Besides, where did the sections of text from the input works that show up in the output text come from? Divine inspiration? God whispering to the ma…
[flagged]
I'm surprised no one in the comments has mentioned overfitting. Perhaps this is too obvious but I think of it as a very clear bug in a model if it asserts something to be true because it has heard it once. I realize that training a model is not easy, but this is something that should've been caught before it was released. Either QA is sleeping on the job or they have intentionally released a model with serious flaws…
Earlier quoted context omitted.
All this study really says, is that models are really good at compressing the text of Harry Potter. You can't get Harry Potter out of it without prompting it with the missing bits - sure, impressively few bits, but is that surprising, considering how many references and fair use excerpts (like discussion of the story in public forums) it's seen? There's also the question of how many bits of originality there actually…
The alternate here is that Harry Potter is written with sentences that match the typical patterns of English and so, when you prompt with a part of the text, the LLM can complete it with above-random accuracy
Earlier quoted context omitted.
To me the primary difference between the potential "copy" that exists in your brain and a potential "copy" that exists in the LLM, is that you can't make copies and distribute your brain to billions of people. If you compressed a copy of HP as a .rar, you couldn't read that as is, but you could press a button and get HP out of it. To distribute that .rar would clearly be a copyright violation. Likewise, you can't rea…
> that exists in the LLM, is that you can't make copies and distribute your brain to billions of people. I can record myself reciting the full Harry Potter book then distribute it on YouTube. Could do the exact same thing with an LLM. The potential for distribution exists in both cases. Why is one illegal and the other not?
At this point you've created an entirely new copy in an audio/visual digital format and took the steps to make it available to the masses. This would almost certainly cross the line into violating copyright laws.
> Could do the exact same thing with an LLM. The potential for distribution exists in both cases. Why is one illegal and the other not?
To my knowledge, the legality of LLMs are still being tested in the courts, like in the NYT vs Microsoft/OpenAI lawsuit. But your video copy and distribution on YouTube would be much more similar to how LLMs are being used than your initial example of reading and memorizing HP just by yourself.
Earlier quoted context omitted.
We cannot get ride of it without finding a way to pay the creators that generate copyrighted works. I’m personally more in favor of significantly reducing the length of the copy right. I think 20-30 years is an interesting range. Artist get roughly a career length of time to profit off their creations, but there is much less incentive for major corporations to buy and horde IP.
We barely pay creators as it is for generating copyrighted works. Nearly every copywritten work is available on the internet, for free, right now . And creators are still getting paid, albeit poorly, but that's a constant throughout history.
I'm surprised no one in the comments has mentioned overfitting. Perhaps this is too obvious but I think of it as a very clear bug in a model if it asserts something to be true because it has heard it once. I realize that training a model is not easy, but this is something that should've been caught before it was released. Either QA is sleeping on the job or they have intentionally released a model with serious flaws…
They know. LLM is a novel compression format for text(holographic memory or whatever). The question is whether the rest of the world accept this technology as it is or not.
Earlier quoted context omitted.
The alternate here is that Harry Potter is written with sentences that match the typical patterns of English and so, when you prompt with a part of the text, the LLM can complete it with above-random accuracy
Anything that can tell you what the typical patterns of English is, is going to be a language model by definition.
EDIT Actually, on rereading, I see I replied to the wrong comment.
Earlier quoted context omitted.
[flagged]
[flagged]
You don't seem to be in a very good position to judge what is and is not obtuse.
Earlier quoted context omitted.
Even if it is recalling it 50 tokens at a time, the half of the book is in some sense in there, right?
I'd say no. More than half of as-yet unwritten books will be in there too, because I bet will will compress text of a freshly published book much better than 50% (and newer models could even compress new books to one fiftieth of their size, which is more like that 1 in 50 tokens suggests)