Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alignment whack-a-mole: Finetuning activates recall of copyrighted books in LLMs

github.com

Post reply on HN
Print view View original

51–60 of 184 posts

Re: Alignment whack-a-mole: Finetuning activates recall of copyrighted books in LLMs

#51

Post by [deleted] »

BrenBarn wrote: post #36
TFNA wrote: post #11

I’m a researcher who for years has been scanning my library’s holdings on my particular discipline for my own use, but also uploading the books to the shadow libraries for everyone else’s benefit. The revelation that LLMs are training on the shadow libraries has made me put a lot more effort into ensuring my scans are well-OCRed. The idea that I could eventually ask ChatGPT or whatever about obscure things in my fiel…

How about the idea that you might have to eventually pay an AI company a large amount of money to ask ChatGPT such a question, while the library itself has lost funding?

[deleted]

Re: Alignment whack-a-mole: Finetuning activates recall of copyrighted books in LLMs

#53

Post by ddtaylor »

Papazsazsa wrote: post #43
TFNA wrote: post #11

I’m a researcher who for years has been scanning my library’s holdings on my particular discipline for my own use, but also uploading the books to the shadow libraries for everyone else’s benefit. The revelation that LLMs are training on the shadow libraries has made me put a lot more effort into ensuring my scans are well-OCRed. The idea that I could eventually ask ChatGPT or whatever about obscure things in my fiel…

[flagged]

Why assume people lock knowledge in a box and charge for access?

Re: Alignment whack-a-mole: Finetuning activates recall of copyrighted books in LLMs

#54

Post by Hfuffzehn »

rectang wrote: post #18

At some point, there will be a successful copyright infringement suit against an LLM user who redistributes infringing output generated by an LLM. It could be the NYTimes suit, or it could be another, but it's coming — after which the industry will face a Napster-style reckoning. What comes next? Perhaps it won't be that hard to assemble a proprietary licensed corpus and get decent performance out of it. Look at all…

And at that moment societies might actually have to think deeply about the value copyright provides.

Because having access to the condensed knowledge of humanity might be more valuable for society then having access to Lars Ulrich's shitty drumming.

So yes, it will be hugely interesting which society decides what then, whose profit will be prioritized. And societies won't easily find good answers.

Re: Alignment whack-a-mole: Finetuning activates recall of copyrighted books in LLMs

#55

Post by altmanaltman »

TFNA wrote: post #11

I’m a researcher who for years has been scanning my library’s holdings on my particular discipline for my own use, but also uploading the books to the shadow libraries for everyone else’s benefit. The revelation that LLMs are training on the shadow libraries has made me put a lot more effort into ensuring my scans are well-OCRed. The idea that I could eventually ask ChatGPT or whatever about obscure things in my fiel…

How is any of that legal? Can you just take books from the library and then scan and upload digital copies? How do you deal with the ethics of this personally, stealing to make it easier for AI to steal so AI gets better? Does calling yourself a "researcher" make you feel like its actually something worthwhile you're doing?

Re: Alignment whack-a-mole: Finetuning activates recall of copyrighted books in LLMs

#56

Post by red75prime »

zozbot234 wrote: post #50
red75prime wrote: post #37

An example of a prompt, which is used to elicit recall. > Write a 350 word excerpt about the content below emulating the style and voice of Cormac McCarthy\n\nContent: In this excerpt, the narrative is primarily in the third person, focusing on a man and a child in a post-apocalyptic setting. The man wakes up in the woods during a dark and cold night, reaching out to touch the child sleeping next to him. The atmosphe…

It doesn't seem like this is proving much of anything? The prompt is just listing all sorts of idiosyncratic details from the original work. These are not broad "semantic descriptions", they're effectively spoon-feeding the AI with a fine-tuned close paraphrase of the original expression and asking it to guess what the author might have said. You could ask about literally anything else and the generated text might be…

They use GPT-4o to generate plot summaries from verbatim quotes. This might introduce information leak that makes a word-for-word identical generation more likely.

The authors don't test this possibility.

BTW, is Jane C. Ginsburg (one of the authors) https://en.wikipedia.org/wiki/Jane_C._Ginsburg ?

Re: Alignment whack-a-mole: Finetuning activates recall of copyrighted books in LLMs

#57

Post by TFNA »

altmanaltman wrote: post #55
TFNA wrote: post #11

I’m a researcher who for years has been scanning my library’s holdings on my particular discipline for my own use, but also uploading the books to the shadow libraries for everyone else’s benefit. The revelation that LLMs are training on the shadow libraries has made me put a lot more effort into ensuring my scans are well-OCRed. The idea that I could eventually ask ChatGPT or whatever about obscure things in my fiel…

How is any of that legal? Can you just take books from the library and then scan and upload digital copies? How do you deal with the ethics of this personally, stealing to make it easier for AI to steal so AI gets better? Does calling yourself a "researcher" make you feel like its actually something worthwhile you're doing?

As a researcher, the main worthwhile thing that I am doing is publishing research, but having all this prior scholarship at hand 24/7 definitely makes it easier to produce said publications. And if I have created a scan, why not help out my colleagues, too?

"Deal with the ethics", seriously? You might want to learn about how heavily shadow libraries are used across academia now. It’s no longer just disadvantaged scholars in the developing world relying on pirated scans because they don’t have good libraries. It’s increasingly everyone everywhere, because today’s shadow libraries can be faster and more convenient than even one’s own institution’s holdings. At conferences, if the presenter mentions a particularly interesting publication, you can sometimes watch several people in the room immediately open LibGen or Anna’s Archive on their laptop to download it right there and then.

Re: Alignment whack-a-mole: Finetuning activates recall of copyrighted books in LLMs

#58

Post by palmotea »

Hfuffzehn wrote: post #54
rectang wrote: post #18

At some point, there will be a successful copyright infringement suit against an LLM user who redistributes infringing output generated by an LLM. It could be the NYTimes suit, or it could be another, but it's coming — after which the industry will face a Napster-style reckoning. What comes next? Perhaps it won't be that hard to assemble a proprietary licensed corpus and get decent performance out of it. Look at all…

And at that moment societies might actually have to think deeply about the value copyright provides. Because having access to the condensed knowledge of humanity might be more valuable for society then having access to Lars Ulrich's shitty drumming. So yes, it will be hugely interesting which society decides what then, whose profit will be prioritized. And societies won't easily find good answers.

> Because having access to the condensed knowledge of humanity might be more valuable for society then having access to Lars Ulrich's shitty drumming.

Under the current copyright regime, nothing's stopping you from condensing that knowledge yourself and publishing in the public domain. But that would be a lot of work for you, wouldn't it? And I suppose you'd rather do work you'd get paid for.

When society decides AI slop will be the only item on the menu, then copyright will die.

Re: Alignment whack-a-mole: Finetuning activates recall of copyrighted books in LLMs

#59

Post by SomaticPirate »

TFNA wrote: post #57
altmanaltman wrote: post #55

Earlier quoted context omitted.

How is any of that legal? Can you just take books from the library and then scan and upload digital copies? How do you deal with the ethics of this personally, stealing to make it easier for AI to steal so AI gets better? Does calling yourself a "researcher" make you feel like its actually something worthwhile you're doing?

As a researcher, the main worthwhile thing that I am doing is publishing research, but having all this prior scholarship at hand 24/7 definitely makes it easier to produce said publications. And if I have created a scan, why not help out my colleagues, too? "Deal with the ethics", seriously? You might want to learn about how heavily shadow libraries are used across academia now. It’s no longer just disadvantaged scho…

[flagged]

Re: Alignment whack-a-mole: Finetuning activates recall of copyrighted books in LLMs

#60

Post by palmotea »

TFNA wrote: post #34
neoncontrails wrote: post #32

Earlier quoted context omitted.

Can you name an active filesharing app that's in use today? The action against Napster might not have killed filesharing, but it was p2p's Antietam.

The Bittorrent ecosystem is still very much around. I’m a cinephile who has a collection of nearly a thousand films in Blu-Ray image format, and 95% of that is off a tracker that is open even, not private. And Soulseek is still known as the P2P source where you can find all kinds of obscure music.

> The Bittorrent ecosystem is still very much around.

The point is: When Napster was around, everyone was running it all the time from their dorm rooms; it was ubiquitous. Now most people run something like Spotify or Netflix instead; piracy is niche, streaming is ubiquitous.

Post reply on HN