Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Not all tokens are meant to be forgotten

arxiv.org

Post reply on HN
Print view View original

11–20 of 27 posts

Re: Not all tokens are meant to be forgotten

#11

Post by genewitch »

lou1306 wrote: post #10
lmm wrote: post #6

Earlier quoted context omitted.

Knowing more might not improve your accuracy but it's not going to harm it. Forcibly forgetting true parts of your knowledge seems far more likely to have unintended consequences.

One obvious consequence: the model might still produce copyright infringement because it thinks its creative ideas are novel.

If the copyrighted content is not in the training data, and I mean explicitly, and the AI produces a copyrighted output, I'd argue it's a clean room re-implementation, and also it ought devalue the original work, moreso if the work is more recent. Maybe.

I get that "first to publish" matters to a lot of people, but, say 5 unrelated people are writing unique screenplays about a series of events that seems important to them or culture or whatever; if they all come up with very similar plots and locations and scenes, it just means that the idea is more obvious than non-obvious.

Please, argue. I haven't fully reconciled a lot of this to myself, but off the cuff this'll do.

The logic being - if an AI without taint produces some other work, that work drew on the same information the model did, and came to the same "conclusion" - which means with a time machine, you could wipe the LLM, go back to the period of the original work, train the LLM, and produce the work contemporaneous to the original. Hope that made sense.

Re: Not all tokens are meant to be forgotten

#12

Post by lmm »

genewitch wrote: post #11
lou1306 wrote: post #10

Earlier quoted context omitted.

One obvious consequence: the model might still produce copyright infringement because it thinks its creative ideas are novel.

If the copyrighted content is not in the training data, and I mean explicitly, and the AI produces a copyrighted output, I'd argue it's a clean room re-implementation, and also it ought devalue the original work, moreso if the work is more recent. Maybe. I get that "first to publish" matters to a lot of people, but, say 5 unrelated people are writing unique screenplays about a series of events that seems important to…

> If the copyrighted content is not in the training data, and I mean explicitly, and the AI produces a copyrighted output, I'd argue it's a clean room re-implementation

You can't claim it's a clean room without actually doing the legwork of making a clean room. Not including the copyrighted work verbatim isn't enough, you would need to show that the AI hadn't seen anything derived from that copyrighted work, or that it had seen only non-copyrightable pieces.

Re: Not all tokens are meant to be forgotten

#13

Post by lynx97 »

johnjreiser wrote: post #4
pixl97 wrote: post #2

> However, they tend to memorize unwanted information, such as private or copyrighted content, I mean humans don't forget copyrighted information. We just typically adjust it enough (some of the time) to avoid getting a copyright strike while modifying it in some way useful. We don't forget 'private' information either. We might not tell other people that information, but it still influences our thoughts. The idea of…

I'd counter with an anecdote; I had a colleague that boasted how he memorized a classmate's SSN in college and would greet him by SSN when seeing him years later. Is the goal of AI to replicate the entirety of the human experience (including social pressures, norms, and shame) or a tool to complement human decision making? While, yes, you can argue the slippery slope, it may be advantageous to flag certain training m…

The goal of AI is to make money. All the moralisation is very human, but also extremely naive.

BTW, I don't really understand what "social pressure" and "shame" has to do with your story? In my book, the person with a good memory isn't to blame. They're just demonstrating a security issue, which is a good thing.

Re: Not all tokens are meant to be forgotten

#14

Post by lou1306 »

genewitch wrote: post #11
lou1306 wrote: post #10

Earlier quoted context omitted.

One obvious consequence: the model might still produce copyright infringement because it thinks its creative ideas are novel.

If the copyrighted content is not in the training data, and I mean explicitly, and the AI produces a copyrighted output, I'd argue it's a clean room re-implementation, and also it ought devalue the original work, moreso if the work is more recent. Maybe. I get that "first to publish" matters to a lot of people, but, say 5 unrelated people are writing unique screenplays about a series of events that seems important to…

> The logic being - if an AI without taint produces some other work, that work drew on the same information the model did, and came to the same "conclusion" - which means with a time machine, you could wipe the LLM, go back to the period of the original work, train the LLM, and produce the work contemporaneous to the original. Hope that made sense.

This logic would immediately get shot down by an "Objection, speculation" in an actual litigation. Besides, the technicalities of how the work was produced don't really play a role in assessing infringement. PK Dick wrote "The man in the high castle" by extensively using the I Ching, but if I use it and recreate the novel by complete accident I would still be infringing.

By the way, I highly suggest Borges's "Pierre Menard, Author of the Quixote" as a great story on the topic of authorship :)

Re: Not all tokens are meant to be forgotten

#15

Post by falcor84 »

lynx97 wrote: post #13
johnjreiser wrote: post #4

Earlier quoted context omitted.

I'd counter with an anecdote; I had a colleague that boasted how he memorized a classmate's SSN in college and would greet him by SSN when seeing him years later. Is the goal of AI to replicate the entirety of the human experience (including social pressures, norms, and shame) or a tool to complement human decision making? While, yes, you can argue the slippery slope, it may be advantageous to flag certain training m…

The goal of AI is to make money. All the moralisation is very human, but also extremely naive. BTW, I don't really understand what "social pressure" and "shame" has to do with your story? In my book, the person with a good memory isn't to blame. They're just demonstrating a security issue, which is a good thing.

In that example, the mnemonist should be demonstrating the security issue to the government, and not to their friend. We have social taboos for this reason. As an extreme example, I wouldn't greet a person by their penis size after noticing it in the locker room - some information should still be considered private, regardless of how we came to obtain it.

Same with an LLM, when it got sensitive information in its weights, regardless of how it obtained it, I think we should apply pressure/shame/deletion/censorship (whatever you call it) to stop it from using that information in any future interactions.

Re: Not all tokens are meant to be forgotten

#16

Post by lynx97 »

falcor84 wrote: post #15
lynx97 wrote: post #13

Earlier quoted context omitted.

The goal of AI is to make money. All the moralisation is very human, but also extremely naive. BTW, I don't really understand what "social pressure" and "shame" has to do with your story? In my book, the person with a good memory isn't to blame. They're just demonstrating a security issue, which is a good thing.

In that example, the mnemonist should be demonstrating the security issue to the government, and not to their friend. We have social taboos for this reason. As an extreme example, I wouldn't greet a person by their penis size after noticing it in the locker room - some information should still be considered private, regardless of how we came to obtain it. Same with an LLM, when it got sensitive information in its wei…

I am probably too autistic to recognize remembering a personal datum as a taboo.

However, I am totally on your side regarding LLMs learning data they shouldn't have seen in the first place. IMO, we as a society are too much chicken to act on the current situation. Its plain insane that everyone and their dog knows that libgen has been used to train models, and the companies who did this experiencing NO consequences at all. After that, we shouldn't be surpised if things go downhill from here on.

Re: Not all tokens are meant to be forgotten

#18

Post by squidbeak »

pixl97 wrote: post #2

> However, they tend to memorize unwanted information, such as private or copyrighted content, I mean humans don't forget copyrighted information. We just typically adjust it enough (some of the time) to avoid getting a copyright strike while modifying it in some way useful. We don't forget 'private' information either. We might not tell other people that information, but it still influences our thoughts. The idea of…

I agree. As far as copyrighted and artistic works go, I've never fully understood what the objection is. If the work is being remixed not copied then it surely falls under fair use? Meanwhile, if it creates something new in an artist's style, it's only doing what talented imitators routinely do. There's the economic argument. But if that's accepted, then for fairness it would have to be extended to every other profession which stands to be wiped out by AI, which would be daft.

New works in familiar styles are something I can't wait for. The idea that the best Beethoven symphony hasn't been composed yet, or that the best Basquiat hasn't been painted yet, or that if the tech ever gets far enough, Game of Thrones might actually be done properly with the same actors, is a pretty mouthwatering prospect. Also styles we haven't discovered, that AI can anticipate. How's it to do that without a full understanding of culture? Hobbling the delight it could bring generally for the sake of protected classes will just make the tech less human and a lot less exciting.

Re: Not all tokens are meant to be forgotten

#19

Post by kgwgk »

genewitch wrote: post #11
lou1306 wrote: post #10

Earlier quoted context omitted.

One obvious consequence: the model might still produce copyright infringement because it thinks its creative ideas are novel.

If the copyrighted content is not in the training data, and I mean explicitly, and the AI produces a copyrighted output, I'd argue it's a clean room re-implementation, and also it ought devalue the original work, moreso if the work is more recent. Maybe. I get that "first to publish" matters to a lot of people, but, say 5 unrelated people are writing unique screenplays about a series of events that seems important to…

I highly suggest Borges's "Pierre Menard, Author of the Quixote" as a great story on the topic of authorship :)

Re: Not all tokens are meant to be forgotten

#20

Post by wizardforhire »

squidbeak wrote: post #18
pixl97 wrote: post #2

> However, they tend to memorize unwanted information, such as private or copyrighted content, I mean humans don't forget copyrighted information. We just typically adjust it enough (some of the time) to avoid getting a copyright strike while modifying it in some way useful. We don't forget 'private' information either. We might not tell other people that information, but it still influences our thoughts. The idea of…

I agree. As far as copyrighted and artistic works go, I've never fully understood what the objection is. If the work is being remixed not copied then it surely falls under fair use? Meanwhile, if it creates something new in an artist's style, it's only doing what talented imitators routinely do. There's the economic argument. But if that's accepted, then for fairness it would have to be extended to every other profes…

Mind if I ask a few questions? Whats your current address, dob, ssn or NINO or equivalent, your full legal name, mothers maiden name, fathers place of birth, mothers place of birth, country of origin, do you drive? Whats your license number? How about a bank? Could I have your account and routing number as well as the answers to any security questions? How about investments I’m gonna need your accounts and passwords for these as well…

> As far as copyrighted and artistic works go, I've never fully understood what the objection is … > But if that's accepted, then for fairness it would have to be extended to every other profession which stands to be wiped out by AI, which would be daft. … > Hobbling the delight it could bring generally for the sake of protected classes will just make the tech less human and a lot less exciting.

So let me get this straight, you want to ruin the livelihoods of everyone so you can have a fancier toy to play with?

When your life is ruined and can’t make a living you’ll have the answers you desire and understand the objections to why you can’t have fancier toys.

But heres the thing, and with the way the world is going atm, not being able to make a living is going to be the least of your and everyone else’s worries that feel the way you do if ya’ll get your way.

People don’t like having their livelihoods taken away, and when you threaten the livelihoods of their children… people tend towards violence.

I really wish there was a more polite way to put this. Alas what you’re proposing is all out war for what? A better game of thrones?

Post reply on HN