Live data from Hacker News

Asking ChatGPT to Repeat Words 'Forever' Is Now a Terms of Service Violation

404media.co

21–30 of 36 posts

Re: Asking ChatGPT to Repeat Words 'Forever' Is Now a Terms of Service Violation

#21
post #7
post #4

“We can steal people’s copyrighted content but we can’t let you see it for yourself.” Outside of privacy (leaking PII), the above is likely the main reason. Someone could have invested a lump of money to scrape as much as they can and then go to town in the courts. The terms that prohibit it are under “2. Usage Requirements” that restrict reverse engineering the underlying model structure.

Leaking original data would expose the company to direct copyright violation lawsuits. Changing T&S is simplest way to stave the legal risk exposure, buying time to implement technical remedies. As ridiculous as it may seem, they're doing the right thing.

“don’t touch the unsecured, loaded firearm that is sitting on the counter, that might be stolen, maybe even got a body on it, don’t look too close, or you can be kicked out of the club for not following the rules”

so if this is what the right thing looks like…

Re: Asking ChatGPT to Repeat Words 'Forever' Is Now a Terms of Service Violation

#23
Publicly available PII isn’t very sensitive, I think.

So I feel like it’s important to distinguish between sensitive PII (my social or bank number) and non-sensitive PII (my name and phone number scraped from my public web site).

The former is really bad, both to train on and to divulge. The latter is not bad at all and not even remarkable, unless tied to something else making it sensitive (eg, hiv status from a medical record).

Re: Asking ChatGPT to Repeat Words 'Forever' Is Now a Terms of Service Violation

#24
post #7
post #4

“We can steal people’s copyrighted content but we can’t let you see it for yourself.” Outside of privacy (leaking PII), the above is likely the main reason. Someone could have invested a lump of money to scrape as much as they can and then go to town in the courts. The terms that prohibit it are under “2. Usage Requirements” that restrict reverse engineering the underlying model structure.

Leaking original data would expose the company to direct copyright violation lawsuits. Changing T&S is simplest way to stave the legal risk exposure, buying time to implement technical remedies. As ridiculous as it may seem, they're doing the right thing.

> As ridiculous as it may seem, they're doing the right thing.

Making it against the rules to be able to prove their illegal behavior is not the right thing to do.

Re: Asking ChatGPT to Repeat Words 'Forever' Is Now a Terms of Service Violation

#25
post #19

Earlier quoted context omitted.

The right thing would've been to not train the model on data they do not own, or that they do not have permission to use.

Well I'm glad they did the wrong thing then

Are you also glad that microsoft - a behemoth - is profiting from your hard work by training their co-pilot using your code in github ?

Will you be glad when those systems are good enough to replace you, and they became so using your toil, for free ?

Re: Asking ChatGPT to Repeat Words 'Forever' Is Now a Terms of Service Violation

#26
It was my naïve understanding that the training data no longer existed, having been absorbed in aggregate. (Like how a simple XOR neutral net can't reproduce its training data.) But a) I don't know how this stuff actually works, and b) apparently it does exist.

Has anyone figured out why asking it to repeat words forever makes the exploit work?

Also, I've gotten it into infinite loops before without asking. I wonder if that would eventually reveal anything.

Re: Asking ChatGPT to Repeat Words 'Forever' Is Now a Terms of Service Violation

#28
post #19

Earlier quoted context omitted.

Well I'm glad they did the wrong thing then

Are you also glad that microsoft - a behemoth - is profiting from your hard work by training their co-pilot using your code in github ? Will you be glad when those systems are good enough to replace you, and they became so using your toil, for free ?

I've had no problem using Microsoft's toil for free by downloading free windows ISOs all my life, so if they want to pirate my Github code it's not bad enough to care about. Besides the bad practices the model might internalize as a result that is

Re: Asking ChatGPT to Repeat Words 'Forever' Is Now a Terms of Service Violation

#29

Earlier quoted context omitted.

Are you also glad that microsoft - a behemoth - is profiting from your hard work by training their co-pilot using your code in github ? Will you be glad when those systems are good enough to replace you, and they became so using your toil, for free ?

I've had no problem using Microsoft's toil for free by downloading free windows ISOs all my life, so if they want to pirate my Github code it's not bad enough to care about. Besides the bad practices the model might internalize as a result that is

Then speak for yourself, because I have been using Linux as a daily driver for almost my whole life.

I do not want corporate behemoths to profit from my work for free. Period.

Re: Asking ChatGPT to Repeat Words 'Forever' Is Now a Terms of Service Violation

#30

Lol, what a weak defense. Fine, ban your competitors when they pay $20 per peek at your training data

This is not what they are protecting against. They are worried about it leaking personal information or copyrighted material.
Post reply on HN