I do not think adding covert metadata (which this seems to be, however well-encoded) to output is a good addition to any program, regardless if it's a language model or something even more sinister. I am not so naive to believe OpenAI refuses to do it because of the goodness of their heart, but it's still a welcome benefit to me.
OpenAI won't watermark ChatGPT text because its users could get caught
21–30 of 84 posts
Re: OpenAI won't watermark ChatGPT text because its users could get caught
#22Remember when GPT was "too dangerous" to release into the world and unless you were twitter-famous you had to apply and wait for months to even get access? Times sure have changed
Did people say GPT from OpenAI was too dangerous to release? Or was it the fact that OpenAI has demonstrated how good LLMs can get and that bad actors can train their own, uncensored, unaligned LLMS to do "dangerous" things?
I knew some people from OpenAI at the time of GPT2 development, and indeed that was their rationale - dangerous and needs to be released responsibly. Never before could people produce so much spam and fake news at once - I think this was the first and main worry.
Re: OpenAI won't watermark ChatGPT text because its users could get caught
#23On a personal level, I'm happy about this. Having all personal info trackable is a tiring aspect of modern life? Remember when social media didn't zero out the exif on photos and anyone could easily grab a map to your family's home? On a society level? I'm stil ok with this. Watermarks are trivially removed by the motivated and unpleasant.
Its hard to remove this watermark, because the "watermark" is adjusting the probabilities of what tokens are generated, rather than just slapping "generated by chatgpt" over the top. You'd have to actually rewrite the text to remove it
Re: OpenAI won't watermark ChatGPT text because its users could get caught
#24On a personal level, I'm happy about this. Having all personal info trackable is a tiring aspect of modern life? Remember when social media didn't zero out the exif on photos and anyone could easily grab a map to your family's home? On a society level? I'm stil ok with this. Watermarks are trivially removed by the motivated and unpleasant.
> Watermarks are trivially removed by the motivated and unpleasant
I heard some chatter from OpenAI folks some 1-2 years ago that the way they'd watermark text is that rather than just using random numbers in the top-k/p sampling process, you have a pseudorandom sequence of numbers that either follows a pattern or you save them directly. This way you could fairly trivially build a tool that determines with very high accuracy whether or not a sequence of words has been generated by your model.
I think such a watermark can't be trivially removed unless you rewrite the text, or at least large portions of it.
Re: OpenAI won't watermark ChatGPT text because its users could get caught
#25Perhaps all documents should be watermarked “grammar improved by Microsoft Word” or “Animation effects provided by Keynote” or have films watermarked with “Automatic Color Correction provided by DaVinci.” It Takes a Village by Hillary Clinton: “Ghostwritten by Barbara Feinman.”
If we want to watermark GPT, fine — then let’s watermark absolutely everything not directly and personally created by the claimed creator. But we’re getting into interesting legal territory here — work for hire agreements would be in jeopardy because authorship of something under work for hire is owned by then company, not the contractor. There’s also a First Amendment issue — requiring companies and individuals to watermark creative works or disclose uncredited authors amounts to compelled speech that doesn’t serve a public interest high enough to provide an exception to First Amendment protections. The unintended (or perhaps subversively intended) consequences of requiring watermarks can be astounding. Journalists could potentially be compelled to reveal sources for instance because the content they’ve created was partially provided by someone else. It’s a stretch, but then again, the gymnastics courts and prosecutors routinely employ make such scenarios plausible (albeit unlikely.)
Re: OpenAI won't watermark ChatGPT text because its users could get caught
#26On a personal level, I'm happy about this. Having all personal info trackable is a tiring aspect of modern life? Remember when social media didn't zero out the exif on photos and anyone could easily grab a map to your family's home? On a society level? I'm stil ok with this. Watermarks are trivially removed by the motivated and unpleasant.
It doesn't seem trackable. And it may not be perfect, but that's no reason to discard it. At the moment, it's being used at a large scale to avoid homework by lazy students, which is nearly everyone. How much further do you want tech to erode education?
Not the parent, but I personally would want tech to erode education exactly to the level where students aren't asked any more to spend time on things that machines can perform trivially for us. Education should prepare us for the real world of today, rather than some make-belief role play version of a bureaucratic office from the late 19th century, where we don't have computers and the only way you have of affecting the world is by writing memos with a pencil.
Re: OpenAI won't watermark ChatGPT text because its users could get caught
#27Why not post the original article that The Verge has basically summarised? https://www.wsj.com/tech/ai/openai-tool-chatgpt-cheating-wri...
Because most people cannot read that article due to the paywall.
Re: OpenAI won't watermark ChatGPT text because its users could get caught
#28Earlier quoted context omitted.
I remember when GPT-2 was "too dangerous" to release. I am confused why people still take these clown claims seriously.
GPT-2 was never too dangerous to release, that's made up. OpenAI were trying to set a precedent to delay releases in anticipation of more dangerous models. This was and remains a good thing.
Maybe "too dangerous" is mainly a PR strategy to hype up the power of the tool.
It's either PR or religious ideology (or both if you drink your own Kool-Aid)
Re: OpenAI won't watermark ChatGPT text because its users could get caught
#29I think it's just too easy to fool it, as article says > But it says techniques like rewording with another model make it “trivial to circumvention by bad actors.”
Am I right that "bad actors" here refers to the actual paying users, i.e. us and our children?
Re: OpenAI won't watermark ChatGPT text because its users could get caught
#30On a personal level, I'm happy about this. Having all personal info trackable is a tiring aspect of modern life? Remember when social media didn't zero out the exif on photos and anyone could easily grab a map to your family's home? On a society level? I'm stil ok with this. Watermarks are trivially removed by the motivated and unpleasant.
I'm not sure what you are saying here. Do you really believe that internally OpenAI does not track and save important info based on your inputs creating a "persona"? This is any intelligence agencies wet dream.
The functionality that was introduced that would save info about you as a person to "improve" responses cross-chats basically made a whole profile out of you as a person.
Oh, and what a surprise - OpenAI directors since this year are ex-CIA or have very close connections to the agency.