Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI won't watermark ChatGPT text because its users could get caught

theverge.com

Post reply on HN
Print view View original

31–40 of 84 posts

Re: OpenAI won't watermark ChatGPT text because its users could get caught

#31

Post by inktrail »

CharlieDigital wrote: post #18
inktrail wrote: post #11

We solved this with a simple but novel solution that is doesn't break with rewording/paraphrasing. The problem we've run into is that teachers don't want to have the hard conversation when they get the evidence of cheating. And school leadership don't want to fix the issue, they want to put a chatbot on their resume. Sadly, I don't think OpenAI releasing this would have any effect fixing the problem. It's a problem w…

It's a problem with the people in education. Sounds absurd to reduce all of the thousands of school districts; millions of educators, K-12 teachers, professors, and administrators; and multitudes of viewpoints into one "ignorant" block of hapless Luddites.

We've had hundreds of conversations with different schools and teachers. It's definitely a generalization, but it's pretty disheartening how common these are the views.

Re: OpenAI won't watermark ChatGPT text because its users could get caught

#32

Post by lewhoo »

falcor84 wrote: post #29
methyl wrote: post #4

I think it's just too easy to fool it, as article says > But it says techniques like rewording with another model make it “trivial to circumvention by bad actors.”

Am I right that "bad actors" here refers to the actual paying users, i.e. us and our children?

If our children use it to cheat in school then I suppose yes, those who do so are bad actors.

Re: OpenAI won't watermark ChatGPT text because its users could get caught

#33

Post by xienze »

pona-a wrote: post #13

What we're seeing is a new profitable industry, with a tendency towards natural monopoly, refusing to act in the public good because its incentives conflict with it. At stake is the entire corpus of 21st century media, at risk of being drowned out in a tsunami of statistically unidentifiable spam, which has already begun and will only get worse, until nobody will even acknowledge the web post 2022 as worth reading or…

> Force all companies training LLMs to add some method of watermarking with a mean error rate below a set value.

How do you watermark plain text?

Re: OpenAI won't watermark ChatGPT text because its users could get caught

#34

Post by EGreg »

matusp wrote: post #9
wongarsu wrote: post #5

Remember when GPT was "too dangerous" to release into the world and unless you were twitter-famous you had to apply and wait for months to even get access? Times sure have changed

I remember when GPT-2 was "too dangerous" to release. I am confused why people still take these clown claims seriously.

Because the trend is the other way. Only a clown would claim GPT-2 is “too dangerous” too release, but not so for GPT-4 or GPT-5

Re: OpenAI won't watermark ChatGPT text because its users could get caught

#35

Post by EGreg »

kolinko wrote: post #22
aurareturn wrote: post #19

Earlier quoted context omitted.

Did people say GPT from OpenAI was too dangerous to release? Or was it the fact that OpenAI has demonstrated how good LLMs can get and that bad actors can train their own, uncensored, unaligned LLMS to do "dangerous" things?

I knew some people from OpenAI at the time of GPT2 development, and indeed that was their rationale - dangerous and needs to be released responsibly. Never before could people produce so much spam and fake news at once - I think this was the first and main worry.

was?

I guess it’s less of a worry now? Why?

Re: OpenAI won't watermark ChatGPT text because its users could get caught

#36

Post by creesch »

methyl wrote: post #4

I think it's just too easy to fool it, as article says > But it says techniques like rewording with another model make it “trivial to circumvention by bad actors.”

That and potentially asking it specifically to not follow certain patterns. I also wonder if the false positive rate in a college/school setting will be higher. Because to me, it seems that these models are often trained on college papers.

Because I see a lot of the patterns you already see on those. Specifically, paragraphs that start with conjunctive adverbs and phrases. Things like; however, furthermore, moreover, in summary, in conclusion, etc.

Besides, smart students that are not utterly lazy will be able to work around it anyway. They'll let chatGPT (or whatever LLM) turn out an entire paper for the contents, then rewrite it themselves.

So a tool like this will only catch the most obvious cases. Meaning that in the end it only battles a symptom by effectively sweeping it under the carpet.

Re: OpenAI won't watermark ChatGPT text because its users could get caught

#37

Post by xena »

CharlieDigital wrote: post #18
inktrail wrote: post #11

We solved this with a simple but novel solution that is doesn't break with rewording/paraphrasing. The problem we've run into is that teachers don't want to have the hard conversation when they get the evidence of cheating. And school leadership don't want to fix the issue, they want to put a chatbot on their resume. Sadly, I don't think OpenAI releasing this would have any effect fixing the problem. It's a problem w…

It's a problem with the people in education. Sounds absurd to reduce all of the thousands of school districts; millions of educators, K-12 teachers, professors, and administrators; and multitudes of viewpoints into one "ignorant" block of hapless Luddites.

To be fair, the Luddites weren't anti technology. They were standing against factories pricing them out of labor.

Re: OpenAI won't watermark ChatGPT text because its users could get caught

#39

Post by winternewt »

notarobot123 wrote: post #28
FeepingCreature wrote: post #15

Earlier quoted context omitted.

GPT-2 was never too dangerous to release, that's made up. OpenAI were trying to set a precedent to delay releases in anticipation of more dangerous models. This was and remains a good thing.

Maybe "too dangerous" is mainly a PR strategy to hype up the power of the tool. It's either PR or religious ideology (or both if you drink your own Kool-Aid)

It is now, I don't think it was then. OpenAI's moral compass has taken a sharp turn.

Re: OpenAI won't watermark ChatGPT text because its users could get caught

#40

Post by creesch »

carbondating wrote: post #6

Kind of pointless when there are many open-source models that are approaching ChatGPT-level people could use instead.

While the models might be there, the hardware to actually run them on is much widely spread, making it per definition less of an issue. The people that have the knowledge to actually run them is even smaller. The latter might not be much of a hurdle given tools like Ollama and jan, for now it still is a bit of a hurdle.
Post reply on HN