Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPTZero Case Study – Exploring False Positives

gonzoknows.com

Post reply on HN
Print view View original

81–90 of 120 posts

Re: GPTZero Case Study – Exploring False Positives

#81

Post by mtlmtlmtlmtl »

robocat wrote: post #79
mtlmtlmtlmtl wrote: post #68

Earlier quoted context omitted.

"Write an email that says I did X and they should do Y, but if Z then W, and we should schedule a meeting with P and Q." I feel like for most emails I write, information density is close to a maximum. This means there's no actual gain to be had from a language model. The email I would write myself is going to be about the same length as the prompt I'd have to write anyway.

There is a huge gain if English is your second language, and you use ChatGPT to rewrite, or translate. Even if English is your mother tongue, if your written English is crappy or you need to write in a style you are unfamiliar with (e.g. formal), then ChatGPT can help.

If you're writing in a style you're unfamiliar with, how do you know the model is doing it correctly?

I also think writing yourself might be far better practice. This tool can easily become a crutch. This is unlikely to be free anytime soon. In fact it's likely to be quite expensive.

Re: GPTZero Case Study – Exploring False Positives

#82

Post by joe_the_user »

panarky wrote: post #9

As millions of people interact with ChatGPT, their writing will subtly, gradually, begin to mimic its style. As future versions of the model are trained on this new text, both human and AI styles will converge until any difference between the two are infinitesimal.

An opposite possibility is that the commonness of ChatGPT will cause people to adopt a style as distinct from it as possible.

Of course, this might mean future Chatbots would successfully emulate that. But it's not impossible an "adversarial style" exists - this wouldn't be impossible to emulate but it might be more likely to cause the emulator to say things the reader can immediately tell are false.

One idea is to "flirt" with all things that people have come up with that AI chokes on. "Back when the golden gate bridge was carried across Egypt..."

Re: GPTZero Case Study – Exploring False Positives

#83

Post by robocat »

photochemsyn wrote: post #40

Just wrote this myself, although I did try to chatGPT-style it a bit. I thought the final third would serve to identify it as non-AI as it goes off on a tangent about isotopes... > "The periodic table is a systematic ordering of elements by certain charcteristics including: the number of protons they contain, the number of electrons they usually have in their outer shells, and the nature of their partially-filled out…

Perhaps you just write very predictably.

A partisan writing cliched slogans and regurgitating tired political statements has a “temperature” closer to 0, likewise an engineer stringing together typical word combinations. A poetical writer with surprising twists and counterintuitive mixtures of words has a temperature closer to 1.0.

I can see a few clichés in your writing, also I did a search on some fragments of sentences which showed a number of results.

If you want to be less “robotic” then you could add whimsical or poetic wording, and less common turns of words (be a phrase rotator).

Re: GPTZero Case Study – Exploring False Positives

#84

Post by robocat »

mtlmtlmtlmtl wrote: post #81
robocat wrote: post #79

Earlier quoted context omitted.

There is a huge gain if English is your second language, and you use ChatGPT to rewrite, or translate. Even if English is your mother tongue, if your written English is crappy or you need to write in a style you are unfamiliar with (e.g. formal), then ChatGPT can help.

If you're writing in a style you're unfamiliar with, how do you know the model is doing it correctly? I also think writing yourself might be far better practice. This tool can easily become a crutch. This is unlikely to be free anytime soon. In fact it's likely to be quite expensive.

I think we all tend to be better at picking out a correct answer than generating a correct answer from scratch.

I can ask ChatGPT to rewrite an email in the style an American news reporter from 1950, and I can judge whether some of the cliches it generates feel correct. I cannot write in that style at all.

Re: GPTZero Case Study – Exploring False Positives

#85

Post by Closi »

theptip wrote: post #75
Closi wrote: post #52

Earlier quoted context omitted.

It fails on false positives, but you don't often get false negatives, which might be enough at the moment for quite a few use-cases. Also false positives are typically "this text is likely to contain parts that were AI generated" rather than "This text is higly likely to be AI generated" (which is what GPT-generated content generally produces). When I've tried to prompt-engineer GPT to produce text that GPTZero will…

Of course false positives matter; the naive heuristic “everything is AI generated” has zero false negatives, and mostly false positives. In the OP half of the positives are false. That’s not a useful signal IMO. You couldn’t use that to police homework for example.

I said they might not matter for quite a few use cases, not that they don't matter for all use-cases.

e.g. If you were Sam Altman at OpenAI and your use-case is mostly looking for training data and wanting to tell if it is AI-Generated or not (so you can exclude this from training data), you probably care much more about false negatives than false positives (false positives just reduce your training data set size slightly, while false negatives pollute it).

Of course they matter if you are marking homework (where conversely false negatives aren't actually that important!), but it's pretty trivial to think of use-cases where the opposite is true.

Re: GPTZero Case Study – Exploring False Positives

#86

Post by andyjohnson0 »

ClumsyPilot wrote: post #33
andyjohnson0 wrote: post #32

Earlier quoted context omitted.

Prediction #1: Once enough ChatGPT output gets posted online, it will inevitably find its way into the training corpus. When that happens, ChatGPT becomes stateful and develops episodic memory. Prediction #2: As more people discuss ChatGPT online, by late 2023 discussion of Roko's Basilisk exceeds discussion of ChatGPT. (half /s)

Or. ChatGPT will overtrain on it's own data and go to shit the way google search did

Meta-ChatGPT's loss function optimises for ChatGPT generating training data that maximises the shittyness of Google's LLM.

Re: GPTZero Case Study – Exploring False Positives

#87

Post by moffkalast »

visarga wrote: post #56
dr_dshiv wrote: post #41

Earlier quoted context omitted.

> it, if ChatGPT thinks the Python list object should have a .is_sorted() property, that’s a pretty good indication that maybe it should. Yes! And when it hallucinates references for articles, often times those articles probably should exist…

And if they don't exist, you ask the model to write them from title and link.

The year is 2145.

When a new person is born their entire life is hallucinated in its entirety by the all great and powerful GPT. Deviation from His plan is met with swift and severe consequences.

Re: GPTZero Case Study – Exploring False Positives

#88

Post by mtsr »

flir wrote: post #76
valine wrote: post #60

Earlier quoted context omitted.

GPT3 is far more accurate than GPT2. Seems reasonable that larger models trained on more data will continue to improve accuracy. I'd also expect larger models to be better at summarizing text, ie potentially fixing the Bing issues where it hallucinates numbers. Our models sizes are a product of our scaling and hardware limitations. There's no reason to believe we are anywhere near optimal.

> Seems reasonable that larger models trained on more data will continue to improve accuracy. It also seems reasonable to assume that they will eventually encounter diminishing returns, and that the current issues, such as hallucinations, are inherent to the approach and may never be resolved. To be clear I don't have a clue which statement is true (though I don't see why scaling would solve the hallucination problem…

Might turn out that for rules based systems such as prescriptive grammars (for grammatically correct language rather than natural spoken) there is still use for a system that explicitly represents those rules.

Then again, we are a big old bulb of wetware and we can generally learn to apply grammar rules correctly most of the time (when explicitly thinking about them, anyway).

Maybe what we need is some kind of meta cognition: being able to apply and evaluate rules that the current LLMs can already correctly reproduce.

Re: GPTZero Case Study – Exploring False Positives

#89

Post by int_19h »

flir wrote: post #76
valine wrote: post #60

Earlier quoted context omitted.

GPT3 is far more accurate than GPT2. Seems reasonable that larger models trained on more data will continue to improve accuracy. I'd also expect larger models to be better at summarizing text, ie potentially fixing the Bing issues where it hallucinates numbers. Our models sizes are a product of our scaling and hardware limitations. There's no reason to believe we are anywhere near optimal.

> Seems reasonable that larger models trained on more data will continue to improve accuracy. It also seems reasonable to assume that they will eventually encounter diminishing returns, and that the current issues, such as hallucinations, are inherent to the approach and may never be resolved. To be clear I don't have a clue which statement is true (though I don't see why scaling would solve the hallucination problem…

The biggest problem is that scaling is non-linear. The returns might well be non-diminishing wrt model size, but if we have to throw N^2 hardware at it to make it (best-case) 2N better, we'll still hit the limit pretty quickly.

Re: GPTZero Case Study – Exploring False Positives

#90

Post by simonh »

mtlmtlmtlmtl wrote: post #81
robocat wrote: post #79

Earlier quoted context omitted.

There is a huge gain if English is your second language, and you use ChatGPT to rewrite, or translate. Even if English is your mother tongue, if your written English is crappy or you need to write in a style you are unfamiliar with (e.g. formal), then ChatGPT can help.

If you're writing in a style you're unfamiliar with, how do you know the model is doing it correctly? I also think writing yourself might be far better practice. This tool can easily become a crutch. This is unlikely to be free anytime soon. In fact it's likely to be quite expensive.

ChatGPT is free now, although there is a paid tier, and MS and Google are building similar capabilities right into their search interfaces.
Post reply on HN