Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPTZero Case Study – Exploring False Positives

gonzoknows.com

Post reply on HN
Print view View original

91–100 of 120 posts

Re: GPTZero Case Study – Exploring False Positives

#91

Post by int_19h »

brookst wrote: post #28
xiphias2 wrote: post #20

Earlier quoted context omitted.

Or rather you can just catch method missing in the runtime and patch it with a chatgpt call

Love it. Get ChatGPT to write the missing method, execute it this once, then store it in a file, update the current source file with the include to cache it for next time.

I can't find it, but someone already did a Python module that plugs into GPT-3 and automatically generates functions on the fly as you call them - and then the same for methods on the returned values etc.

Re: GPTZero Case Study – Exploring False Positives

#92

Post by mtlmtlmtlmtl »

simonh wrote: post #90
mtlmtlmtlmtl wrote: post #81

Earlier quoted context omitted.

If you're writing in a style you're unfamiliar with, how do you know the model is doing it correctly? I also think writing yourself might be far better practice. This tool can easily become a crutch. This is unlikely to be free anytime soon. In fact it's likely to be quite expensive.

ChatGPT is free now, although there is a paid tier, and MS and Google are building similar capabilities right into their search interfaces.

ChatGPT won't be free forever.

Whatever LLM search stuff comes along will only be free as long as it brings in ad revenue. Which involves making the models fundamentally worse most likely. Or they'll use it to collect personal data. Probably both.

Computationally, GPT is wildly expensive. This idea people have that it's gonna be used all over the place for all sorts of tiny tasks, as if it's just another REST API, is nuts. Unless something fundamentally new comes along that makes these models much cheaper, adoption is likely going to end up much more limited than people expect. Or siloed off into expensive business-facing products.

Re: GPTZero Case Study – Exploring False Positives

#93

Post by int_19h »

dsign wrote: post #72
FPGAhacker wrote: post #62

Earlier quoted context omitted.

Did chatgpt post the reference as a footnote (or parenthetical)? At least for now,I was thinking it didn’t do that, and maybe the lack of references would be an indicator of unedited gpt output.

It did: ... > However, according to a study published in the Journal of Dairy Science, the diacetyl content of butter can range from approximately 0.5 to 14 parts per million (ppm) ( https://doi.org/10.3168/jds.S0022-0302(03)73775-3 ). The doi I could not find, so I'm pretty sure is bogus. I asked it to produce a full reference: > Sure, the full reference for the study I mentioned is: > Yvon, M., Chambellon, E., & Bo…

It's okay, GPT just gets confused about which of the quantum many-worlds it is currently in. Just ask it to write the article as needed. ~

Re: GPTZero Case Study – Exploring False Positives

#94

Post by lwhi »

mattnewton wrote: post #36
lwhi wrote: post #24

Perhaps AI generated text should be created with a specific signature in mind _specifically_ to be identifiable?

isn’t this essentially asking anyone who runs a model to flip the evil bit[0]? People who want to misrepresent model output as human written output will trivially be able to beat this protection by removing the signature or using a version of the model that simply doesn’t add it. [0] https://en.m.wikipedia.org/wiki/Evil_bit

I think you're correct, but it would promote the idea that what's produced is a basis or source for further work .. and would mean that effort is required.

Feels like a good basis tech for something like ChatGPT.

Re: GPTZero Case Study – Exploring False Positives

#95

Post by lwhi »

wongarsu wrote: post #51
lwhi wrote: post #24

Perhaps AI generated text should be created with a specific signature in mind _specifically_ to be identifiable?

There's a large body of research into invisible text watermarking, so this would certainly be possible. Maybe the simplest to implement in LLMs would be to bias the token generation slightly, for example by making tokens that include the letter i slightly more likely. In a long enough text you could then see the deviation from normal human text characteristics.

Yep, I think it's a good idea.

Re: GPTZero Case Study – Exploring False Positives

#96

Post by stevenhuang »

wand3r wrote: post #12

I saw this[1] interview with Sam Altman touching on interim AI impact. I really agree with his point that basically detecting output from LLMs is basically going to be futile and only really relevant in the near term. Accuracy is obviously going to improve in models and detection isnt that difficult now but will be in the future, especially if output is modified or an attempt to obfuscate origin is made. [1] https://…

It sounds feasible to detect cases of low temperature output, as the text output this way sounds very generic (like an average of all content seen near the prompt's latent space).

However once you prompt the LLM with a higher temperature, or tell it to roleplay as someone with elaborate personas, or suggest to use certain linguistic styles, or train it on example text... then it becomes much harder.

I imagine pathological cases of formulaic word use, sentence/paragraph structure will only be detectable in longer form text. After all text is already pretty low-resolution, not much for adversarial models to work with.

Re: GPTZero Case Study – Exploring False Positives

#97

Post by PeterisP »

flir wrote: post #76
valine wrote: post #60

Earlier quoted context omitted.

GPT3 is far more accurate than GPT2. Seems reasonable that larger models trained on more data will continue to improve accuracy. I'd also expect larger models to be better at summarizing text, ie potentially fixing the Bing issues where it hallucinates numbers. Our models sizes are a product of our scaling and hardware limitations. There's no reason to believe we are anywhere near optimal.

> Seems reasonable that larger models trained on more data will continue to improve accuracy. It also seems reasonable to assume that they will eventually encounter diminishing returns, and that the current issues, such as hallucinations, are inherent to the approach and may never be resolved. To be clear I don't have a clue which statement is true (though I don't see why scaling would solve the hallucination problem…

Scaling of models is a very researched area, and currently all the experiments show that scaling doesn't really get diminishing returns - that was checked in GPT-2 "era" with model sizes from very small up to GPT-2, and reconfirmed with GPT-3 and then with newer models. As far as we can see, scaling does not result in diminishing returns; and while it's certainly possible that we eventually encounter diminishing returns, it is not reasonable to presume that we actually will any time soon (as we have literally zero evidence for that and at least some evidence to the contrary), and even if we will, there's currently no reason to assume that the eventual breaking point is somewhere at "GPT-5" and not "GPT-15" or "GPT-55".

Re: GPTZero Case Study – Exploring False Positives

#98

Post by PeterisP »

wongarsu wrote: post #51
lwhi wrote: post #24

Perhaps AI generated text should be created with a specific signature in mind _specifically_ to be identifiable?

There's a large body of research into invisible text watermarking, so this would certainly be possible. Maybe the simplest to implement in LLMs would be to bias the token generation slightly, for example by making tokens that include the letter i slightly more likely. In a long enough text you could then see the deviation from normal human text characteristics.

All the main scenarios for detecting generating text that I can imagine do have to assume that the LLM isn't "cooperative" but actually is specifically designed (or fine-tuned) to avoid detection.

Re: GPTZero Case Study – Exploring False Positives

#100

Post by moralestapia »

ominous wrote: post #48
moralestapia wrote: post #13

Earlier quoted context omitted.

+1 We train AIs but they also train us.

Some related idea, in case you like to see that thought explore: https://medium.com/@freddavis/we-shape-our-tools-and-thereaf...

Great read, thanks for sharing!
Post reply on HN