Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPTZero Case Study – Exploring False Positives

gonzoknows.com

Post reply on HN
Print view View original

101–110 of 120 posts

Re: GPTZero Case Study – Exploring False Positives

#101

Post by Jensson »

PeterisP wrote: post #97
flir wrote: post #76

Earlier quoted context omitted.

> Seems reasonable that larger models trained on more data will continue to improve accuracy. It also seems reasonable to assume that they will eventually encounter diminishing returns, and that the current issues, such as hallucinations, are inherent to the approach and may never be resolved. To be clear I don't have a clue which statement is true (though I don't see why scaling would solve the hallucination problem…

Scaling of models is a very researched area, and currently all the experiments show that scaling doesn't really get diminishing returns - that was checked in GPT-2 "era" with model sizes from very small up to GPT-2, and reconfirmed with GPT-3 and then with newer models. As far as we can see, scaling does not result in diminishing returns; and while it's certainly possible that we eventually encounter diminishing retu…

If significantly bigger models than now got better results we would have seen papers about that a long time ago so that the team/company can get more funding, lots of rich actors has worked on that for years.

If it doesn't produce better results however then they want their competitors to waste lots of money to make the same mistakes, there is really no benefit from publishing that and lots of drawbacks.

Otherwise it seems too much of a coincidence that Google and OpenAI ended up with models of basically the same size. Google could have trained a model 5x-10x larger easily, it isn't that expensive to them, but for some reason we didn't see that, and GPT-4 just never seems to launch.

Re: GPTZero Case Study – Exploring False Positives

#102

Post by LelouBil »

andyjohnson0 wrote: post #32
panarky wrote: post #9

As millions of people interact with ChatGPT, their writing will subtly, gradually, begin to mimic its style. As future versions of the model are trained on this new text, both human and AI styles will converge until any difference between the two are infinitesimal.

Prediction #1: Once enough ChatGPT output gets posted online, it will inevitably find its way into the training corpus. When that happens, ChatGPT becomes stateful and develops episodic memory. Prediction #2: As more people discuss ChatGPT online, by late 2023 discussion of Roko's Basilisk exceeds discussion of ChatGPT. (half /s)

Did you see the new Bing chat ?

#1 is already happening !

See here (other HN thread) : https://twitter.com/tobyordoxford/status/1627414519784910849

Re: GPTZero Case Study – Exploring False Positives

#103

Post by throwaway675309 »

ericmcer wrote: post #63
gtsnexp wrote: post #61

Detecting text generated by large language models like ChatGPT is a challenging task. One of the main difficulties is that the generated text can be highly variable and can cover a wide range of topics and styles. These models have learned to mimic human writing patterns and can produce text that is grammatically correct, semantically coherent, and even persuasive, making it difficult for humans to distinguish betwee…

It was weird how easy this was to identify if you have read any amount of ChatGPT content. It has a particular writing style that is pretty obvious. I am not sure how you would code something to detect an author based on writing style. It feels like something people would have tried to do before. Probably using a similar approach that LLMs use but with a separate predictor for specific authors.

It's because it reads exactly like every middle schooler writing their first MLA formatted five paragraph essay.

Re: GPTZero Case Study – Exploring False Positives

#104

Post by anothernewdude »

Closi wrote: post #52
sebzim4500 wrote: post #44

Earlier quoted context omitted.

>detection isnt that difficult now I would have thought this, but every attempt I've seen at detecting chatGPT generated text has failed miserably.

It fails on false positives, but you don't often get false negatives, which might be enough at the moment for quite a few use-cases. Also false positives are typically "this text is likely to contain parts that were AI generated" rather than "This text is higly likely to be AI generated" (which is what GPT-generated content generally produces). When I've tried to prompt-engineer GPT to produce text that GPTZero will…

You don't prompt-engineer, this is solvable by changing parameters. You need a higher temperature

Re: GPTZero Case Study – Exploring False Positives

#105

Post by Dylan16807 »

mtlmtlmtlmtl wrote: post #92
simonh wrote: post #90

Earlier quoted context omitted.

ChatGPT is free now, although there is a paid tier, and MS and Google are building similar capabilities right into their search interfaces.

ChatGPT won't be free forever. Whatever LLM search stuff comes along will only be free as long as it brings in ad revenue. Which involves making the models fundamentally worse most likely. Or they'll use it to collect personal data. Probably both. Computationally, GPT is wildly expensive. This idea people have that it's gonna be used all over the place for all sorts of tiny tasks, as if it's just another REST API, is…

The cost estimates I see are things like less than half a cent per query and a few cents per conversation.

If I'm making (or valuing my free time at) $20 an hour, then ChatGPT only needs to save me one or two seconds and half a minute respectively.

Re: GPTZero Case Study – Exploring False Positives

#106

Post by RC_ITR »

valine wrote: post #60
RC_ITR wrote: post #45

Earlier quoted context omitted.

> Accuracy is obviously going to improve in models Well, to be clear, they can put rules based filters and other things on top of the neural net, but the core GPT will never get more accurate since it has no mechanism to understand what words mean.

GPT3 is far more accurate than GPT2. Seems reasonable that larger models trained on more data will continue to improve accuracy. I'd also expect larger models to be better at summarizing text, ie potentially fixing the Bing issues where it hallucinates numbers. Our models sizes are a product of our scaling and hardware limitations. There's no reason to believe we are anywhere near optimal.

> GPT3 is far more accurate than GPT2

Please say more about what you mean here because I disagree.

It’s certainly more eloquent, but it still can’t multiple 2 4-digit numbers…

Re: GPTZero Case Study – Exploring False Positives

#107

Post by valine »

Jensson wrote: post #101
PeterisP wrote: post #97

Earlier quoted context omitted.

Scaling of models is a very researched area, and currently all the experiments show that scaling doesn't really get diminishing returns - that was checked in GPT-2 "era" with model sizes from very small up to GPT-2, and reconfirmed with GPT-3 and then with newer models. As far as we can see, scaling does not result in diminishing returns; and while it's certainly possible that we eventually encounter diminishing retu…

If significantly bigger models than now got better results we would have seen papers about that a long time ago so that the team/company can get more funding, lots of rich actors has worked on that for years. If it doesn't produce better results however then they want their competitors to waste lots of money to make the same mistakes, there is really no benefit from publishing that and lots of drawbacks. Otherwise it…

It’s not just the cost of training the model, it’s the cost of doing inference at scale. ChatGPT boarder line too expensive to operate already. It’s hard to imagine a larger model that both economical and used by millions of people with our current hardware.

Re: GPTZero Case Study – Exploring False Positives

#108

Post by chrisdbanks »

mtlmtlmtlmtl wrote: post #92
simonh wrote: post #90

Earlier quoted context omitted.

ChatGPT is free now, although there is a paid tier, and MS and Google are building similar capabilities right into their search interfaces.

ChatGPT won't be free forever. Whatever LLM search stuff comes along will only be free as long as it brings in ad revenue. Which involves making the models fundamentally worse most likely. Or they'll use it to collect personal data. Probably both. Computationally, GPT is wildly expensive. This idea people have that it's gonna be used all over the place for all sorts of tiny tasks, as if it's just another REST API, is…

And like everything, the price will drop rapidly as the models become smaller through advances. There's already sparseGPT.

Re: GPTZero Case Study – Exploring False Positives

#109

Post by chrisdbanks »

RC_ITR wrote: post #106
valine wrote: post #60

Earlier quoted context omitted.

GPT3 is far more accurate than GPT2. Seems reasonable that larger models trained on more data will continue to improve accuracy. I'd also expect larger models to be better at summarizing text, ie potentially fixing the Bing issues where it hallucinates numbers. Our models sizes are a product of our scaling and hardware limitations. There's no reason to believe we are anywhere near optimal.

> GPT3 is far more accurate than GPT2 Please say more about what you mean here because I disagree. It’s certainly more eloquent, but it still can’t multiple 2 4-digit numbers…

They probably mean that if you test it on common NLP benchmarks it performs better.

Re: GPTZero Case Study – Exploring False Positives

#110

Post by Nevermark »

scotty79 wrote: post #80
wand3r wrote: post #12

I saw this[1] interview with Sam Altman touching on interim AI impact. I really agree with his point that basically detecting output from LLMs is basically going to be futile and only really relevant in the near term. Accuracy is obviously going to improve in models and detection isnt that difficult now but will be in the future, especially if output is modified or an attempt to obfuscate origin is made. [1] https://…

I hope gpt detectors will evolve into general bs detectors.

I wonder how good the best language models are at detecting bs.

So many experiments to run...

Post reply on HN