Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPTZero Case Study – Exploring False Positives

gonzoknows.com

Post reply on HN
Print view View original

41–50 of 120 posts

Re: GPTZero Case Study – Exploring False Positives

#41

Post by dr_dshiv »

brookst wrote: post #16
panarky wrote: post #9

As millions of people interact with ChatGPT, their writing will subtly, gradually, begin to mimic its style. As future versions of the model are trained on this new text, both human and AI styles will converge until any difference between the two are infinitesimal.

One of the big complaints with LLMs is the confident hallucination of incorrect facts, like software APIs that don’t exist. But the way I see it, if ChatGPT thinks the Python list object should have a .is_sorted() property, that’s a pretty good indication that maybe it should. I work in PM (giant company, not Python), and one of these days my self-control will fail me and I will open a bug for “product does not suppo…

> it, if ChatGPT thinks the Python list object should have a .is_sorted() property, that’s a pretty good indication that maybe it should.

Yes! And when it hallucinates references for articles, often times those articles probably should exist…

Re: GPTZero Case Study – Exploring False Positives

#42

Post by jejeyyy77 »

sigmoid10 wrote: post #6
Maken wrote: post #2

Alternative title: Most academic papers are indistinguishable from AI generated babble.

...to AI. It's kinda funny how this is yet another area where these models suck very much in the same way that most humans do. LLMs are bad at arithmetic? So are most people. Can't tell science from babble? I already wouldn't ask a non-expert to rate any aspect of an academic paper. Trusting the average Joe who has only completed some basic form of education would be tremendously stupid. Same with these models. Maybe…

I mean, humans can't distinguish AI written text either - which is why this tool was built?

I don't see how it will be possible to build such a tool either as the combination of words that can come after another is finite.

Re: GPTZero Case Study – Exploring False Positives

#43

Post by jmfldn »

MonkeyMalarky wrote: post #18
jmfldn wrote: post #15

It's important to shine a light on the limitations of AI detection software, and this case study on GPTZero does just that. False positives can have serious consequences, particularly in sensitive areas such as healthcare.

Thank you for your heroic effort in copying and pasting a chat log, I am left in awe by it. ...nice edit...

Lighten up mate.

Re: GPTZero Case Study – Exploring False Positives

#44

Post by sebzim4500 »

wand3r wrote: post #12

I saw this[1] interview with Sam Altman touching on interim AI impact. I really agree with his point that basically detecting output from LLMs is basically going to be futile and only really relevant in the near term. Accuracy is obviously going to improve in models and detection isnt that difficult now but will be in the future, especially if output is modified or an attempt to obfuscate origin is made. [1] https://…

>detection isnt that difficult now

I would have thought this, but every attempt I've seen at detecting chatGPT generated text has failed miserably.

Re: GPTZero Case Study – Exploring False Positives

#45

Post by RC_ITR »

wand3r wrote: post #12

I saw this[1] interview with Sam Altman touching on interim AI impact. I really agree with his point that basically detecting output from LLMs is basically going to be futile and only really relevant in the near term. Accuracy is obviously going to improve in models and detection isnt that difficult now but will be in the future, especially if output is modified or an attempt to obfuscate origin is made. [1] https://…

> Accuracy is obviously going to improve in models

Well, to be clear, they can put rules based filters and other things on top of the neural net, but the core GPT will never get more accurate since it has no mechanism to understand what words mean.

Re: GPTZero Case Study – Exploring False Positives

#46

Post by thomastjeffery »

photochemsyn wrote: post #40

Just wrote this myself, although I did try to chatGPT-style it a bit. I thought the final third would serve to identify it as non-AI as it goes off on a tangent about isotopes... > "The periodic table is a systematic ordering of elements by certain charcteristics including: the number of protons they contain, the number of electrons they usually have in their outer shells, and the nature of their partially-filled out…

Technical writing, in order to be relatively unambiguous - the "technical" part - defines itself as a subset of English with a constrained grammar and vocabulary.

You just illustrated technical writing. Naturally, your writing style is very similar to that of other technical writing.

Take one guess what kind of writing exists in most of the text GPT is trained on.

Re: GPTZero Case Study – Exploring False Positives

#47

Post by catchnear4321 »

photochemsyn wrote: post #40

Just wrote this myself, although I did try to chatGPT-style it a bit. I thought the final third would serve to identify it as non-AI as it goes off on a tangent about isotopes... > "The periodic table is a systematic ordering of elements by certain charcteristics including: the number of protons they contain, the number of electrons they usually have in their outer shells, and the nature of their partially-filled out…

How has your “success” been with chatgpt? Qualitatively, generally, positive/negative.

Being able to speak to machines would likely correlate with “sounding like one.”

There could be a different (mis)categorization here but also por que no los dos.

Re: GPTZero Case Study – Exploring False Positives

#48

Post by ominous »

moralestapia wrote: post #13
panarky wrote: post #9

As millions of people interact with ChatGPT, their writing will subtly, gradually, begin to mimic its style. As future versions of the model are trained on this new text, both human and AI styles will converge until any difference between the two are infinitesimal.

+1 We train AIs but they also train us.

Some related idea, in case you like to see that thought explore: https://medium.com/@freddavis/we-shape-our-tools-and-thereaf...

Re: GPTZero Case Study – Exploring False Positives

#49

Post by zdragnar »

panarky wrote: post #9

As millions of people interact with ChatGPT, their writing will subtly, gradually, begin to mimic its style. As future versions of the model are trained on this new text, both human and AI styles will converge until any difference between the two are infinitesimal.

So long as ChatGPT is forbidden from communicating in certain ways (swearing, speaking ill or positive of controversial people or topics, etc), convergence will never happen. People interact with other people more than they do ChatGPT, so the majority force will remain dominant.

Re: GPTZero Case Study – Exploring False Positives

#50

Post by martinflack »

wand3r wrote: post #12

I saw this[1] interview with Sam Altman touching on interim AI impact. I really agree with his point that basically detecting output from LLMs is basically going to be futile and only really relevant in the near term. Accuracy is obviously going to improve in models and detection isnt that difficult now but will be in the future, especially if output is modified or an attempt to obfuscate origin is made. [1] https://…

It's probably a short-term social phenomenon. We don't bother detecting mathematical output from calculators or spreadsheets; we just like that folks give us the right answer, even if they had easy tooling to produce it. However, watching someone do things the old way would seem bemusing. If you watched a manager notating all over a physical spreadsheet with a pencil (as was commonly done at one time) it would seem quaint or backwards depending on context. Likewise, waiting for someone to write a letter and taking more than 90 seconds because they didn't co-author it with AI might seem slow.
Post reply on HN