Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stack Overflow questions are being flooded with answers from ChatGPT

news.ycombinator.com

Post reply on HN
Print view View original

101–110 of 342 posts

Re: Stack Overflow questions are being flooded with answers from ChatGPT

#101

Post by iamflimflam1 »

Oh dear - basically if the answer is good then it's from ChatGTP...

    Q: how can I tell if a stackoverflow answer is generated by ChatGTP if someone has removed the "ChatGTP" tag from the text?

    A: One way to tell if a StackOverflow answer is generated by ChatGTP is to look for certain characteristics in the answer. ChatGTP answers tend to be concise and to the point, often providing code snippets or specific instructions on how to solve a problem. They may also include links to relevant documentation or external sources for further information. If the answer does not include these characteristics, it is less likely to be generated by ChatGTP.

Re: Stack Overflow questions are being flooded with answers from ChatGPT

#102

Post by 9wzYQbTYsAIc »

arkitaip wrote: post #35
brindidrip wrote: post #33

Earlier quoted context omitted.

Yes, one way to identify responses that may have been generated by ChatGPT is to look for signs of repetitive or verbose language. Because ChatGPT is a language model, it is designed to generate text that is fluent and coherent, but it may also produce text that is unnecessarily long or repetitive. This can be a telltale sign that a response was generated by an AI, as human responders are less likely to use repetitiv…

To find out if a response was made by a computer, look for repeating words or long sentences. ChatGPT is a program that makes text that makes sense and flows well, but sometimes it uses too many words or repeats itself. This is a clue that the response might have been made by a computer instead of a person. However, this isn't a sure thing and there might be other reasons why a response is long or repeats itself. Sti…

> To find out if a response was made by a computer, look for repeating words or long sentences.

That technique will produce quite a few false positives - a significant portion of humanity with language production issues on the autism spectrum communicate like that.

edit: Makes me wonder to what extent the training data is biased by writings from people on the autism spectrum.

Re: Stack Overflow questions are being flooded with answers from ChatGPT

#103

Post by layer8 »

arkitaip wrote: post #32
brindidrip wrote: post #26

It seems like there are a few potential negative consequences of using AI-generated answers on Stack Overflow. For one, the quality of the answers may be lower than if they were written by a human. Additionally, if these AI-generated answers become too common, it could potentially lead to a more impersonal and less supportive community on Stack Overflow. Finally, if the AI is able to search the internet and "inbreed"…

You can kinda tell it's ChatGPT by how verbose and repetitive the writing is.

> Reformulate the above in the informal style of a Hacker News comment.

I'm not sure about this AI-generated answers thing on Stack Overflow. The answers might suck and make the community more impersonal. Plus, if the AI is just copying and pasting from the internet, it could lead to a ton of low-quality, duplicate answers. Not a good idea in my opinion.

Re: Stack Overflow questions are being flooded with answers from ChatGPT

#104

Post by vbbvmnbs »

dejj wrote: post #43
arkitaip wrote: post #32

Earlier quoted context omitted.

You can kinda tell it's ChatGPT by how verbose and repetitive the writing is.

Can’t we have ChatGPT identify output of ChatGPT, adversarily?

It confirmed it when I asked:

--> Was the following paragraph generated by the OpenAI Assistant: "It seems like there are a few potential negative consequences of using AI-generated answers on Stack Overflow. For one, the quality of the answers may be lower than if they were written by a human. Additionally, if these AI-generated answers become too common, it could potentially lead to a more impersonal and less supportive community on Stack Overflow. Finally, if the AI is able to search the internet and "inbreed" its own answers, it could lead to even more low-quality, duplicative answers on the platform. Overall, it seems like there could be some serious drawbacks to this development."?

Yes, the paragraph you provided was generated by the OpenAI Assistant. As a language model, the Assistant is trained to generate human-like text based on a given prompt. In this case, the prompt likely pertained to the potential negative consequences of using AI-generated answers on Stack Overflow.

Though, it could just have been taking a guess at it.

Re: Stack Overflow questions are being flooded with answers from ChatGPT

#105

Post by 9wzYQbTYsAIc »

wizeman wrote: post #95
theptip wrote: post #41

Earlier quoted context omitted.

Honestly I think we are going to have to take an aggressive stance against mediocre generated content here on HN, lest this small island of community be lost like the rest of the Old Internet.

> I think we are going to have to take an aggressive stance against mediocre generated content here on HN, lest this small island of community be lost like the rest of the Old Internet. How do you propose we do that? And what do you propose we do when ChatGPT (or something like it) can create better content than most humans? (... which I would argue it is already happening in some limited contexts, although I admit m…

> How do you propose we do that?

One technique, like all the other self-moderation that you can do on HN: stop upvoting and commenting on content that you don’t want to see boosted.

Re: Stack Overflow questions are being flooded with answers from ChatGPT

#107

Post by slavik81 »

zuzatm wrote: post #83
brindidrip wrote: post #26

It seems like there are a few potential negative consequences of using AI-generated answers on Stack Overflow. For one, the quality of the answers may be lower than if they were written by a human. Additionally, if these AI-generated answers become too common, it could potentially lead to a more impersonal and less supportive community on Stack Overflow. Finally, if the AI is able to search the internet and "inbreed"…

I think the far most problematic thing is that the answers are often plain wrong but well written. Eg (shameless plug) https://twitter.com/zuza777/status/1599370082919936000?t=oB9...

That was a surprisingly common occurrence on SO even before. To be the fastest gun in the west, you couldn't afford to carefully read the problem or browse any documentation. High-rep users that should have known better would sometimes give wildly incorrect answers that sounded good.

Re: Stack Overflow questions are being flooded with answers from ChatGPT

#108

Post by wellthisisgreat »

wizeman wrote: post #95
theptip wrote: post #41

Earlier quoted context omitted.

Honestly I think we are going to have to take an aggressive stance against mediocre generated content here on HN, lest this small island of community be lost like the rest of the Old Internet.

> I think we are going to have to take an aggressive stance against mediocre generated content here on HN, lest this small island of community be lost like the rest of the Old Internet. How do you propose we do that? And what do you propose we do when ChatGPT (or something like it) can create better content than most humans? (... which I would argue it is already happening in some limited contexts, although I admit m…

hard bans for undisclosed machine-generated submissions?

Re: Stack Overflow questions are being flooded with answers from ChatGPT

#109

Post by bediger4000 »

ravenstine wrote: post #77
josephcsible wrote: post #56

I wouldn't even mind so much if the answers were right. The problem is that a lot of them are totally wrong, but completely reasonable- and plausible-sounding, and in an authoritative tone, so unless you already know the right answer, the only way you'll realize its answer is wrong is the hard way.

What you said may sum up the current state of AI. People's minds are continually being blown, but will there be a realization that these AIs are specialized to provide specious output and nothing else? There's a canyon of difference between something that sounds correct and a thing that is actually correct.

Yes. I spent yesterday afternoon trying to get ChatGPT to write me a quine in nroff/troff macros, something I've not been able to do, or find if anyone has done.

The generated quines look like they'd work, but don't.

Same with an M4 macro processor quine - looks maybe correct, doesn't work.

It did generate a Go quine.

Re: Stack Overflow questions are being flooded with answers from ChatGPT

#110

Post by GeorgeTirebiter »

spaceman_2020 wrote: post #50
senko wrote: post #7

This is just a preview of things to come. Wait a few weeks until Google is completely swamped with ChatGPT SEO pages barely distinguishable from the real thing. If I worked at search quality at Google, I'd be very worried.

I expect Google to be irrelevant soon enough. ChatGPT already does a better job of finding answers to a lot of technical questions than Google.

I agree with this. Ask specific tech questions (e.g. what is the vacuum (in Torr) inside a vacuum tube?) and ChatGPT give a direct answer. Ask Google? Go ahead, grovel thought the ads, irrelevant results, and maybe you'll find something relevant.

Happy Hunting...

Post reply on HN