Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stack Overflow questions are being flooded with answers from ChatGPT

news.ycombinator.com

Post reply on HN
Print view View original

111–120 of 342 posts

Re: Stack Overflow questions are being flooded with answers from ChatGPT

#111

Post by mk_stjames »

ycombobreaker wrote: post #76
zone411 wrote: post #36

Earlier quoted context omitted.

That's just its default setting. If you do some prompt engineering to tell it to be to the point or concise, it can do better. E.g. I got: The repercussions of ChatGPT flooding Stack Overflow with answers could include decreased quality and relevance of answers, decreased user engagement, damage to the platform's reputation, and potential loss of revenue.

Still sounds too formal. To me, it reads like the intro to a high school paper. The next four paragraphs would elaborate on each of the points listed.

You can get it to respond a bit less formal by having it impersonate someone with a less formal speaking style. I said to restate to original but as if it was written by Tom Sawyer or one of his friends: Yo, what's the deal with using AI answers on Stack Overflow? It seems like it could be a real bummer. For one thing, the answers might not be as good as ones made by a human. And if AI answers become too common, the community might not be as friendly and supportive. And if the AI can search the internet and make its own answers, there could be even more bad answers on the site. All in all, this could be a major problem." Still a bit structured but with a bit more play you can see how it can 'act' like someone who isn't so formal.

Re: Stack Overflow questions are being flooded with answers from ChatGPT

#112

Post by luckylion »

gtirloni wrote: post #64
josephcsible wrote: post #56

I wouldn't even mind so much if the answers were right. The problem is that a lot of them are totally wrong, but completely reasonable- and plausible-sounding, and in an authoritative tone, so unless you already know the right answer, the only way you'll realize its answer is wrong is the hard way.

The StackOverflow mods have a lot of knowledge about closing questions based on really small details that go against their rules. I'm sure they will do well spotting AI-generated answers.

I doubt it. They've had problems with cheap automated answers for years where bots would essentially search for the question on SO and then copy an answer from another question verbatim. The answers were rarely useful because questions happen to be different even though they use the same keywords.

Not only did they never bother to block that, they also didn't mind it and wanted to rely on the community down-voting those answers instead of at least blocking the bot -- and that's with a trivial check (identical answer already in DB). With something that's AI-generated, there's no chance. And with the general quality of many of the answers, there's no way to tell apart wrong answers from AI or humans.

Re: Stack Overflow questions are being flooded with answers from ChatGPT

#113

Post by hgsgm »

9wzYQbTYsAIc wrote: post #105
wizeman wrote: post #95

Earlier quoted context omitted.

> I think we are going to have to take an aggressive stance against mediocre generated content here on HN, lest this small island of community be lost like the rest of the Old Internet. How do you propose we do that? And what do you propose we do when ChatGPT (or something like it) can create better content than most humans? (... which I would argue it is already happening in some limited contexts, although I admit m…

> How do you propose we do that? One technique, like all the other self-moderation that you can do on HN: stop upvoting and commenting on content that you don’t want to see boosted.

"Ignore it" has never been a solution to "other people misbehaving and destroying shared spaces".

Re: Stack Overflow questions are being flooded with answers from ChatGPT

#114

Post by hgsgm »

wellthisisgreat wrote: post #108
wizeman wrote: post #95

Earlier quoted context omitted.

> I think we are going to have to take an aggressive stance against mediocre generated content here on HN, lest this small island of community be lost like the rest of the Old Internet. How do you propose we do that? And what do you propose we do when ChatGPT (or something like it) can create better content than most humans? (... which I would argue it is already happening in some limited contexts, although I admit m…

hard bans for undisclosed machine-generated submissions?

No post body was provided.

Re: Stack Overflow questions are being flooded with answers from ChatGPT

#115

Post by boppo1 »

egypturnash wrote: post #75

Well, guess the genie's out of the bottle and we can never stop this. Bow down to the inevitability of technological progress, Luddites! Good luck retraining into a new job, I hear "prompt engineer automation" is the new hotness. Or at least that's what all of you kept telling me when I was expressing my unhappiness at the way corporate-sponsored image generating black boxes are built atop a shaky moral foundation th…

Artisans lost that battle 100 years ago with the rise of modernism.

Re: Stack Overflow questions are being flooded with answers from ChatGPT

#116

Post by formerly_proven »

wizeman wrote: post #95
theptip wrote: post #41

Earlier quoted context omitted.

Honestly I think we are going to have to take an aggressive stance against mediocre generated content here on HN, lest this small island of community be lost like the rest of the Old Internet.

> I think we are going to have to take an aggressive stance against mediocre generated content here on HN, lest this small island of community be lost like the rest of the Old Internet. How do you propose we do that? And what do you propose we do when ChatGPT (or something like it) can create better content than most humans? (... which I would argue it is already happening in some limited contexts, although I admit m…

No post body was provided.

Re: Stack Overflow questions are being flooded with answers from ChatGPT

#117

Post by rsync »

clusterhacks wrote: post #96

Human-curated content from trusted sources for top 1% information probably only available to subscribers will become more valuable and sought after. I suspect the days of generally trusting forums populated by anonymous users are done? I would not be surprised if the quality of human writing actually goes up. I have this weird feeling that ChatGPT and similar tools will become almost equivalent to calculators for mat…

The anonymity isn’t the problem - it’s the cost free aspect.

Anonymous content can work very well if there are costs incurred…

Re: Stack Overflow questions are being flooded with answers from ChatGPT

#118

Post by tpxl »

fouric wrote: post #51
ChrisMarshallNY wrote: post #16

I assume that this is by folks wanting to up their scores. That's a huge problem with "gamification." I'm not especially a fan of the concept, in a venue like SO. I think it has led to a rather nasty community, and I hardly ever go there, anymore. I assume that we'll be seeing a lot of robotic HN content (I would not be surprised if it is already here, but has been sidelined by the mods).

> I would not be surprised if it is already here, but has been sidelined by the mods I can virtually guarantee you that there's been a nontrivial amount of GPT-generated content on HN that has not been caught by mods since ChatGPT, and likely since GPT-2/3, as well. Dang (and the other one whose tag I can't recall) already have their hands full trying to keep the tone civil across thousands (tens of thousands?) of co…

> people actually reading the content of comments very carefully and upvoting logically sound arguments

You can make a perfectly logical comment that is completely wrong.

Re: Stack Overflow questions are being flooded with answers from ChatGPT

#119

Post by SheinhardtWigCo »

spaceman_2020 wrote: post #45
darkwater wrote: post #38

Earlier quoted context omitted.

This is fun but it's already starting to NOT be fun.

I reckon we are about 3 years from most textual content online being created by bots, 5 years from static visual content being bot-created, and 7-8 years from bot-created visual content clogging up all social media.

Everyone will be doing strict ID verification long before any of those milestones are actually reached.

Re: Stack Overflow questions are being flooded with answers from ChatGPT

#120

Post by xdennis »

brindidrip wrote: post #29

We need to start developing software to detect AI responses. To detect a response generated by ChatGPT, we could first analyze the content of the response to see if it contains any unnatural or repetitive language. We could also check the formatting of the response to see if it follows the typical conventions used by human responders on the platform. Additionally, we could check for any unusual patterns in the timest…

"We could also... Additionally, we could... Finally, we could" is a dead giveaway.

But to take it seriously, it would be quite sad when actual people will be banned for sounding too much like a bot.

Post reply on HN