Live data from Hacker News

Stack Overflow is almost dead

blog.pragmaticengineer.com

101–110 of 285 posts

Re: Stack Overflow is almost dead

#101
> This tallies with my memory of feeling that site moderators had gone on a power trip by closing legitimate questions.

Respectfully: outsiders like the author of this piece are not the ones entitled to decide whether a question is "legitimate", or "valid" (another term I see used all the time by people who have no understanding either of Stack Overflow's standards or its goals).

Reference reading:

What is Stack Overflow’s goal? (https://meta.stackoverflow.com/questions/254770)

How much research effort is expected of Stack Overflow users? (https://meta.stackoverflow.com/questions/261592)

Question Close Reasons - Definitions and Guidance (https://meta.stackoverflow.com/questions/417476)

How long should we wait for a poster to clarify a question before closing? (https://meta.stackoverflow.com/questions/260263)

A satirical answer to "The rudeness on Stack Overflow is too damn high" (https://meta.stackoverflow.com/questions/262791/_/309018#309...)

What is the point of closing questions for details and clarity, debugging details, needs more focus, or very low quality? (https://meta.stackoverflow.com/questions/405519)

Why should I help close "bad" questions that I think are valid, instead of helping the OP with an answer? (https://meta.stackoverflow.com/questions/429808)

Why is the rate of positively scoring questions and answers steadily declining? (https://meta.stackoverflow.com/questions/393032)

When is Stack Overflow going to stop demonizing the quality-concerned users who have made the site a success? (https://meta.stackoverflow.com/questions/366858)

Is ChatGPT and LLM killing Stack Overflow (https://meta.stackoverflow.com/questions/430994)

Re: Stack Overflow is almost dead

#102
post #85

How ironic. "AI" feeds off structured knowledge, artistic creations and otherwise any human production to generate its output. As a consequence of its widespread adoption, people start to lean even more towards consuming rather than producing, a tendency which was already increasing before the advent of LLMs and modern machine-learning. This, in turn, leaves "AI" implementations with no new human content to feed off…

> This, in turn, leaves "AI" implementations with no new human content to feed off of. Now what?

You seem to be under the impression that AI needs more than all recorded human knowledge up until 2024 to reach the same level as an average SO contributor. It doesn't. Because none of the average SO contributors did.

It is unclear what algorithmic improvements are required to leverage the available data to get AI to AGI, but a lack of data is definitely not the bottleneck.

One could say that these AI systems aren't sharing their solutions (or questions) with other AI systems and that the world would benefit from it if they did, though. Perhaps it's a good idea to have some shared space for AI systems where they share the validated solutions they synthesized.

Re: Stack Overflow is almost dead

#103
post #97
post #85

How ironic. "AI" feeds off structured knowledge, artistic creations and otherwise any human production to generate its output. As a consequence of its widespread adoption, people start to lean even more towards consuming rather than producing, a tendency which was already increasing before the advent of LLMs and modern machine-learning. This, in turn, leaves "AI" implementations with no new human content to feed off…

Perhaps the next step is having the LLMs ask questions on SO when they routinely fumble particular topics. I could see a system of knowledge bounties where people are compensated for providing accurate, in-depth training data on niche topics.

LLM content is banned everywhere on Stack Overflow, in both questions and answers, by policy, since mere days after the public announcement of ChatGPT (because it was immediately causing a huge problem): https://meta.stackoverflow.com/questions/421831

Moderators (actual elected moderators, the two dozen or so that exist for ~29 million user accounts and ~24 million non-deleted questions) went on strike in mid 2023, largely because the site staff/owners interfered with their ability to remove such content (an overwhelmingly popular policy with strong community consensus): https://meta.stackoverflow.com/questions/425000 and this decision propagated across the Stack Exchange network (as most SE sites had adopted similar policies): https://meta.stackexchange.com/questions/389811/

A large fraction of the userbase is explicitly opposed to helping LLMs out in any way whatsoever. I personally have ceased contributing new question or answer content, and only edit existing posts. I contribute new content on Codidact (https://software.codidact.com/) instead (disclosure: I have recently become a moderator there).

Re: Stack Overflow is almost dead

#104
post #33
post #17

Earlier quoted context omitted.

Where's the AI of tomorrow going to learn from if nobody is posting Q&As online anymore?

From the official docs? AI is good at sumarizing after all

I've been trying to get ChatGPT to write some Emacs Lisp for me and it sucks. Few things are better documented than Emacs. There's several hundreds of pages of documentation, and several million lines of Elisp, but apparently that's not enough.

And I'm not asking for some beautiful architecture from ChatGPT, I'm just asking for simple hacks that get the job done. Elisp is designed to make simple hacks easy, but not easy enough for ChatGPT I guess.

Like, I asked it to make a command which would move the "mark" and the "point" so that the full line was selected. If a selection covered only part of a line, I wanted the selection to expand to cover the full line. To do this, all you have to do is move the mark and the point so that they surround a line. ChatGPT couldn't do it. It would only move the point, but not the mark, it wouldn't do anything with the mark. I explicitly told ChatGPT "no, you have to set both the point and the mark correctly", and then it wrote even more code that only adjusts the point, but not the mark--it would move the point to the beginning of the line, and then move the point to the end of the line, never touching the mark--it's stupid.

Re: Stack Overflow is almost dead

#105
post #36
post #11

Earlier quoted context omitted.

They still used it, they just didn't contribute. It's not that StackOverflow is useless, but rather that it already answers most questions.

Given the speed at which tech evolves, I do not see how that can be true. I think it may answer most questions from five years ago, but I would phrase it as “it has answers for most questions, but many are outdated and there’s no easy way to tell whether an answer is outdated”

"Rank by recent votes" has been a feature for perhaps five years already.

Re: Stack Overflow is almost dead

#106
Per the post, it seems as if SO was going to going to slide beneath the surface anyway but that LLMs were one of a number of factors that maybe accelerated the process by maybe a few years. Without having deep knowledge of the area, this feels about right.

Re: Stack Overflow is almost dead

#107
post #92

Earlier quoted context omitted.

The next AI totally needs to be more snarky to make it feel more like we're dealing with actual "thinksperts", people that think they are experts even if their answers are demonstrably wrong.

"You are Comic Book Guy from The Simpsons. You are very knowledgeable in the _____ language; in fact, you believe you are the foremost expert on it. You have taken time from your busy schedule to help the unwashed masses by answering the following question..."

"Be sure to edit the original question to better fit your answer"

Re: Stack Overflow is almost dead

#108
post #85

How ironic. "AI" feeds off structured knowledge, artistic creations and otherwise any human production to generate its output. As a consequence of its widespread adoption, people start to lean even more towards consuming rather than producing, a tendency which was already increasing before the advent of LLMs and modern machine-learning. This, in turn, leaves "AI" implementations with no new human content to feed off…

> This, in turn, leaves "AI" implementations with no new human content to feed off of. Now what? You seem to be under the impression that AI needs more than all recorded human knowledge up until 2024 to reach the same level as an average SO contributor. It doesn't. Because none of the average SO contributors did. It is unclear what algorithmic improvements are required to leverage the available data to get AI to AGI,…

> You seem to be under the impression that AI needs more than all recorded human knowledge up until 2024 to reach the same level as an average SO contributor.

Replacing the average SO cobtributor isn't adequate to replace SO, and AI is able to “replace” SO effectively only since major models have gotten not only SO-as-training-data but web search (including SO) for immediate grounding.

And without SO or something like it with active human contributions it’ll have even more trouble replacing the value SO would provide for new questions and new domains where it will neither have SO traijing data nor SO query-time-search-results to use to synthesize answers.

Re: Stack Overflow is almost dead

#109
my business has a similar revenue curve, and of course LLMs are a big cause, but it's more to do with me being distracted from moving with the times. I just didn't fancy the fight and saw enough value in the curve to let it follow ranged decline

I'm assuming the owners of stack felt similar? Don't know anything about them so could be easily wrong

Re: Stack Overflow is almost dead

#110
post #94
post #80

Earlier quoted context omitted.

I think this is the first time I’ve ever heard someone describe the stack overflow community as “kind”. Usually it’s the exact opposite: “I asked a question and just got 30 questions asking why I’m trying to do this” or “my question was closed for seemingly no reason”. It’s literally the most blunt and aggressive website I’ve ever been on that wasn’t a straight-up troll site like 4-Chan.

Question closed; here's a link to another one that sounds vaguely related but doesn't actually address your problem. But seriously, I'd love to see some sentiment analysis of the SO corpus classifying tone by tag.

"Having your problem addressed" is not a valid reason to post on Stack Overflow. You are expected before posting to have done enough analysis to the point where if your question is answered, you can solve the underlying problem yourself. When you are linked to a duplicate, it's because the person doing so believes in good faith that, to the extent that you have a question that meets the site's standards, answers to the other question will answer yours as well. This also means you are responsible for overlooking irrelevant details, reading the answers, making your own attempts to apply them, etc.

If the other question is actually different, you are expected to edit the question to make this clear - not by adding an "Edit:" section like in a forum post, but by fixing the wording such that it's directly clear what you're looking for and how it's different. This might mean fixing your specification of input or desired output.

It's difficult sometimes, and curators do make mistakes. Most frustratingly, it's entirely possible for two completely different problems to be reasonably described with all the same keywords. I personally had a hell of a time disentangling https://stackoverflow.com/questions/9764298 from https://stackoverflow.com/questions/18016827, while also explaining that https://stackoverflow.com/questions/6618515 really is the same as the first problem despite different phrasing.

But curators much more often get it right. Not only that, a few of us go out of our way to create artificial Q&A (https://meta.stackoverflow.com/questions/426205) for beginner issues that beginners never know how to explain, and put immense effort into both the question and answer. Some popular examples in the Python tag:

"I'm getting an IndentationError (or a TabError). How do I fix it?" (https://stackoverflow.com/questions/45621722) was written to replace "IndentationError: unindent does not match any outer indentation level, although the indentation looks correct" (https://stackoverflow.com/questions/492387) and a few others, with reasoning stated there.

"Asking the user for input until they give a valid response" (https://stackoverflow.com/questions/23294658)

"Why does "a == x or y or z" always evaluate to True? How can I compare "a" to all of those?" (https://stackoverflow.com/questions/20002503) was written largely as an alternative to the organic "How to test multiple variables for equality against a single value?" (https://stackoverflow.com/questions/15112125) after the latter was found not to help beginners very well (the original example was quite unclear, although it's since been improved).

Post reply on HN