With that being said, I imagine the quality of the questions have also improved quite a bit. I definitely don’t condone the rude behaviour on SO, but I also understand that the site used to be bombarded constantly with low quality questions that now thankfully LLMs can handle.
Large language models reduce public knowledge sharing on online Q&A platforms
91–100 of 366 posts
Re: Large language models reduce public knowledge sharing on online Q&A platforms
#92Earlier quoted context omitted.
I haven't really found stackoverflow to be that humiliating (compared to some IRC rooms or forums), basic questions get asked and answered all the time. But the worst part is when you want to do something off the beaten path. Q: how do I do thing X in C? A: Why do you need to know this? The C standard doesn't say anything about X. The answer will depend on your compiler and platform. Are you sure you want to do X ins…
The first time that I asked a question on #cpp @Freenode was a unique experience for my younger self. My message contained greetings and the question in the same message. I was banned immediately and the response from the mods was: - do not greet; we don't have time for that bullshit - do not use natural language questions; submit a test case and we will understand what you mean through your code - do not abbreviate…
> do not use natural language questions
That is really absurd! AFAIK, it is not possible to pose a question to a human in C++.
This level of dogmatism and ignorance of human communication reminds me of a TL I worked with once who believed that their project's C codebase was "self-documenting". They would categorically reject PRs that contained comments, even "why" comments that were legitimately informative. It was a very frustrating experience, but at least I have some anecdotes now that are funny in retrospect.
Re: Large language models reduce public knowledge sharing on online Q&A platforms
#93The authors claim that LLM are reducing public knowledge sharing and that the effect is not merely displacing duplicate, low-quality, or beginner-level content.
However their claim is weak and the effect is not quite as sensational as they make it sound.
First, they only present Figure 3 and not regression results for their suggested tests of LLMs being substitutes of bad quality posts. In contrast, they report tests for their random qualification by user experience (where someone is experienced if they posted 10 times). Now, why would they omit tests by post quality but show results by a random bucketing of user “experience”?
Second, their own Figure 3 “shows” a change in trends for good and neutral questions. Good questions were downtrending and now they are flat, and neutral questions (arguably the noise) went from an uptrend to flat. Bad question continue to go down, no visible change in the trend. This suggests the opposite, ie that LLMs are in fact substituting bad quality content.
I feel the conclusion needed a stronger statement and research doesn’t reward meticulous but unsurprising results. Hence the sensational title and the somewhat redacted results.
Re: Large language models reduce public knowledge sharing on online Q&A platforms
#94Stackoverflow mods and power users being arseholes reduces the use of Stackoverflow. ChatGPT is just the first viable alternative.
Easy to keep saying this, but SO was useful because it wasn't wild west.
It was useful and not the wild west as long as a very small group of intelligent and highly motivated individuals moderated it. First and foremost Jeff Atwood used to do a lot of moderation himself - not unlike dang on HN.
When that stopped, the site (and to some degree its growing number of sister sites) continued on its ballistic curve, slowly but continuously descending into the abyss.
My primary take away is that we have not found a way to scale moderation. SO was doomed anyway, LLMs have just sped up that process.
Re: Large language models reduce public knowledge sharing on online Q&A platforms
#95I have no idea where to ask questions nowadays. Stackoverflow is way "too slow" (Go to website, write a nice well formatted thread, wait for answers). But there's way faster solutions now, namely from message groups.
For example, I was wondering if its okay to move my home directory to a different filesystem altogether and create a symlink from /home/. Where do I ask such questions? The freaking ZFS mailing list? SO? It was just a passerby question, and what I wanted more than the answer is the sense of community.
The only place that I know that have a wide enough range of interest, with many people that each know some of these stuff quite deep, is public, is easily accessible is unironically 4chan /g/.
I would rather go there then Discord where humanity's knowledge will be piped to /dev/null.
Re: Large language models reduce public knowledge sharing on online Q&A platforms
#96I have no idea where to ask questions nowadays. Stackoverflow is way "too slow" (Go to website, write a nice well formatted thread, wait for answers). But there's way faster solutions now, namely from message groups. For example, I was wondering if its okay to move my home directory to a different filesystem altogether and create a symlink from /home/. Where do I ask such questions? The freaking ZFS mailing list? SO?…
I guess I'm out of the loop. What does "/g/" mean?
Re: Large language models reduce public knowledge sharing on online Q&A platforms
#97It’s been a relief to find a platform where I can ask questions without the fear of being humiliated Half joking, but I am pretty tired of SO pedantry.
I haven't really found stackoverflow to be that humiliating (compared to some IRC rooms or forums), basic questions get asked and answered all the time. But the worst part is when you want to do something off the beaten path. Q: how do I do thing X in C? A: Why do you need to know this? The C standard doesn't say anything about X. The answer will depend on your compiler and platform. Are you sure you want to do X ins…
My questions always get closed and marked as a duplicate with a comment linking to a question that's unrelated
Re: Large language models reduce public knowledge sharing on online Q&A platforms
#98Earlier quoted context omitted.
I think this is more about a drop in questions, than a drop in answers.
I mean part of the reason to not ask about stuff on SO, there are several types of questions that one might like to ask - such as: I don't know the first thing about this thing, help me get to where I know the first thing. This is not allowed any more. I want to know the pros and cons of various things compared. this is not allowed. I have quality questions regarding an approach that I know how to do, but I want to k…
I can see how frustrating it might be, but the overall idea of SO is "no duplicates". They don't want to have 1000 questions which are exactly the same but with slightly different phrasing. It can be problematic for total newcomers, but at the same time it makes it more useful for professionals: instead of having 1000 questions how to X with 1 reply, you have one canonical question with 20 replies sorted by upvotes and you can quickly see which one is likely the best.
FWIW, I found LLMs to be actually really good at those basic questions where I'm at expert at language X and I ask how to do similar thing in Y, using Y's terms (which might be named differently in X).
I believe this actually would work well:
- extra basic things, or things that depend on opinion etc: ask LLMs and let they infer and steer you
- advanced / off the beaten path questions that LLMs hallucinate on: ask on SO
Re: Large language models reduce public knowledge sharing on online Q&A platforms
#99Earlier quoted context omitted.
I haven't really found stackoverflow to be that humiliating (compared to some IRC rooms or forums), basic questions get asked and answered all the time. But the worst part is when you want to do something off the beaten path. Q: how do I do thing X in C? A: Why do you need to know this? The C standard doesn't say anything about X. The answer will depend on your compiler and platform. Are you sure you want to do X ins…
The first time that I asked a question on #cpp @Freenode was a unique experience for my younger self. My message contained greetings and the question in the same message. I was banned immediately and the response from the mods was: - do not greet; we don't have time for that bullshit - do not use natural language questions; submit a test case and we will understand what you mean through your code - do not abbreviate…
That contradiction is funny, tho:
> - do not greet; we don't have time for that bullshit
and
> do not abbreviate words (you have abbreviated "you" as "u"); if you do not have time to type the words, we do not have time to read them
So they have apparently enough time to read full words, it seems!
Re: Large language models reduce public knowledge sharing on online Q&A platforms
#100Earlier quoted context omitted.
The main issue with Stack Overflow (and similar public Q&A platforms) is that many contributors do not know what they do not know, leading to inaccurate answers. Additionally, these platforms tend to attract a fair amount of spam (self promotion etc) which can make it very hard to find high-quality responses.
I've gotten answers from OpenAI that were technically correct but quite horrible in the longer term. I've gotten the same kinds of answers on Stack Overflow, but there other people are eager to add the necessary feedback. I got the same feedback from an LLM but only because in that case I knew enough to ask for it. Maybe we can get this multi-headed advantage back from LLMs by applying a team of divergent AIs to the…
You can usually even ask the same LLM:
- do a task
- criticize your job on that task
- redo that task based on criticism
I find giving the LLM a process greatly improves the results.