Live data from Hacker News

Large language models reduce public knowledge sharing on online Q&A platforms

academic.oup.com

331–340 of 366 posts

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#331

People don't post where they aren't visiting. They're not visiting Stack Overflow for well-read material (the popular languages) because perplexity.ai, ChatGPT, Claude, etc., not only answer questions better than reading pages of StackOverflow, but will cut and paste the (right or wrong) answer for you faster. If you're not on StackOverflow while asking questions, you aren't answering things there either. Nothing els…

Competing with AI is an explicit non-goal of the Stack Overflow community, as well as the one other Q&A forum I use (Codidact).

AI won't "answer questions better". It will cut out the middleman of parsing your question and matching it up with words in the shape of an answer. It will also frequently hallucinate, and essentially never sanity-check what you're trying to do.

But the main thing giving it a speed/convenience advantage over Q&A forums is that it doesn't give a damn about whether your question, and its answer, could help someone else (by being discoverable with a search engine and understood by that other person as being the same question, and being focused on a single issue). Aside from not being designed to care, it wouldn't benefit: it can just re-generate the same answer content for the next person who asks, in a different low-quality way. Unlike a human expert, AI will never tire of that task.

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#332
post #72

Earlier quoted context omitted.

The first time that I asked a question on #cpp @Freenode was a unique experience for my younger self. My message contained greetings and the question in the same message. I was banned immediately and the response from the mods was: - do not greet; we don't have time for that bullshit - do not use natural language questions; submit a test case and we will understand what you mean through your code - do not abbreviate…

This being HN, I'd love to hear from one of the many IRC channel mods who literally typed (I'd guess copy/pasted) this kind of text into their chat room topics and auto-responders. If you're out there-- how does it feel to know that what you meant as a efficient course-correction for newcomers was instead a social shaming that cut so deep that the message you wrote is still burned verbatim into their memory after all…

>If you're out there-- how does it feel to know that what you meant as a efficient course-correction for newcomers was instead a social shaming that cut so deep that the message you wrote is still burned verbatim into their memory after all these years?

I've never been a IRC mod for a channel of anywhere near that significance. But as someone who has put in considerable effort for the last few years trying to explain basic fundamental ideas about what Stack Overflow is and how it's supposed to work to people (including to people whose accounts are 15+ years old but who insist on continuing to treat the Q&A like a discussion forum)...

... I'm kinda envious.

Being presented with a text entry box, and the implied contract that what you type there will be broadcast to a wide audience, is not supposed to imply a right to reject the community's telos and substitute your own. Gatekeeping of this sort is important; otherwise you end up with the Wikipedia page for "Dog" being flooded with people trying to get free veterinary consultation. Communities are allowed to have goals and purposes that aren't obvious and which don't match the apparent design, and certainly they aren't required to have goals which encompass everything possible with the sites software.

There are countless places on the Internet that work like a discussion forum where you can post a question, or a request for help, or just a general state of confusion, about a problem where you're just completely lost; and where you can expect to have a back-and-forth multi-way conversation with others to try and diagnose things or come to a state of understanding; and where nobody cares if your real concern ever surfaces in the form of an explicit, coherent question; and where there's no expectation that any of this dialogue should ever be useful to anyone else.

Stack Overflow is not that, explicitly and by design; and that came about specifically so that people who do have some clue and want to find an answer to an actual question, can do so without having to pick through a dialogue of the sort described above, following an arbitrarily long chain of posts of questionable relevance (that might well end in "never mind, I fixed it" with no explanation).

But in order to become a repository of such questions, people presented with the question submission form... need to be restricted to asking such questions.

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#333

The problem is eventually what are LLMs going’s to draw from? They’re not creating new information, just regurgitating and combining existing info. That’s why they perform so poorly on code for which there aren’t many many publicly available samples, SO/reddit answers etc.

> The problem is eventually what are LLMs going’s to draw from? Published documentation. I'm going to make up a number but I'll defend it: 90% of the information content of stackoverflow is regurgitated from some manual somewhere. The problem is that the specific information you're looking for in the relevant documentation is often hard to find, and even when found is often hard to read. LLMs are fantastic at reading…

> regurgitated from some manual somewhere

yes, a lot of stuff is like that, and LLMs are a good replacement for searching the docs; but the most useful SO answers are regarding issues that are either not documented or poorly documented, and someone knows because they tried it and found what did or did not work

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#334

Earlier quoted context omitted.

There are a great many problems with LLM's and AI in general. The underlying problem is, LLMs break the social contract in about as many ways as a human can dream up. There is no useful beneficial purpose that doesn't also open the door to intractable destructive forces manifold over that benefit; its a modern day pandora's box, and its hubris to think otherwise. This underlying societal problem has knock-on effects…

> There are types of people who will just take until no more can be given which is a type of evil, which is why you need to vet people beforehand but an LLM is simply a programmatic implementation of taking. Your first argument largely rests on this, but isn't the fact that so many LLM-created answers get posted actually proof of the opposite? The LLM is, after all, "giving" here - and it's not like those answers are…

If the latter were true, then we would have seen the demand increase for all those specialized fields from graduates, but we have seen its just like what it was, no jobs and more debt, Older people fulfilling roles, until they retire or die and positions that aren't getting filled despite advertising for months to years. My generation will be dead within that time period, you should see at least some replacement by now but if you look closely, there isn't much replacement going on at all because there's no money in it. Greybeards are the only minders left running many of these IT departments if they are left at all and not completely outsourced.

For the amount of responsibility to get things right, you now have jobs that have significantly less responsibilities or entry skills that are now competing with the same positions thanks to inflation. Who would bother signing up for that stress for no economic benefit when they can get any number of other jobs at 3/4 of the pay and none of the responsibility. This type of chaos is what you've discounted as not happening (ECP).

> The economic calculation problem ... is solved

Then why are we having chaotic disruptions, in growing frequency and amplitude, continually more concentrated business sectors based in Lenin's comments on attacking capitalism through debauching currency through inflation as he mentioned to Keynes (towards non-market socialism), and more importantly shortages that are starting to self-sustain at the individual level.

When common food goods are not getting replenished on a weekly basis, and they are 2-3x the normal price, there is an issue. When it hits 2-3 weeks it starts to compound as demand skyrockets for non-discretionary goods (and they can't keep up). Goods sell out almost immediately (as they have been more recently), and people start hoarding (which they are doing again with TP).

You don't seem to realize this is the problem in action. It is hardly solved, we aren't even to the non-market part yet and we are far enough along to see that chaos is increasing, distortions are increasing, and shortages are increasing and not being corrected within JIT shipping schedules.

Those three observations are cause for concern to stop and lookaround because chaos is a major characteristic of the ECP. Moving goods around doesn't perform economic calculation, and soviet-style demand planning failed historically once shortages occurred, the next things coming up in the near future is sustaining shortage and death from famine. India stopped exporting its food because they see this on the horizon and have a intimate history with famine.

The 1921 famine (bolshevik non-market) reduced national population by 4% YOY and dropped the birthrate negative; Mao killed off more than 33+ million during his non-market stint which caused famine. No accurate numbers to draw up a percentage YOY but it was significant.

Every single non-market socialist state has failed in the history of socialism, and that includes its derivatives. Its not appropriate to call it ideology when there is a historic record where not even a single non-market state exists today. The fact that they all had to have markets tied to capitalist states to get their prices shows the gravity of the issue, and now that the last capitalist state is falling to non-market socialism (through intentional design/sabotage), what do you suppose is going to happen to all those market-socialist countries that can no longer get their price data. Its already happening, and chaos is unpredictable; and inflation/deflation measures are limited in resolution (as a high hysteresis problem).

Non-market socialist states run into the slavery and death part, but you are not recognizing that, and discounting it because it hasn't happened to us yet (though its likely within 5 years for a point of no return), and it has happened to every non-market system.

The point of no return to non-market socialism is when debt growth exceeds GDP (outflows exceed inflows). This is structural of any ponzi scheme. Who decides, the producers. This is when rational producers abandon the market, and start shutting down because its no longer possible to make a profit in terms of purchasing power. From there its a short but inevitable jaunt to collapse.

Have you noticed that China's stimulus through printing isn't doing much? Every printing requires exponentially greater amounts, with diminishing returns.

I think you've sorely confused objective observation with ideology and discounted it without taking into account the factors rationally; these are factors showing a clear and present danger to all existing market-socialist states, as well as those capitalist bastions that will be pulled down in the near future from ongoing subversion.

Amazon doesn't produce, its a logistics company. Shortages self-sustain because producers are leaving the market, we saw this first with baby formula in 2020 when the last producer was closed, but what's coming will leave only state-run capacity for most sectors and industry.

Most knowledgeable people know these systems are brittle and parasitic and aren't capable of ECP for a myriad of already written and rational reasons by themselves.

Socialist production absent a market, doesn't work without slave labor. It doesn't even work longer-term either with slave labor because of inefficiencies at current population don't scale well with tech levels.

You might be fine claiming everything is fine now, despite a lot of evidence to the contrary that things are getting progressively worse, and will continue down that path because of these consequences, but what about in 5 years when you can't get food. Will it matter then, when you can't do anything about it, because the time to do something came and passed?

You think the Elites in China don't know this is going to happen? They are pushing so hard for the BRICS because they know this is going to happen, because they helped cause it in their dogmatic and blind fervor and orchestration.

Name one modern real non-market socialist state that exists today that has not failed, and I'll be willing to consider your last statement. As far as I know, none exist and that should frighten every single person today because that is where we are going whether we like it or not (as a function of system dynamics).

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#335

It’s been a relief to find a platform where I can ask questions without the fear of being humiliated Half joking, but I am pretty tired of SO pedantry.

Many of the forums I enjoyed in the past have become heavily burdened by rules, processes, and expectations. They are frequented by people who spend hours every day reading everything and calling out any misstep. Some of them are so overburdened that navigating all of the rules and expectations becomes a skill in itself. A single innocent misstep turns simple questions into lectures about how you’ve violated the rule…

Just go to a different community for the same topic ?

Sounds like that one is on its way out anyway...

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#336

Earlier quoted context omitted.

I haven't really found stackoverflow to be that humiliating (compared to some IRC rooms or forums), basic questions get asked and answered all the time. But the worst part is when you want to do something off the beaten path. Q: how do I do thing X in C? A: Why do you need to know this? The C standard doesn't say anything about X. The answer will depend on your compiler and platform. Are you sure you want to do X ins…

Yes, immature people are everywhere, but SO took it to a new level before they had to implement a code of conduct. I remember asking questions and getting "this is is common misconception, maybe you're looking for X instead" type of actually helpful and kind answers. After some point it came to a point that if you're not asking a complete problem which can't be modeled as a logic statement, you're labeled as stupid f…

Stack Overflow always had a code of conduct - it just wasn't always called that, and it started out based in some naive assumptions about human behaviour.

Someone who tells you something along the lines of "this is is common misconception, maybe you're looking for X instead" is being helpful and kind, and is not at all "labeling you as stupid". On Stack Overflow, it's not at all required that you actually need the question answered, as asked, in order to ask it. (You're even actively encouraged to ask and answer your own questions, as long as both question and answer meet the usual standards, as a way to share your expertise.)

It has never been acceptable to insult others openly on the site - but moderation resources are, and always have been, extremely strained, and the community simply doesn't see certain things as insulting or "mean" that others might. In particular, downvotes aren't withheld out of sympathy, because they are understood to be purely for content rating - but many users take them personally anyway. Curators commonly get yelled at when they comment (thus identifying themselves) to explain policy in polite but blunt copy-paste terms; and they get seethed at when they don't comment. There is a standard set of reasons to close questions (https://meta.stackoverflow.com/questions/417476), and a how-to-ask guide (https://stackoverflow.com/help/how-to-ask), and a site tour (https://stackoverflow.com/tour), and an entire Meta site with a variety of FAQ entries and other common references (for example, I wrote the "proposed faq" of https://meta.stackoverflow.com/questions/429808); but very few people seem interested in checking the extant documentation for the community norms, and then feel slighted when the community norms aren't what they assume they ought to be (because it works that way everywhere else). Communities should be allowed to have their own norms, so that they can pursue their own goals (https://meta.stackoverflow.com/questions/254770).

A ton of users confuse "know better or find the answer ahead of time" for debugging. Stack Overflow allows for what are commonly called "debugging questions", but this does not mean questions wherein you ask someone to debug the code for you. Why? Because that can't ever be helpful to someone else - nobody else has your code, and thus they can't benefit from someone locating the bug in your code. The "debugging questions" that Stack Overflow does want are questions about behaviour that isn't understood, after you have isolated the misbehaving part. These are ideally presented in the form of a "minimal reproducible example" or MRE (https://stackoverflow.com/help/minimal-reproducible-example).

Stack Overflow explicitly expects you to do research before asking your question (https://meta.stackoverflow.com/questions/261592) - because most of the goal of the site is to cover questions that can't be answered that way. Traditional reference documentation is only part of the puzzle (https://diataxis.fr/); the Q&A site format allows for explanations (for "debugging questions") and how-to guides (in response to simple queries about how to do some atomic, well-specified task). (Tutorials don't fit in this format because they don't start with a clear question from the student.)

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#337

Earlier quoted context omitted.

I haven't really found stackoverflow to be that humiliating (compared to some IRC rooms or forums), basic questions get asked and answered all the time. But the worst part is when you want to do something off the beaten path. Q: how do I do thing X in C? A: Why do you need to know this? The C standard doesn't say anything about X. The answer will depend on your compiler and platform. Are you sure you want to do X ins…

My questions always get closed and marked as a duplicate with a comment linking to a question that's unrelated

A large majority of the complaints we get of this form on the Meta site, whenever there's an actual object case, turn out to be really clear-cut duplicates where the OP is quibbling on an irrelevant detail instead of trying to understand the explanation in the answer and adapt the code to individual circumstances.

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#338
post #67

Earlier quoted context omitted.

The main issue with Stack Overflow (and similar public Q&A platforms) is that many contributors do not know what they do not know, leading to inaccurate answers. Additionally, these platforms tend to attract a fair amount of spam (self promotion etc) which can make it very hard to find high-quality responses.

What you mention has been serious from day one indeed. But to me the worst issue is it's now "Dead Overflow": most answers are completely, totally and utterly outdated. And seen that they made the mistake of having the concept of an "accepted answer" (which should never have existed), it only makes the issue worse. If it's a question about things that don't change often, like algorithms, then it's OK. But for anythin…

I agree that "accepted answer" is a misfeature; but the default is for questions to remain open to new answers indefinitely.

On the one hand, good new answers can remain buried under mediocre and outdated answers more or less indefinitely, because they received hundreds of questionable upvotes in the past. (Despite popular perception, the average culture of Stack Overflow is very heavily weighted towards upvoting. I have statistical analysis of this on the meta site somewhere.)

On the other hand, there are tons of popular reference questions that had to be "protected" so that people couldn't just sign up and contribute the 100th answer (yes, the count really does range into triple digits in several cases, especially if you have access to deleted answers) to a question where there are maybe five actually reasonable-considered-distinct answers possible. And this protection is a quite low barrier - 10 reputation, IIRC. (I'm not sure what motivates people to write these new answers. It might have something to do with being able to point to a Stack Overflow profile which boasts that you've "reached" millions of people, due to its laughably naive algorithm for that.)

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#339
Note for those suffering from deja vu: this was extensively highlighted and touted by critics back in July 2023 as "Are Large Language Models a Threat to Digital Public Goods? Evidence from Activity on Stack Overflow" https://arxiv.org/abs/2307.07367 including HN: https://news.ycombinator.com/item?id=36763718

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#340

It’s been a relief to find a platform where I can ask questions without the fear of being humiliated Half joking, but I am pretty tired of SO pedantry.

Many of the forums I enjoyed in the past have become heavily burdened by rules, processes, and expectations. They are frequented by people who spend hours every day reading everything and calling out any misstep. Some of them are so overburdened that navigating all of the rules and expectations becomes a skill in itself. A single innocent misstep turns simple questions into lectures about how you’ve violated the rule…

Stack Overflow's policies exist for clearly documented reasons that are directly motivated by a clearly documented purpose (which happens not to be aligned with what most new users want the site to be, but they aren't the ones who should get to decide what the site is). They are certainly not politically motivated or created for the sake of a power trip. They came out of years of discussion on the meta site, and have changed over time due to the community realizing that previous iterations missed the mark.
Post reply on HN