Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large language models reduce public knowledge sharing on online Q&A platforms

academic.oup.com

Post reply on HN
Print view View original

171–180 of 366 posts

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#171

Post by imoverclocked »

empath75 wrote: post #157
insane_dreamer wrote: post #120

The problem is eventually what are LLMs going’s to draw from? They’re not creating new information, just regurgitating and combining existing info. That’s why they perform so poorly on code for which there aren’t many many publicly available samples, SO/reddit answers etc.

AI companies are already paying humans to produce new data to train on and will continue to do that. There's also additional modalities -- they've already added text, video, and audio, and there's probably more possible. Right now almost all the content being fed into these AIs is stuff that humans can sense and understand, but why does it have to limit itself to that? There's probably all kinds of data types it coul…

> AI companies are already paying humans to produce new data to train on and will continue to do that.

The sheer difference in scale between the domain of “here are all the people in the world that have shared data publicly until now” and “here is the relatively tiny population of people being paid to add new information to an LLM” dooms the LLM to become outdated in an information hoarding society. So, the question in my mind is, “Why will people keep producing public information just for it to be devalued into LLMs?”

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#172

Post by layer8 »

finolex1 wrote: post #141
insane_dreamer wrote: post #120

The problem is eventually what are LLMs going’s to draw from? They’re not creating new information, just regurgitating and combining existing info. That’s why they perform so poorly on code for which there aren’t many many publicly available samples, SO/reddit answers etc.

There is still publicly available code and documentation to draw from. As models get smarter and bootstrapped on top of older models, they should need less and less training data. In theory, just providing the grammar for a new programming language should be enough for a sufficiently smart LLM to answer problems in that language. Unlike freeform writing tasks, coding also has a strong feedback loop (i.e. does the cod…

> In theory, just providing the grammar for a new programming language should be enough for a sufficiently smart LLM to answer problems in that language.

I doubt it. Take a language like Rust or Haskell or even modern Java or Python. Without prolonged experience with the language, you have no idea how the various features interact in practice, what the best practices and typical pitfalls are, what common patterns and habits have been established by its practitioners, and so on. At best, the system would have to simulate building a number of nontrivial systems using the language in order to discover that knowledge, and in the end it would still be like someone locked in a room without knowledge of how the language is actually applied in the real world.

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#173

Post by stemlord »

elicksaur wrote: post #155
PhilipRoman wrote: post #55

Earlier quoted context omitted.

I haven't really found stackoverflow to be that humiliating (compared to some IRC rooms or forums), basic questions get asked and answered all the time. But the worst part is when you want to do something off the beaten path. Q: how do I do thing X in C? A: Why do you need to know this? The C standard doesn't say anything about X. The answer will depend on your compiler and platform. Are you sure you want to do X ins…

On the other hand, I find it to be a fatal flaw that LLMs can’t say, “Hey you probably don’t actually want to do it that way.”

Sure but most common LLMs aren't going to be patronizing and presumptuous while they say so

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#174

Post by timhh »

manojlds wrote: post #45
timhh wrote: post #8

Stackoverflow mods and power users being arseholes reduces the use of Stackoverflow. ChatGPT is just the first viable alternative.

Easy to keep saying this, but SO was useful because it wasn't wild west.

I disagree. It was useful because the UI was (and is!) great. Easy to use markdown input, lists of answers sorted by votes, very limited ads, etc. The gamification was also well done.

Compared to anything before it (endless phpBB forums, expertsexchange, etc.) it was just light years ahead.

Even today compared the SO UI with Quora. It's still 10x better.

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#175

Post by skywhopper »

PhilipRoman wrote: post #55
melenaboija wrote: post #38

It’s been a relief to find a platform where I can ask questions without the fear of being humiliated Half joking, but I am pretty tired of SO pedantry.

I haven't really found stackoverflow to be that humiliating (compared to some IRC rooms or forums), basic questions get asked and answered all the time. But the worst part is when you want to do something off the beaten path. Q: how do I do thing X in C? A: Why do you need to know this? The C standard doesn't say anything about X. The answer will depend on your compiler and platform. Are you sure you want to do X ins…

I mean, those all sound like good questions. You might be a super genius, but most people who ask how to do X actually want to do Y. And if they DO want X, then those other questions about compiler and OS version really matter. The fact that you didn’t include them in your question shows you aren’t really respecting the time of the experts on the platform. If you know you are doing something unusual, then you need to provide a lot more context.

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#176

Post by stemlord »

LeadB wrote: post #57
melenaboija wrote: post #38

It’s been a relief to find a platform where I can ask questions without the fear of being humiliated Half joking, but I am pretty tired of SO pedantry.

For the major programming languages, it must be a pretty esoteric question if it does not have an answer yet. Increasingly, the free products of experts are stolen from them with the pretext that "users need to be protected". Entire open source projects are stolen by corporations and the experts are removed using the CoC wedge. Now SO answers are stolen because the experts are not trained like hotel receptionists (wh…

Hm fair point. Rudeness is actually a sign of humanity. Like that one black mirror episode

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#177

Post by shagie »

simonw wrote: post #71
jmyeet wrote: post #30

Earlier quoted context omitted.

> These LLMs are polymaths that can spit out content at a super human rate. Do you mean in theory or currently? Because currently, LLMs make simple errors (eg [1]) and are more capable of spitting out, well, nonsense. I think it's safe to say we're a long way from LLMs producing anything creatively good. I'll put it this way: you won't be getting The Godfather from LLMs anytime soon but you can probably get an indust…

Both the "count the Rs in strawberry" and the "multiply two large numbers" things have been solved for over a year now by the tool usage pattern: give an LLM the ability to delegate to a code execution environment for things it's inherently bad at and train it how to identify when to use that option.

(I did an earlier attempt at this with a "ok, longer conversation" ... and then did a "well, what if I just asked it directly?")

https://chatgpt.com/share/670bfdbd-8624-8011-bc31-2ba66eab3e...

I didn't realize that it had come that far with the delegating of those problems to the code writing and executing part of itself.

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#178

Post by ravenstine »

bqmjjx0kac wrote: post #92
haolez wrote: post #72

Earlier quoted context omitted.

The first time that I asked a question on #cpp @Freenode was a unique experience for my younger self. My message contained greetings and the question in the same message. I was banned immediately and the response from the mods was: - do not greet; we don't have time for that bullshit - do not use natural language questions; submit a test case and we will understand what you mean through your code - do not abbreviate…

> do not use natural language questions That is really absurd! AFAIK, it is not possible to pose a question to a human in C++. This level of dogmatism and ignorance of human communication reminds me of a TL I worked with once who believed that their project's C codebase was "self-documenting". They would categorically reject PRs that contained comments, even "why" comments that were legitimately informative. It was a…

Self-documenting code is one of the worst ideas in programming. Like you, I've had to work with teams where my PRs would be blocked until I removed my comments. I'm not talking pointless comments like "# loop through the array" but JSdoc style comments describing why a function was needed.

I will no longer work anywhere that has this kind of culture.

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#179

Post by klabb3 »

joshdavham wrote: post #88
verdverm wrote: post #64

For me, many of my questions about open source projects have moved to GitHub and Discord, so there is platform migration besides LLMs. I also tend to start with Gemini for more general programming things, because it will (1) answer in the terms of my problem instead of me having to visit multiple pages to piece it together, or (2) what it's wrong, I often get better jump off points when searching. Either way, LLMs sa…

> many of my questions about open source projects have moved to GitHub and Discord Exact same experience here. Plus, being able to talk to maintainers directly has been great!

No doubt that discord has struck a good balance. Much better than GitHub imo. Both for maintainers to get a soft understanding of their users, and equally beneficial for users who can interact casually without being shamed for filing an issue the wrong way.

There’s some weird blind spot with techies who are unable to see the appeal. UX matters in a “the medium is the message”-kind of way. Also, GitHub is only marginally more open than discord. It’s indexable at the moment, yes, but would not surprise me at all if MS is gonna make an offensive move to protect “their” (read our) data from AI competitors.

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#180

Post by lynx23 »

Vegenoid wrote: post #134
melenaboija wrote: post #38

It’s been a relief to find a platform where I can ask questions without the fear of being humiliated Half joking, but I am pretty tired of SO pedantry.

Yeah, Stackoverflow kinda dug their own grave by making their platform and community very unpleasant to engage with.

Well, I believe the underlying problem of platforms like StackOverflow, ticketing systems (in-house and public) and even CRMs is not really solvable. The problem is, the quality of an answer is actually not easy to determine. All the mechanisms we have are hacks, and better solutions would need more resources... which leads to skewed incentives, and ultimately to a "knwoledge" db thats actually not very good. People are incentiviszed to collect karma points, or whatever it is. But these metrics are not really resembling the quality of their work... Crowdsourcing this mechanisms via upvotes or whatever does also not really work, because quantity is not quality... As said, I believe this is a problem we can not solve.
Post reply on HN