Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large language models reduce public knowledge sharing on online Q&A platforms

academic.oup.com

Post reply on HN
Print view View original

71–80 of 366 posts

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#71

Post by simonw »

jmyeet wrote: post #30
vlovich123 wrote: post #12

Earlier quoted context omitted.

> At some point it won't be necessary to train an LLM on the entirety of Reddit plus everything else ever written because there are obvious limits to statistical models like this and, as a counter point, that's not how humans learn. You may have read hundres of books in your life, maybe even thousands. You haven't read a million. You don't need to. I agree but I think it may be privileging the human intelligence mech…

> These LLMs are polymaths that can spit out content at a super human rate. Do you mean in theory or currently? Because currently, LLMs make simple errors (eg [1]) and are more capable of spitting out, well, nonsense. I think it's safe to say we're a long way from LLMs producing anything creatively good. I'll put it this way: you won't be getting The Godfather from LLMs anytime soon but you can probably get an indust…

Both the "count the Rs in strawberry" and the "multiply two large numbers" things have been solved for over a year now by the tool usage pattern: give an LLM the ability to delegate to a code execution environment for things it's inherently bad at and train it how to identify when to use that option.

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#72

Post by haolez »

PhilipRoman wrote: post #55
melenaboija wrote: post #38

It’s been a relief to find a platform where I can ask questions without the fear of being humiliated Half joking, but I am pretty tired of SO pedantry.

I haven't really found stackoverflow to be that humiliating (compared to some IRC rooms or forums), basic questions get asked and answered all the time. But the worst part is when you want to do something off the beaten path. Q: how do I do thing X in C? A: Why do you need to know this? The C standard doesn't say anything about X. The answer will depend on your compiler and platform. Are you sure you want to do X ins…

The first time that I asked a question on #cpp @Freenode was a unique experience for my younger self.

My message contained greetings and the question in the same message. I was banned immediately and the response from the mods was:

- do not greet; we don't have time for that bullshit

- do not use natural language questions; submit a test case and we will understand what you mean through your code

- do not abbreviate words (you have abbreviated "you" as "u"); if you do not have time to type the words, we do not have time to read them

The ban lasted for a week! :D

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#73

Post by tomrod »

verdverm wrote: post #69
rkncland wrote: post #31

Earlier quoted context omitted.

ChatGPT plagiarizes the anwers of those whom you call "arseholes". How is using Stackoverflow in read-only mode different from using ChatGPT? Except of course that reading Stackoverflow directly has better retention rates, better explanations and more in-depth discussions. (My view is that moderators can be annoying but the issue is overblown.)

Plagiarizing means violating copyright, loosely speaking. When you, as a human, use SO, you assign your rights to the content to SO. That company is licensing the content to 3rd parties, including those who want to train their LLMs. What I find is that the LLMs are not spitting out SO text word for word, as one would when plagiarizing. Rather, the LLM uses the context and words of my question when answering, making t…

I thought plagarizing was producing new work substantially copied from prior work, regardless who owns the copyright? I thought this because self-plagarizing exists.

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#74

Post by falcor84 »

jmyeet wrote: post #5

It's a losing battle to try and maintain walled gardens for these corpuses of human-generated text that have become valuable to train LLMs. The horse has probably already bolted. I see this as a temporary problem however because LLMs are transitional. At some point it won't be necessary to train an LLM on the entirety of Reddit plus everything else ever written because there are obvious limits to statistical models l…

> that's not how humans learn

I've been thinking about this a lot lately. Could we train an AI, e.g. using RL and GAN, where it gets an IT task to perform based on a body of documentation, such that its fitness would then be measured based on both direct success on the task, and on the creation of new (distilled and better written) documentation that would allow an otherwise context-less copy of itself to do well on the task?

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#75

Post by krtalc »

simonw wrote: post #53
optimiz3 wrote: post #15

If a site aims to commoditize shared expertise, royalties should be paid. Why would anyone willingly reduce their earning power, let alone hand away the right for someone else to profit from selling their knowledge, unattributed no less. Best bet is to book publish, and require a license from anyone that wants to train on it.

... you just shared your expertise here on Hacker News in the form of this comment without any expectation of royalties. How is posting on StackOverflow different?

One could answer that question to people whose salary does not depend upon not understanding the answer.

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#76

Post by vlovich123 »

simonw wrote: post #71
jmyeet wrote: post #30

Earlier quoted context omitted.

> These LLMs are polymaths that can spit out content at a super human rate. Do you mean in theory or currently? Because currently, LLMs make simple errors (eg [1]) and are more capable of spitting out, well, nonsense. I think it's safe to say we're a long way from LLMs producing anything creatively good. I'll put it this way: you won't be getting The Godfather from LLMs anytime soon but you can probably get an indust…

Both the "count the Rs in strawberry" and the "multiply two large numbers" things have been solved for over a year now by the tool usage pattern: give an LLM the ability to delegate to a code execution environment for things it's inherently bad at and train it how to identify when to use that option.

I think the point is that playing whack a mole is an effective practical strategy to shore up individual weaknesses (or even classes of weaknesses) but that doesn’t get you to general reasoning unless you think that intelligence evolved this way. Given the adaptability of intelligence across the animal kingdom to novel environments never seen before, I don’t think that can be anything other than a short term strategy for AGI.

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#77

Post by cjauvin »

tlogan wrote: post #67
melenaboija wrote: post #38

It’s been a relief to find a platform where I can ask questions without the fear of being humiliated Half joking, but I am pretty tired of SO pedantry.

The main issue with Stack Overflow (and similar public Q&A platforms) is that many contributors do not know what they do not know, leading to inaccurate answers. Additionally, these platforms tend to attract a fair amount of spam (self promotion etc) which can make it very hard to find high-quality responses.

I find that LLMs are precisely that: marvelous engines to explore "what you don't know that you don't know", about anything.

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#78

Post by 0x1ceb00da »

simonw wrote: post #65
0x1ceb00da wrote: post #46

Earlier quoted context omitted.

> You may have read hundres of books in your life, maybe even thousands. You haven't read a million. You don't need to. Sometimes online forums are the only place where you can find solutions for niche situations and edge cases. Tricks which would have been very difficult to figure out on your own. LLMs can train on the official documentation of tools l/libraries but they can't experiment and figure out solutions to…

"LLMs can train on the official documentation of tools l/libraries but they can't experiment and figure out solutions to weird problems" LLMs train on way more than just the official documentation: they train on the code itself, the unit tests for that code (which, for well written projects, cover all sorts of undocumented edge-based) and - for popular projects - thousands of examples of that library being used (and…

> they train on the code itself, the unit tests for that code

If understanding the code was enough, we wouldn't have any bugs or counterintuitive behaviors.

> and - for popular projects - thousands of examples of that library being used (and unit tested) "in the wild".

If people stopped contributing to forums, we won't have any such data for new things that are being made.

Re: Large language models reduce public knowledge sharing on online Q&A platforms

#79

Post by malicka »

optimiz3 wrote: post #15

If a site aims to commoditize shared expertise, royalties should be paid. Why would anyone willingly reduce their earning power, let alone hand away the right for someone else to profit from selling their knowledge, unattributed no less. Best bet is to book publish, and require a license from anyone that wants to train on it.

While there is a thing to be said about the unethical business practices of Quora/StackOverflow, I reject the framing of “reducing your earning power.” Not everything is about transactions or self-benefit, especially when it comes to knowledge; it’s about contributing and collaboration. There is immense intrinsic value to that. I’m glad we don’t live in your world, where libre software is a pipe-dream and hackers hoard their knowledge like sickly dragons.
Post reply on HN