Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are steroids for your Dunning-Kruger

bytesauna.com

Post reply on HN
Print view View original

261–270 of 308 posts

Re: LLMs are steroids for your Dunning-Kruger

#261

Post by underlipton »

ecocentrik wrote: post #156
ThrowawayR2 wrote: post #129

Earlier quoted context omitted.

> " They imagined that 1000s of self-interested chaos monkeys would spend all of their energy destroying what 2-3 hard working people has spent hours creating instead of the inverse. " Isn't that exactly what happens on any controversial Wikipedia page?

There's not that many controversial topics at any given time. One of Wikipedia's solutions was to lock pages until a controversy subsided. Perma-controversy has been managed in other ways, like avoiding the statement of opinion as fact, the use of clear and uncontroversial language, using discussion pages to hash out acceptable and unacceptable content, competent moderators... Rage burns itself and people get bored w…

It doesn't always work. There are many topics that are perpetual edit wars because both (multiple) sides see the proliferation of their perspective as a matter of life and death. In many cases, one side is correct in this assessment and the others are delusional, but it's not always easy to align the side that's correct with the people who effectively control the page, because editors indeed do have their own biases (whether because of ideology, a philosophy, a political party, a nation, or whatever else). For those topics, Wikipedia can never be a source of "truth".

Re: LLMs are steroids for your Dunning-Kruger

#262

Post by TeMPOraL »

GMoromisato wrote: post #44

Speaking of uncertainty, I wish more people would accept their uncertainty with regards to the future of LLMs rather than dash off yet another cocksure article about how LLMs are {X}, and therefore {completely useless}|{world-changing}. Quantity has a quality of its own. The first chess engine to beat Gary Kasparov wasn't fundamentally different than earlier ones--it just had a lot more compute power. The original Go…

> Yes, LLMs are literally just matmul. How can anything useful, much less intelligent, emerge from multiplying numbers really fast? But then again, how can anything intelligent emerge from a wet mass of brain cells? After all, we're just meat. How can meat think?

LLMs actually hint at an answer to that, but most people seem to be focusing too much on matmuls or (on the other end) specific training inputs to pay attention to where the interesting things happen.

Training an LLM builds up a structure in high-dimensional space, and inference is a way to query the shape of that structure. That's literally the "quality of quantity", reified. This is what all those matmuls are doing.

How can anything useful, much less intelligent, emerge from a bunch of matmuls or wet mass of brain cells? That's the wrong level of abstraction. How can a general-purpose quasi-intelligence emerge from a stupidly high-dimensional latent space that embeds rich information about the world? That's the interesting question to ponder, and it starts with an important realization: it's not obvious why it couldn't.

Re: LLMs are steroids for your Dunning-Kruger

#263

Post by jjk166 »

versteegen wrote: post #247
jjk166 wrote: post #153

Earlier quoted context omitted.

That's not at all the reason. Encyclopedias are tertiary sources, compilations of information generated by others. They are neither sources of first hand information (primary sources) nor original analysis (secondary sources). You can't cite encyclopedias because there's nothing to cite. The encyclopedia was not the first place the claim was made, even if it was the first place you happened to read it. You don't attr…

What about scholarly encyclopedias? For example, the Stanford Encyclopedia of Philosophy. The articles are written in the style of a survey article, and if they're merely tertiary, I can't tell. If the intention behind a citation is a reference for a concept (an "existence proof" of it) rather than identifying its source or providing evidence, then a tertiary source such as to a textbook seems adequate.

There is some nuance. Wikipedia is a tertiary source for the subjects of its articles. However, it is a primary source for what is on wikipedia. You can cite an encyclopedia the same way you would cite the dictionary (which is also a tertiary source) as a way of establishing that information is in circulation.

Likewise, primary sources for some claims may be tertiary sources for others. If you read the memoirs of a soldier in WW1 who is comparing his exploits to those of a roman general from antiquity, he is a primary source for the WW1 history and a tertiary source for the roman history.

Survey articles and textbooks are generally tertiary. They may include analysis which is secondary and citable, but even then only the parts which are original are citable.

As a more general rule, you can't cite a piece of information from a work which is itself citing that piece of information (or ought to be).

Re: LLMs are steroids for your Dunning-Kruger

#264

Post by jordanb »

miningape wrote: post #52
Insanity wrote: post #11

Earlier quoted context omitted.

I remember back when I was in secondary school, something commonly heard was "Don't just trust wikipedia, check it's resources, because it's crowdsourced and can be wrong". Now, almost 2 decades later, I rarely hear this stance and I see people relying on wikipedia as an authoritative source of truth. i.e, linking to wikipedia instead of the underlying sources. In the same sense, I can see that "Don't trust LLMs" wil…

> "Don't just trust wikipedia, check it's resources, because it's crowdsourced and can be wrong" This comes from decades of teachers misremembering what the rule was, and eventually it morphed into the Wikipedia specific form we see today - the actual rule is that you cannot cite an encyclopaedia in an academic paper. full stop. Wikipedia is an encyclopaedia and therefore should not be cited. Wikipedia is the only en…

Yeah you weren't allowed to cite encyclopedias when I was a kid because:

1) encyclopedias are a tertiary source. They cite information collected by others. (Primary source: the actual account/document etc, Secondary source: books or articles about the subject, Tertiary source: Summaries of secondary sources.)

2) The purpose of writing a research paper was.. doing research and looking up an entry in an encylopedia is a very superficial form of research.

Also the overall quality of Wikipedia articles has improved over the years. I remember when it was much more like HHG with random goofy stuff in articles, poor citations, etc. Comparing it to, for instance, Encarta was often fun.

Re: LLMs are steroids for your Dunning-Kruger

#265

Post by LeroyRaz »

galaxyLogic wrote: post #188

> LLMs should not be seen as knowledge engines but as confidence engines. The thing I like best about LLM is when I ask question about some technical problem, and it tells that it is a KNOWN problem. It thus gives me confiidence that I don't need to spend time) to look for solution where there is no good soloution. Just go around it somehow. It let's me know I'm not the only person with this problem. And that way it…

Be careful. The models easily hallucinate problems and misdiagnose. For example, I had an issue with some GPU code, and it assured me, with utter conviction that my problem was caused by some subtle race condition ('a known issue') that the model described in great when the real issue was just a trivial typo - no race condition, no subtly or complexity.

Re: LLMs are steroids for your Dunning-Kruger

#266

Post by LeroyRaz »

habibur wrote: post #161

I think it's ok. When wikipedia arrived, everyone was up in arms that people are learning from something that's open for anyone to edit. But it rectified itself. The same thing happened when Internet arrived. "Don't believe anything you read on the Internet." I guess the reaction was same when printed media arrived. But the thing is, things get better over time.

I don't think things get better over time. What is your source for that? Here's an article (with sources) describing a massive down trend in literacy and reading comprehension: https://jmarriott.substack.com/p/the-dawn-of-the-post-litera...

In short, college students nowdays have lower reading comprehension than young children in the 1850s. That is not what I would call progress.

Speaking personally, I believe I would potentially have significantly worse critical reasoning abilities if I had grown up using LLMs. It is very clear to me the temptation of using them as an ersatz for engagement and thought.

I think you are perhaps conflating technological progress (yes technology has improved) with demographic progress. Demographic progress is far from monotonically increasing (reading comprehension is newly plummeting, maths scores are dropping in America, science per scientist is stalling compared to 50 years ago, etc...)

Re: LLMs are steroids for your Dunning-Kruger

#267

Post by LeroyRaz »

mattcantstop wrote: post #145

I think this is true. It can super charge some bad takes. But I've had the opposite experience. The average person is never going to read a scientific study, nor invest the time to find out the real details of any topic they are opinionated about other than simply typing a Youtube search and finding a video that is: - Entertaining - The person has their same biases - the present the information in a short, consumable…

You realize that they will gladly hallucinate science...

You should check the papers it claims to reference as see if the claims it makes are actually backed up.

In my experience, it can completely mischaracterize scientific literature. For example, I asked it if a codebase was a faithful implementation of an algorithm described in a CS paper, and is said "no" and then proceeded to list a dozen small changes. Every single change was incorrect. The codebase was in fact a completely faithful implementation.

Re: LLMs are steroids for your Dunning-Kruger

#268

Post by collingreen »

sho_hn wrote: post #4

I'm not sure this is something I really worry about. Whenever I use an LLM I feel dumber, not smarter; there's a sensation of relying on a crutch instead of having done the due diligence of learning something myself. I'm less confident in the knowledge and less likely to present it as such. Is anyone really cocksure on the basis of LLM received knowledge? > As I ChatGPT user I notice that I’m often left with a sense…

I agree but I've personally seen some egregious examples of people who are not only extremely confident in their new "knowledge" and "ability" but simultaneously think everyone else is extremely stupid. It's been absolutely wild to watch people paste chatgpt output and claim they wrote it, over and over again, even though every time I actually read it and ask a few "what does this mean" questions they have no idea and simply ask chatgpt then confidently say the response. It's so bad it's like a pathology; I wouldn't believe it if I hadn't seen it with my own eyes.

Something is happening here. Hopefully it's just revealing something that was already there in society and it isn't something new.

Re: LLMs are steroids for your Dunning-Kruger

#269

Post by LeroyRaz »

Chabsff wrote: post #3

> I feel like LLMs are a fairly boring technology. They are stochastic black boxes. The training is essentially run-of-the-mill statistical inference. There are some more recent innovations on software/hardware-level, but these are not LLM-specific really. This is pretty ironic, considering the subject matter of that blog post. It's a super-common misconception that's gained very wide popularity due to reactionary (a…

How is that a misconception? LLMs are just advanced statistical modelling (unsupervised machine learning) with small tweaks (e.g., some fine-tuning for human preference).

At the core, they are just statistical modelling. The fact that statistical modelling can produce coherent thoughts is impressive (and basically vindicates materialism) but that doesn't change the fact it is all based on statistical modelling. ...? What is your view?

Re: LLMs are steroids for your Dunning-Kruger

#270

Post by LeroyRaz »

Chabsff wrote: post #32
yannyu wrote: post #17

Earlier quoted context omitted.

What's the misconception? LLMs are probabilistic next-token prediction based on current context, right?

Yeah, but that's their interface. That informs surprisingly little about their inner workings. ANNs are arbitrary function approximators. The training process uses statistical methods to identify a set of parameters that approximate the function as best as possible. That doesn't necessarily mean that the end result is equivalent to a very fancy multi-stage linear regression. It's a possible outcome of the process, bu…

What do you mean? what do you think statistical modelling is?

I am very confused by your stance.

The aim of the function approximation is to maximize the likelihood of the observed data (this is standard statistical modelling), using machine learning (e.g., stochastic gradient decent) on a class of universal function approximators is a standard approach to fitting such a model.

What do you think statistical modelling involves?

Post reply on HN