Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using secondary school maths to demystify AI

raspberrypi.org

Post reply on HN
Print view View original

191–200 of 264 posts

Re: Using secondary school maths to demystify AI

#191

Post by terminalshort »

softwaredoug wrote: post #155
dang wrote: post #190

[stub for offtopicness] (in this case, thinkiness)

LLMs don't really think, they emulate their training data. Which has a lot of examples of humans walking through problems to arrive at an answer. So naturally, if we prompt an LLM to do the same, it will emulate those examples (which tend to be more correct). LLMs are BAD at evaluating earlier thinking errors, precisely because there's not copious examples of text where humans thinking through a problem, screwing up,…

Claude code is actually great at that

Re: Using secondary school maths to demystify AI

#193

Post by terminalshort »

frozenlettuce wrote: post #146
terminalshort wrote: post #56

Earlier quoted context omitted.

I can make a claim much stronger than "you could probably" The counterclaim here is that the brain may not obey physical laws that can be described by mathematics. This is a "5G causes covid" level claim. The overwhelming burden of proof is on you.

There are some quantum effects in the brain (for some people, that's a possible source of consciousness). We can simulate quantum effects, but here comes the tricky part: even if our simulation matches the probability, say 70/30 of something happening, what guarantees that our simulation would take the same path as the object being simulated?

This is just non-determinism. Not only can't your simulation reproduce the exact output, but neither can your brain reproduce its own previous state. This doesn't mean it's a fundamentally different system.

Re: Using secondary school maths to demystify AI

#194

Post by terminalshort »

dang wrote: post #190

[stub for offtopicness] (in this case, thinkiness)

Given that the headline is:

> Secondary school maths showing that AI systems don’t think

And the article contains the quotes:

> the team wants to tackle a major and common misconception: that students think that ANN systems learn, recognise, see, and understand, when really it’s all just maths.

> The team is taking very complex ideas and reducing them to such an extent that we can use secondary classroom maths to show that AI is not magic and AI systems do not think.

This is not off topic

Re: Using secondary school maths to demystify AI

#195

Post by Kim_Bruning »

sublinear wrote: post #156
IgorPartola wrote: post #134

Earlier quoted context omitted.

I feel like these conversations really miss the mark: whether an LLM thinks or not is not a relevant question. It is a bit like asking “what color is an Xray?” or “what does the number 7 taste like?” The reason I say this is because an LLM is not a complete self-contained thing if you want to compare it to a human being. It is a building block. Your brain thinks. Your prefrontal cortex however is not a complete syste…

The part that's most infuriating is that we don't have to speculate at all. Any discussion or philosophizing beyond the literal computer science is simply misinformation. There's absolutely no similarity between what computer hardware does and what a brain does. People will twist and stretch things and tickle the imagination of the naive layperson and that's just wrong. We seriously have to cut this out already. Anth…

> There's absolutely no similarity between what computer hardware does and what a brain does.

But at very least there's also no similarity between what computer hardware does and what even the simplest of LLMs do. They don't run on eg. x86_64 , else qemu would be sufficient for inferencing.

Re: Using secondary school maths to demystify AI

#197

Post by jeffmcmahan »

omnicognate wrote: post #124
tracerbulletx wrote: post #102

Earlier quoted context omitted.

Yeah. This whole AI situation has really exposed how bad most people are at considering the ontological and semantic content of the words they use.

Indeed, people confidently assert as established fact things like "brains are bound by the laws of physics" and therefore "there can't be anything special" about them, so "consciousness is an illusion" and "the mind is a computer", all with absolute conviction but with very little understanding of what physics and maths really do and do not say about the universe. It's a quasi-religious faith in a thing not fully com…

This. People do not understand the implications of the most basic facts of modern science. Gravitation is instantaneous action at a distance via an "occult" force (to quote Newton's contemporaries).

Re: Using secondary school maths to demystify AI

#198

Post by sokka_h2otribe »

lisbbb wrote: post #120
tracerbulletx wrote: post #102

Earlier quoted context omitted.

Yeah. This whole AI situation has really exposed how bad most people are at considering the ontological and semantic content of the words they use.

A large number of adults I encounter are functionally illiterate, including business people in very high up positions. They are also almost 100% MATHEMATICALLY illiterate, not only unable to solve basic algebra and geometry problems, but completely unable to reason about statistical and probabilistic situations encountered in every day life. This is why gambling is so popular and why people are constantly fooled by p…

Gambling exists more because of people dopamine systems than math...though I get the overall drift. People are fooled by politicians because ?? Also not really math related I think.

Re: Using secondary school maths to demystify AI

#199

Post by kipchak »

Tadpole9181 wrote: post #91
kipchak wrote: post #80

Earlier quoted context omitted.

It seems like the brain "just" being a giant number of neurons is an assumption. As I understand it's still an area of active research, for example the role of glial cells. The complete function may or may not be pen and paper-able.

> The complete function may or may not be pen and paper-able. Would you mind expanding on this? At a base read, it seems you implying magic exists.

It could well be the case that the brain can be simulated, but presently we don't know exactly what variables/components must be simulated. Does ongoing neuroplasticity for example need to be a component of simulation? Is there some as of yet unknown causal mechanisms or interactions that may be essential?

Re: Using secondary school maths to demystify AI

#200

Post by wisty »

IgorPartola wrote: post #134
dang wrote: post #190

[stub for offtopicness] (in this case, thinkiness)

I feel like these conversations really miss the mark: whether an LLM thinks or not is not a relevant question. It is a bit like asking “what color is an Xray?” or “what does the number 7 taste like?” The reason I say this is because an LLM is not a complete self-contained thing if you want to compare it to a human being. It is a building block. Your brain thinks. Your prefrontal cortex however is not a complete syste…

This is the underlying problem behind syncophantcy.

I saw a YouTube video about a investigative youtuber Eddy Burback who very easily convinced chat4 that he should cut off all contact with friends and family, move to a cabin in the desert, eat baby food, wrap himself in alfoil, etc just feeding his own (faked) mistakes and delusions. "What you are doing is important, trust your instincts".

Wven if AI could hypothetically be 100x as smart as a human under the hood, it still doesn't care. It doesn't do what it thinks it should, it doesn't do what it needs to do, it does what we train it to.

We train in humanities weaknesses and follies. AI can hypothetically exceed humanity in some respects, but in other respects it is a very hard to control power tool.

AI is optimised, and optimised functions always "hack" the evaluation function. In the case of AI, the evaluation function includes human flaws. AI is trained to tell us what we want to hear.

Elon Musk sees the problem, but his solution is to try to make it think more like him, and even if that succeeds it just magnifies his own weaknesses.

Has anyone read the book criticising Ray Dalio? He is a very successful hedge fund manager, who decided that he could solve the problem of finding a replacement by psychology evaluation and training people to think like him. But even his smartest employees didn't think like him, they just (reading between the lines) gamed his system. Their incentives weren't his incentives - he could demand radical honesty and integrity but that doesn't work so well when he would (of course) reward the people who agreed with him, rather than the people who would tell him he was screwing up. His organisation (apparently) became a bunch of even more radical syncopants due to his efforts to weed out syncophantcy.

Post reply on HN