Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using secondary school maths to demystify AI

raspberrypi.org

Post reply on HN
Print view View original

231–240 of 264 posts

Re: Using secondary school maths to demystify AI

#231

Post by [deleted] »

t23414321 wrote: post #229
Retric wrote: post #210

Earlier quoted context omitted.

The general public have no issue saying a computer is thinking when you’re sitting there waiting for it to calculate a route or doing a similar process like selecting a chess move. The connotation is simply an internal process of indeterminate length rather than one of reflexive length. So they don’t apply it when a GPU is slinging out 120 FPS in a first person shooter.

That's right when saying selecting not calculating a chess move - assuming you are outside of Plato's cave (Popper). But now, I see this: the truth is static and non-profit, but calculating something can be sold again and again, if you have a hammer (processing) everything looks like a nail, to sell well the word thinking had to be used instead of excuse for every time results being different (like the shadows) - the…

[deleted]

Re: Using secondary school maths to demystify AI

#232

Post by t23414321 »

t23414321 wrote: post #229
Retric wrote: post #210

Earlier quoted context omitted.

The general public have no issue saying a computer is thinking when you’re sitting there waiting for it to calculate a route or doing a similar process like selecting a chess move. The connotation is simply an internal process of indeterminate length rather than one of reflexive length. So they don’t apply it when a GPU is slinging out 120 FPS in a first person shooter.

That's right when saying selecting not calculating a chess move - assuming you are outside of Plato's cave (Popper). But now, I see this: the truth is static and non-profit, but calculating something can be sold again and again, if you have a hammer (processing) everything looks like a nail, to sell well the word thinking had to be used instead of excuse for every time results being different (like the shadows) - the…

.. and confront about Prolog or else in recent years - likes: "intended benefit requires an unreasonably (or impossibly?) smart compiler" (https://news.ycombinator.com/item?id=14441045) - isn't quite similar to LLMs, for that, requiring.. impossibly smart users ?? (there were few - assuming they got what they wanted . not peanuts)

Re: Using secondary school maths to demystify AI

#233

Post by pegasus »

CamperBob2 wrote: post #218
pegasus wrote: post #204

Earlier quoted context omitted.

> For one thing, yes, they can That's post-training. The complaint I'm referring to is to the huge amounts of data (end energy) required during training - which is also a form of learning, after all. Sure, there are counter-arguments, for example pointing to the huge amount of non-textual data a child ingests, but these counter-arguments are not waterproof themselves (for example, one can point out that we are discus…

That's post-training No, it is not. Read the paper. They are discussing an emergent property of the context itself: "For all tasks, GPT-3 is applied without any gradient updates or fine-tuning, with tasks and few-shot demonstrations specified purely via text interaction with the model." I'm talking about tasks like multiplying two 4-digit numbers (let's say 8-digit, just to be safe, for reasoning models), which 5th o…

> No, it is not.

Yes, it is. You seem to have misunderstood what I wrote. The critique I was pointing to is of the amount of examples and energy needed during model training, which is what the "learning" in "machine learning" actually refers to. The paper uses GPT-3 which had already absorbed all that data and electricity. And the "learning" the paper talks about is arguably not real learning, since none of the acquired skills persists beyond the end of the session.

> So am I.

This is easy to settle. Go check any frontier model and see how far they get with multiplying numbers with tool calling disabled.

> Not a great time for you to rest on your intellectual laurels. Same goes for Penrose.

Neither am I resting, nor are there much laurels to rest on, at least compared to someone like Penrose. As for him, give the man a break, he's 94 years old and still sharp as a tack and intellectually productive. You're the one who's resting, imagining you've settled a question which is very much still open. Certainty is certainly intoxicating, so I understand where you're coming from, but claiming anyone who doubts computationalism is not bringing any arguments to the table is patently absurd.

Re: Using secondary school maths to demystify AI

#234

Post by fatata123 »

t23414321 wrote: post #203
dang wrote: post #188

Earlier quoted context omitted.

Yes, that word 'think' is a hell of a distraction. We've replaced that bit of the title with a less provocative / more interesting phrase from the article. Hopefully we can talk about the actual math and stuff (although the article doesn't go into much of that).

artificial intelligence in place of artificial knowledge is confusing distraction too, at least artificial is a good reminder since beginning (IMHO its not provocative but well catching a point.. about so called "intelligence" - what if we could look for intelligent knowledge - made in not statistic but semantic(?) and converging way - instead - of being distracted, afraid and.. outpriced, peanuts for the essence it…

[dead]

Re: Using secondary school maths to demystify AI

#235

Post by hackinthebochs »

hnfong wrote: post #230
hackinthebochs wrote: post #201

Earlier quoted context omitted.

>If we don't think the candle in a simulated universe is a "real candle", why do we consider the intelligence in a simulated universe possibly "real intelligence"? I can smell a "real" candle, a "real" candle can burn my hand. The term real here is just picking out a conceptual schema where its objects can feature as relata of the same laws, like a causal compatibility class defined by a shared causal scope. But this…

Right, but doesn't your argument imply that the only "real" consciousness is mine? I'm not against this conclusion ( https://en.wikipedia.org/wiki/Philosophical_zombie ) but it doesn't seem to be compatible with what most people believe in general.

That's a fair reading but not what I was going for. I'm trying to argue for the irrelevance of causal scope when it comes to determining realness for consciousness. We are right to privilege non-virtual existence when it comes to things whose essential nature is to interact with our physical selves. But since no other consciousness directly physically interacts with ours, it being "real" (as in physically grounded in a compatible causal scope) is not an essential part of its existence.

Determining what is real by judging causal scope is generally successful but it misleads in the case of consciousness.

Re: Using secondary school maths to demystify AI

#236

Post by [deleted] »

t23414321 wrote: post #232
t23414321 wrote: post #229

Earlier quoted context omitted.

That's right when saying selecting not calculating a chess move - assuming you are outside of Plato's cave (Popper). But now, I see this: the truth is static and non-profit, but calculating something can be sold again and again, if you have a hammer (processing) everything looks like a nail, to sell well the word thinking had to be used instead of excuse for every time results being different (like the shadows) - the…

.. and confront about Prolog or else in recent years - likes: "intended benefit requires an unreasonably (or impossibly?) smart compiler" ( https://news.ycombinator.com/item?id=14441045 ) - isn't quite similar to LLMs, for that, requiring.. impossibly smart users ?? (there were few - assuming they got what they wanted . not peanuts)

[deleted]

Re: Using secondary school maths to demystify AI

#237

Post by omnicognate »

mediaman wrote: post #208
omnicognate wrote: post #124

Earlier quoted context omitted.

Indeed, people confidently assert as established fact things like "brains are bound by the laws of physics" and therefore "there can't be anything special" about them, so "consciousness is an illusion" and "the mind is a computer", all with absolute conviction but with very little understanding of what physics and maths really do and do not say about the universe. It's a quasi-religious faith in a thing not fully com…

If your position is that brains are not actually bound by the laws of physics -- that they operate on some other plane of existence unbound by any scientifically tested principle -- then it is not only your ideological opposites who have quasi-religious faith in a thing not fully comprehended.

My "position" isn't remotely that. The problem with "brains are bound by the laws of physics" isn't that there's something special about brains. It's that physics doesn't consist of "laws" that things are "bound" by. It consists of theories that attempt to describe.

These theories are enormously successful, but they are also known to be variously incomplete, inconsistent, non-deterministic, philosophically problematic, open to multiple interpretations and only partially understood in their implications, with links between descriptions of things at different scales a particularly challenging and little understood topic. The more you learn about physics (and while I'm no physicist, I have a degree in the subject and have learned a great deal more since) the more you understand the limits of what we know.

Anybody who thinks there's no mystery to physics just doesn't know much about it. Anybody who confidently asserts as fact things like "the brain consists of protons, neutrons and electrons so it's impossible for it to do anything a computer can't do" is deducing things from their own ignorance.

Re: Using secondary school maths to demystify AI

#238

Post by thrance »

kipchak wrote: post #80
sigmoid10 wrote: post #59

Earlier quoted context omitted.

And the counter argument is also exactly the same. Imagine you take one neuron from a brain and replace it with an artificial piece of electronics (e.g. some transistors) that only generates specific outputs based on inputs, exactly like the neuron does. Now replace another neuron. And another. Eventually, you will have the entire brain replaced with a huge set of fundamentally super simple transistors. I.e. a comput…

It seems like the brain "just" being a giant number of neurons is an assumption. As I understand it's still an area of active research, for example the role of glial cells. The complete function may or may not be pen and paper-able.

There are indeed many people trying to justify this magical thinking by seeking something, anything in the brain that is out of the ordinary. They've been unsuccessful so far.

Penrose comes to mind, he will die on the hill that the brain involves quantum computations somehow, to explain his dualist position of "the soul being the entity responsible for deciding how the quantum states within the brain collapse, hence somehow controlling the body" (I am grossly simplifying). But even if that was the case, if the brain did involve quantum computations, those are still, well, computable. They just involve some amount of randomness, but so what? To continue with grandparent's experiment, you'd have to replace biological neurons with tiny quantum computer neurons instead, but the gist is the same.

Re: Using secondary school maths to demystify AI

#239

Post by mrwrong »

gowld wrote: post #95
qsort wrote: post #16

Earlier quoted context omitted.

I agree saying "they don't think" and leaving it at that isn't particularly useful or insightful, it's like saying "submarines don't swim" and refusing to elaborate further. It can be useful if you extend it to "they don't think like you do". Concepts like finite context windows, or the fact that the model is "frozen" and stateless, or the idea that you can transfer conversations between models are trivial if you kno…

> Concepts like > finite context windows like a human has > or the fact that the model is "frozen" and stateless, much like a human adult. Models get updated at a slower frequency than humans. AI systems have access to fetch new information and store it for context. > or the idea that you can transfer conversations between models are trivial because computers are better-organized than humanity.

> much like a human adult.

it doesn't sound like you really understand what these statements mean. if LLMs are like any humans it's those with late stage dementia, not healthy adults

Re: Using secondary school maths to demystify AI

#240

Post by sigmoid10 »

thrance wrote: post #238
kipchak wrote: post #80

Earlier quoted context omitted.

It seems like the brain "just" being a giant number of neurons is an assumption. As I understand it's still an area of active research, for example the role of glial cells. The complete function may or may not be pen and paper-able.

There are indeed many people trying to justify this magical thinking by seeking something, anything in the brain that is out of the ordinary. They've been unsuccessful so far. Penrose comes to mind, he will die on the hill that the brain involves quantum computations somehow, to explain his dualist position of "the soul being the entity responsible for deciding how the quantum states within the brain collapse, hence…

You wouldn't even need quantum computer neurons. We can simulate quantum nature on normal circuits, albeit not very efficiently. But for the experiment this wouldn't matter. The only important thing would be that you can measure it, which in turn would allow you to replicate it in some non-human circuit. And if you fundamentally can't measure this aspect for some weird reason, you will once again reach the same conclusion as above.
Post reply on HN