Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is AI reasoning right for the wrong reasons?

quantamagazine.org

Post reply on HN
Print view View original

241–250 of 268 posts

Re: Is AI reasoning right for the wrong reasons?

#241

Post by thesmtsolver2 »

quietbritishjim wrote: post #194
thesmtsolver2 wrote: post #173

Earlier quoted context omitted.

It is not semantics. For decades, logic and CS researchers have known what reasoning is. LLM folks suddenly can’t claim an approximation of that is what constitutes full scale reasoning just because they can achieve only an approximation. Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples but fails on simple multiplications often as it doesn’t hav…

> Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples This is a poor analogy because: * Multiplying numbers has a single objective answer. Whether a code is good (sometimes even just whether it's correct) can be quite subjective. * LLMs certainly do some level of composition between the data sources they were trained on i.e. they are more than just…

I think you are just agreeing with me and restating the inputs to my argument.

I didn’t say LLMs are capable of zero reasoning. An approximation is just an approximation no matter how good.

Will you bet your wealth or critical safety systems on:

1. Accuracy of standard computer arithmetic: Yes

2. Accuracy of Lean or automated theorem provers: Yes (you already do)

3. Accuracy of LLM reasoning: No

https://en.wikipedia.org/wiki/Handbook_of_Automated_Reasonin...

Re: Is AI reasoning right for the wrong reasons?

#242

Post by svachalek »

randomImmigrant wrote: post #229
esikich wrote: post #168

Earlier quoted context omitted.

And our nerves are a series of electrochemical impulses. There are nerves that do optical processing before it hits the brain. It's not obvious, and that's what's frustrating around here. "Computers can't possibly experience things". I'd argue it's just as unlikely for meat to do it. Again, philosophy 101 stuff.

It's very obvious actually, because it isn't just nerves in the brain. There's an equal number of glial cells, which form a continuous network, or syncitium. Their activity is in the form of calcium waves, which are both responsive to and shape neuronal oscillations. What biologists find frustrating is folks looking at one kind of dancer in the field and ignoring the rest of the spectacle, and calling it done.

And how do glial cells and calcium waves "obviously" create a fully conscious mind while tensor operations do not?

Re: Is AI reasoning right for the wrong reasons?

#243

Post by svachalek »

janalsncm wrote: post #120
esperent wrote: post #82

Earlier quoted context omitted.

Automated problem solving... of specifically the kind of problems we'd normally need human intelligence to solve (coding, translation, etc.). So "automated intelligence" would make sense to me.

The issue is that “intelligence” is doing a lot of work, and people do not agree on how to define it. You can take a look at the definitions section here: https://en.wikipedia.org/wiki/Intelligence

That's an argument for not using the word for anything, not specific to this case.

Re: Is AI reasoning right for the wrong reasons?

#244

Post by svachalek »

yakbarber wrote: post #197

I keep hearing we don’t know “how llm’s work”, in mean yes we know the algorithms but the WHY I suppose. I’m not sure if that’s really true, does the best researcher at OpenAI, Anthropic, Gemini not know why it works? Would they say that? Anyway these discussions always make me think we are too generous to humans. I know very few humans who are good at reasoning. It’s surprisingly hard to just think really hard throu…

We've come a long way since the first LLMs were created. They originally produced a lot of surprises that were very hard to explain, it's true. But current research is very tightly directed at producing forward results; it's not just throw in more training data and make a bigger model and see what happens anymore (although there's still a bit of that too). The researchers have much more advanced understanding of how they work now, it just doesn't propagate out to the public which is still repeating "autocomplete on steroids" years later.

Re: Is AI reasoning right for the wrong reasons?

#245

Post by drekipus »

crent wrote: post #231
drekipus wrote: post #218

Earlier quoted context omitted.

I suggest you print out this exchange and spend a week away from the computer, then re-read what was printed. I'm not trying to be rude but I gave you the answers. Alien philosophy is irrelevant to us because we're not aliens. Tree philosophy is irrelevant to us because we're not trees. Dolphin philosophy is irrelevant to us because we're not dolphins, etc. Everything else you mentioned is "hallucinatory" and you nee…

I’d respectfully disagree here. All phenomena are defined not just by what they are but also by what they are not. I can tell something is relatively tall because I can compare its size to something small. In a fully isolated environment with no reference points, the concept of a given object being small or tall is meaningless. If we apply that same thought process to philosophy and reason, how can you define it with…

I think that's kind of what I'm trying to say.

"A philosophy of LLMs" - has no reference point without being grounded in human experience.

It is only meaningful when we apply what it means to humans.

"a philosophy of a bash script." does not make sense. Only "philosophy of the action of writing the bash script, of which this is an artifact of" - the human who wrote it intentionally.

LLMs apply statistics to mash inputs and produce outputs, almost like `cat` with extra steps: We can't ask for "what was the philosophy behind the action `cat` applied" - why would we bother asking about the philosophy of an LLM? Better questions would be the philosophy of those who prompt it, built it, the material that went into it, the people who write the safeguards, the people who architected it. Etc.

It is not an isolated environment without reference points.

Re: Is AI reasoning right for the wrong reasons?

#246

Post by jhbadger »

drekipus wrote: post #218
esikich wrote: post #189

Earlier quoted context omitted.

"Philosophy is an intrinsically human thing. It has to be." What is philosophy? Thinking? Reasoning? Thinking about thinking? You're asserting that only a human brain can think and reason? Surely other animals can as well. Couldn't aliens as well? Why can't this crazy electronic box? Where do we draw the line? Do dolphins not have internal experiences? Dogs? Cats? Mice? Fish? Worms? Bugs? Amoebas? We don't even reall…

I suggest you print out this exchange and spend a week away from the computer, then re-read what was printed. I'm not trying to be rude but I gave you the answers. Alien philosophy is irrelevant to us because we're not aliens. Tree philosophy is irrelevant to us because we're not trees. Dolphin philosophy is irrelevant to us because we're not dolphins, etc. Everything else you mentioned is "hallucinatory" and you nee…

"Alien philosophy is irrelevant to us because we're not aliens"

Plenty of non-Chinese people study Chinese philosophy. It isn't at all irrelevant if the goal is to understand the Chinese (not to mention that some of their ideas may be relevant to us as well). If we learned that aliens existed, I'm sure plenty of humans would want to study their philosophy too.

Re: Is AI reasoning right for the wrong reasons?

#247

Post by randomImmigrant »

svachalek wrote: post #242
randomImmigrant wrote: post #229

Earlier quoted context omitted.

It's very obvious actually, because it isn't just nerves in the brain. There's an equal number of glial cells, which form a continuous network, or syncitium. Their activity is in the form of calcium waves, which are both responsive to and shape neuronal oscillations. What biologists find frustrating is folks looking at one kind of dancer in the field and ignoring the rest of the spectacle, and calling it done.

And how do glial cells and calcium waves "obviously" create a fully conscious mind while tensor operations do not?

You mean apart from all the biological evidence? I explained some of that in another thread in this topic.

Re: Is AI reasoning right for the wrong reasons?

#248

Post by mdp2021 »

ForHackernews wrote: post #232
mdp2021 wrote: post #230

Earlier quoted context omitted.

The standard is "do grow up; build a society that facilitates growing up; and do not hinder adults with rules for children - that is very much not a society". And your insulting attitude disservices the loser more than the normal people: you know, there are people who can count, there are people who won't lie, there are people who do not take drugs... There do exist adults.

You're arguing that we should be deliberately build evil and manipulative technology... why? To assert your own superior discernment?

Nobody has said that one should build «evil and manipulative technology». An interlocutor that says "I" is not «evil and manipulative technology».

People (normal people) are not manipulated by a generator that says "I". There is an I in that generator (it is the generator that outputs its output); and those who have weakminded reactions in front of innocent phenomena (e.g. those who would believe that "My name is Ishmael" declares the name of the book, that their instance of Moby Dick would want to be called Ishmael) cannot propose their fault as a norm for the rest.

Those who said that e.g. the colour red arouses them beyond self control cannot demand that we put red paint out of commerce: they have no "right" to be wierdly aroused by the colour red, and we are entitled to using red and every other normal thing that is so grave to people with unjustified problems. If people fainted when one said "pudding", they should seek for health, and they cannot demand that we stop saying "pudding".

Their mental illness is "their problem" (not something that should condition us), and a societal problem - in terms that people should be facilitated in becoming healthy.

Re: Is AI reasoning right for the wrong reasons?

#249

Post by mdp2021 »

drob518 wrote: post #209
mdp2021 wrote: post #198

Earlier quoted context omitted.

> is a machine following an algorithm (even a stochastic algorithm) “thinking” or “reasoning”? If the process is sound, yes, it can be said to be thinking or reasoning - it produced "sound thought or reasoning". The "reasoning" is there and elsewhere the absolute (from the producing entity) output which is valid as a representation of proper reasoning. If "A is B and B is C, then A is C" is reasoning, than what produ…

Yes, a proper definition of reasoning must be provided. How are you defining reasoning, exactly? You imply that formal logic is reasoning. Is reasoning just formal logic or is formal logic just a subset of reasoning? If a subset, what’s the rest of it? Since they implement logic, do Prolog programs “reason” or are they just performing inference according to the SLD resolution algorithm?

> proper definition of reasoning must be provided

It will be related to the context. If a process tells me that "if the light is blue and the sound is sharp then the lever must be lowered", it outputs reasoning; if what the activity did was running on conditional paths like an Expert System, what it did was different from the content and only improper reasoning; if the activity produced semantic evaluations about the light, the blue, the sound, the sharpness, the lever etc., then it was reasoning even if it was a process that reasoned.

But all of this makes sense only when it is there for the proper implementation of reasoning. Otherwise, Dijkstra is valid.

Re: Is AI reasoning right for the wrong reasons?

#250

Post by mdp2021 »

Borealid wrote: post #220
mdp2021 wrote: post #202

Earlier quoted context omitted.

You must have misunderstood the statement: «In order to properly engineer things, we must know how they work»: of course they may work anyway, of course there may be emergent properties (as was explicitly written), but the deontic part of knowledge augmentation, the scientific part, is missing. AI is full of non deterministic devices like genetic algorithms, but there exists a problem of transparency which is paramou…

No, I think you misunderstood me. We engineer the model-training algorithm and then we USE the model. The model is a mess of random gobbledegook that happens to be the best optimizer for the training algorithm's reward function. This is still science, in the same way it's still science when we breed cows to produce more milk without understanding the full bovine genome. If something isn't right with the model we pour…

> in the same way it's still science when we breed cows to produce more milk

If you cross cows and the amount or quality of milk varies, you have a question in the queue: why did it happen. The question is scientifically duly in abstract terms, and is practically, technically more duly if we benefit from a theory of cows crossing to achieve targeted milk yield.

> require peering into the slurry of random digits that make up the model itself

It is required by the duty of understanding the engine. When we understand engines, we can build better engines. And when we understand things, already our general knowledge is richer.

Post reply on HN