Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

thebullshitmachines.com

Post reply on HN
Print view View original

421–430 of 652 posts

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#421

Post by oblio »

randomNumber7 wrote: post #415
pjs_ wrote: post #288

Not sure why everyone rates this. It’s full of very confidently made statements like “the AI has no ground truth” (obviously it does, it has ingested every paper ever), it “can’t reason logically” which seems like a stretch if you ever read the CoT of a frontier reasoning model and “can’t explain how they arrived at conclusions” where - I mean just try it yourself with o1, go as deep as you like asking how it arrived…

> “can’t explain how they arrived at conclusions” Imagine I would tell my wife, that whenever we have a discussion, her opinion would only be valid when she can explain how she arrived at her conclusion.

Your wife is one of the end products of cutthroat competition across several billion years so let's just say her general intelligence has a fair bit more validation than 20 years of research.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#422

Post by oblio »

danenania wrote: post #384
meroes wrote: post #383

Earlier quoted context omitted.

My prior is reasoning is a conscious activity. There is a first person perspective. LLMs are so far removed mechanically from brains the idea they reason is not even remotely worth considering. Modeling neurons can be done with a series of pipes and flowing water, and that is not expected to give arise to consciousness either. Nor are nuerons and synapses likely to be sufficient for consciousness. You know how we ins…

Are we then conferring some kind of supernatural or religious properties to the brain’s particular implementation of neurons? If not, then why shouldn’t differently constructed but algorithmically similar systems be able to produce similar phenomena?

Because we know practically nothing about brains so comparing them to LLMs is useless and nature is so complex that we're constantly discovering signs of hubris in human research.

See C-sections versus natural birth. Formula versus mother's milk. Etc.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#424

Post by falcor84 »

yathaid wrote: post #416
joenot443 wrote: post #396

Earlier quoted context omitted.

It's 1994. Larry Llyod Mayer has read the entire internet, hundreds of thousands of studies across every field, and can answer queries word for word the same as modern LLMs do. He speaks every major language. He's not perfect, he does occasionally make mistakes, but the sheer breadth of his knowledge makes him among the most employable individuals in America. The Pentagon, IBM, and Deloitte are begging to hire him. I…

Does his accuracy take a sudden precipitous fall when going from multiplying two three-digit numbers to two four-digit numbers?

I don't know about you, but when I do math without a calculator, my accuracy also drops precipitously whenever they add a digit.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#425

Post by falcor84 »

oblio wrote: post #421
randomNumber7 wrote: post #415

Earlier quoted context omitted.

> “can’t explain how they arrived at conclusions” Imagine I would tell my wife, that whenever we have a discussion, her opinion would only be valid when she can explain how she arrived at her conclusion.

Your wife is one of the end products of cutthroat competition across several billion years so let's just say her general intelligence has a fair bit more validation than 20 years of research.

Sexual selection applies an evolutionary pressure against men who challenge women too much about the validity of their reasoning.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#426

Post by poulpy123 »

pjs_ wrote: post #288

Not sure why everyone rates this. It’s full of very confidently made statements like “the AI has no ground truth” (obviously it does, it has ingested every paper ever), it “can’t reason logically” which seems like a stretch if you ever read the CoT of a frontier reasoning model and “can’t explain how they arrived at conclusions” where - I mean just try it yourself with o1, go as deep as you like asking how it arrived…

But we know how the LLM works, and that's exactely how the authors explain it. And that explain also the weird mistakes they do, that nothing with the ability of reason or having a ground truth would do.

I really do not understand how technical people can think they are sentient

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#427

Post by throwaway290 »

falcor84 wrote: post #424
yathaid wrote: post #416

Earlier quoted context omitted.

Does his accuracy take a sudden precipitous fall when going from multiplying two three-digit numbers to two four-digit numbers?

I don't know about you, but when I do math without a calculator, my accuracy also drops precipitously whenever they add a digit.

Do you have self awareness to anticipate the drop in your accuracy and refuse to perform the operation?

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#428

Post by Lerc »

grepLeigh wrote: post #303
pjs_ wrote: post #288

Not sure why everyone rates this. It’s full of very confidently made statements like “the AI has no ground truth” (obviously it does, it has ingested every paper ever), it “can’t reason logically” which seems like a stretch if you ever read the CoT of a frontier reasoning model and “can’t explain how they arrived at conclusions” where - I mean just try it yourself with o1, go as deep as you like asking how it arrived…

LLMs that use Chain of Thought sequences have been demonstrated to misrepresent their own reasoning [1]. The CoT sequence is another dimension for hallucination. So, I would say that an LLM capable of explaining its reasoning doesn't guarantee that the reasoning is grounded in logic or some absolute ground truth. I do think it's interesting that LLMs demonstrate the same fallibility of low quality human experts (i.e.…

I think you'll find that humans have also demonstrated that they will misrepresent their own reasoning.

That does not mean that they cannot reason.

In fact, to come up with a reasonable explanation of behaviour, accurate or not, requires reasoning as I understand it to be. LLMs seem to be quite good at rationalising which is essentially a logic puzzle trying to manufacture the missing piece between facts that have been established and the conclusion that they want.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#430

Post by hnthrow90348765 »

fmbb wrote: post #329
pjs_ wrote: post #288

Not sure why everyone rates this. It’s full of very confidently made statements like “the AI has no ground truth” (obviously it does, it has ingested every paper ever), it “can’t reason logically” which seems like a stretch if you ever read the CoT of a frontier reasoning model and “can’t explain how they arrived at conclusions” where - I mean just try it yourself with o1, go as deep as you like asking how it arrived…

Training on all papers does not mean the model believes or knows the truth. It is just a machine that spits out words.

It has some pieces of the puzzle to intelligence. That's a deal breaker for some people, and useful/promising to others.
Post reply on HN