Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

thebullshitmachines.com

Post reply on HN
Print view View original

351–360 of 652 posts

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#351

Post by lanstin »

Grimblewald wrote: post #302
bwfan123 wrote: post #293

Earlier quoted context omitted.

the machine is fooling you with a mimicry of reasoning. and you are falling for it.

If it's mimicry of reason is indistinguishable from real reasoning, how is it not reasoning? Ultimately, an LLM models language and the process behind it's creation to some degree of accuracy or another. If that model includes a way to approximate the act of reasoning, then it is reasoning to some extent. The extent I am happy to agree is open for discussion, but that reasoning is taking place at all is a little hard…

I am getting two contradictory but plausible seeming replies when I ask about a certain set being the same when adding 1 to every value in the set, asked on how I ask the question.

Correct answer: https://chatgpt.com/share/67a9500b-2360-8007-b70e-0bc2b84bc1...

Incorrect answer (I think): https://chatgpt.com/share/67a950df-d4e0-8007-8105-95a9e5be19...

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#352

Post by llm_trw »

onemoresoop wrote: post #325
Grimblewald wrote: post #302

Earlier quoted context omitted.

If it's mimicry of reason is indistinguishable from real reasoning, how is it not reasoning? Ultimately, an LLM models language and the process behind it's creation to some degree of accuracy or another. If that model includes a way to approximate the act of reasoning, then it is reasoning to some extent. The extent I am happy to agree is open for discussion, but that reasoning is taking place at all is a little hard…

No, it is distinguishable from real reasoning. Real reasoning, while flawed in various ways, goes through personal experience of the evaluator. LLMs don't have that capability at all. They're just sifting though tokens and associate statistical parameters to it with no skin in the game so to speak.

>goes through personal experience of the evaluator

Real reasoning is being able to manipulate symbolic expressions in a consistent manner while preserving some invariants.

Personal experience as logic is how you end up with the Holocaust.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#353

Post by habinero »

mr_toad wrote: post #337
ctbergstrom wrote: post #322

Earlier quoted context omitted.

That's entirely true. We've tried hard to stick with general principles that we don't think will readily be overturned. But doubtless we've been too assertive for some people's taste and doubtless we'll be wrong in places. Hence the choice to develop not a static book but rather living document that will evolve with time. The field is developing too fast for anything else. With respect to what the future brings, we d…

> we don't think will readily be overturned I think that’s entirely the problem. You’re making linear predictions of the capabilities of non-linear processes. Eventually the predictions and the reality will diverge.

There's no evidence to support that's the case.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#354

Post by llm_trw »

habinero wrote: post #350
nmca wrote: post #314

Earlier quoted context omitted.

The discourse around synthetic data is like the discourse around trading strategies — almost anyone who really understands the current state of the art is massively incentivised not to explain it to you. This makes for piss-poor public epistemics.

Nah, you don't need to know the details to evaluate something. You need the output and the null hypothesis. If a trading firm claims they have a wildly successful new strategy, for example, then first I want to see evidence they're not lying - they are actually making money when other people are not. Then I want to see evidence they're not frauds - it's easy to make money if you're insider trading. Then I want to see…

I don't get offended when people call my work a stochastic parrot.

I just put them in the same bucket of intelligence as an 8b model and weight their inputs accordingly.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#356

Post by maxdoop »

bwfan123 wrote: post #341
maxdoop wrote: post #313

Earlier quoted context omitted.

What is reasoning? What is understanding? Do humans do either? How do you know?

this is the question that the greeks wrestled with over 2000 years ago. at the time there were the sophists (modern llm equivalents) that could speak persuasively like a politician. over time this question has been debated by philosophers, scientists, and anyone who wanted to have better cognition in general.

So how can you claim what an LLM is doing if we cannot define it regardless?

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#359

Post by lifthrasiir »

fmbb wrote: post #329
pjs_ wrote: post #288

Not sure why everyone rates this. It’s full of very confidently made statements like “the AI has no ground truth” (obviously it does, it has ingested every paper ever), it “can’t reason logically” which seems like a stretch if you ever read the CoT of a frontier reasoning model and “can’t explain how they arrived at conclusions” where - I mean just try it yourself with o1, go as deep as you like asking how it arrived…

Training on all papers does not mean the model believes or knows the truth. It is just a machine that spits out words.

I would be very careful to claim exactly that as emergent properties seem kinda crucial for artificial and human intelligences. (Not to say that they are equally functioning nor useful.)

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#360

Post by habinero »

s2radhak wrote: post #342

Fascinating. The article repeatedly makes the claim that “LLMs work by predicting likely next words in a string of text”. Yet there’s the seemingly contradictory implication that we don’t know how LLMs work (ie we don’t know their secret sauce). How does one reconcile this? They’re either fancy autocompletes, or magic autocompletes (in which case the magic qualifier seems more important in understanding what they are…

We...do know how they work?
Post reply on HN