Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

thebullshitmachines.com

Post reply on HN
Print view View original

321–330 of 652 posts

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#321

Post by dullcrisp »

mr_toad wrote: post #318
ctbergstrom wrote: post #307

Earlier quoted context omitted.

Yes! First, thank you for the link about CoT misrepresentation. I've written a fair bit about this on Bluesky etc but I don't think much if any of that made it into the course yet. We should add this to lesson 6, "They're Not Doing That!" Your point about humanities courses is just right and encapsulates what we are trying to do. If someone takes the course and engages in the dialectical process and decides we are mu…

Current LLMs are not the end-all of LLMs, and chain of thought frontier models are not the end-all of AI. I’d be wary of confidently claiming what AI can and can’t do, at the risk of looking foolish in a decade, or a year, or at the pace things are moving, even a month.

They saw you coming in part 16.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#322

Post by ctbergstrom »

mr_toad wrote: post #318
ctbergstrom wrote: post #307

Earlier quoted context omitted.

Yes! First, thank you for the link about CoT misrepresentation. I've written a fair bit about this on Bluesky etc but I don't think much if any of that made it into the course yet. We should add this to lesson 6, "They're Not Doing That!" Your point about humanities courses is just right and encapsulates what we are trying to do. If someone takes the course and engages in the dialectical process and decides we are mu…

Current LLMs are not the end-all of LLMs, and chain of thought frontier models are not the end-all of AI. I’d be wary of confidently claiming what AI can and can’t do, at the risk of looking foolish in a decade, or a year, or at the pace things are moving, even a month.

That's entirely true. We've tried hard to stick with general principles that we don't think will readily be overturned. But doubtless we've been too assertive for some people's taste and doubtless we'll be wrong in places. Hence the choice to develop not a static book but rather living document that will evolve with time. The field is developing too fast for anything else.

With respect to what the future brings, we do try to address a bit of that in Lesson 16: https://thebullshitmachines.com/lesson-16-the-first-step-fal...

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#323

Post by onemoresoop »

grepLeigh wrote: post #303
pjs_ wrote: post #288

Not sure why everyone rates this. It’s full of very confidently made statements like “the AI has no ground truth” (obviously it does, it has ingested every paper ever), it “can’t reason logically” which seems like a stretch if you ever read the CoT of a frontier reasoning model and “can’t explain how they arrived at conclusions” where - I mean just try it yourself with o1, go as deep as you like asking how it arrived…

LLMs that use Chain of Thought sequences have been demonstrated to misrepresent their own reasoning [1]. The CoT sequence is another dimension for hallucination. So, I would say that an LLM capable of explaining its reasoning doesn't guarantee that the reasoning is grounded in logic or some absolute ground truth. I do think it's interesting that LLMs demonstrate the same fallibility of low quality human experts (i.e.…

The ground truth is chopped off into tokens and statistically evaluated. It is of course just a soup of ground truth that can freely be used in more or less twisted ways that have nothing to do or are tangent to the ground truth. While I enjoy playing with LLMs I don't believe they have any intrinsic intelligence to them and they're quite far from being intelligent in the same sense that autonomous agents such as us humans are.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#324

Post by kykeonaut »

mr_toad wrote: post #318
ctbergstrom wrote: post #307

Earlier quoted context omitted.

Yes! First, thank you for the link about CoT misrepresentation. I've written a fair bit about this on Bluesky etc but I don't think much if any of that made it into the course yet. We should add this to lesson 6, "They're Not Doing That!" Your point about humanities courses is just right and encapsulates what we are trying to do. If someone takes the course and engages in the dialectical process and decides we are mu…

Current LLMs are not the end-all of LLMs, and chain of thought frontier models are not the end-all of AI. I’d be wary of confidently claiming what AI can and can’t do, at the risk of looking foolish in a decade, or a year, or at the pace things are moving, even a month.

The post seems to be talking about the current capabilities of large language models. We can certainly talk about what they can or cannot do as of today, as that is pretty much evidence based.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#325

Post by onemoresoop »

Grimblewald wrote: post #302
bwfan123 wrote: post #293

Earlier quoted context omitted.

the machine is fooling you with a mimicry of reasoning. and you are falling for it.

If it's mimicry of reason is indistinguishable from real reasoning, how is it not reasoning? Ultimately, an LLM models language and the process behind it's creation to some degree of accuracy or another. If that model includes a way to approximate the act of reasoning, then it is reasoning to some extent. The extent I am happy to agree is open for discussion, but that reasoning is taking place at all is a little hard…

No, it is distinguishable from real reasoning. Real reasoning, while flawed in various ways, goes through personal experience of the evaluator. LLMs don't have that capability at all. They're just sifting though tokens and associate statistical parameters to it with no skin in the game so to speak.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#326

Post by mr_toad »

jvanderbot wrote: post #258

> We (the authors of this website) have at times sought insight into the inner workings of an LLM by asking it “why did you just do that?” > But the LLM can’t tell us. It’s not a person. It doesn’t have the metacognitive abilities necessary to reflect on its past actions and report the motivations underlying them*. > With no clue why it did whatever it just did, the LLM is forced to guess wildly at a plausible explan…

> I am almost convinced that we ourselves are a narrator riding along inside an animal's mind, trying desperately to put together explanations for our action

This is more or less the position of Daniel Dennett.

Also, it’s the premise of the one of Greg Egan’s (IMO) best short stories, Mr. Volition.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#327

Post by whattheheckheck »

onemoresoop wrote: post #323
grepLeigh wrote: post #303

Earlier quoted context omitted.

LLMs that use Chain of Thought sequences have been demonstrated to misrepresent their own reasoning [1]. The CoT sequence is another dimension for hallucination. So, I would say that an LLM capable of explaining its reasoning doesn't guarantee that the reasoning is grounded in logic or some absolute ground truth. I do think it's interesting that LLMs demonstrate the same fallibility of low quality human experts (i.e.…

The ground truth is chopped off into tokens and statistically evaluated. It is of course just a soup of ground truth that can freely be used in more or less twisted ways that have nothing to do or are tangent to the ground truth. While I enjoy playing with LLMs I don't believe they have any intrinsic intelligence to them and they're quite far from being intelligent in the same sense that autonomous agents such as us…

Any all of the tricks getting tacked on are overfitting to the test sets. It's all the tactics we have right now and they do provide assistance in a wide variety of economically valuable tasks with the only signs of stopping or slowing down is data curation efforts

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#328

Post by danenania »

onemoresoop wrote: post #325
Grimblewald wrote: post #302

Earlier quoted context omitted.

If it's mimicry of reason is indistinguishable from real reasoning, how is it not reasoning? Ultimately, an LLM models language and the process behind it's creation to some degree of accuracy or another. If that model includes a way to approximate the act of reasoning, then it is reasoning to some extent. The extent I am happy to agree is open for discussion, but that reasoning is taking place at all is a little hard…

No, it is distinguishable from real reasoning. Real reasoning, while flawed in various ways, goes through personal experience of the evaluator. LLMs don't have that capability at all. They're just sifting though tokens and associate statistical parameters to it with no skin in the game so to speak.

It seems like an arbitrary distinction. If an LLM can accomplish a task that we’d all agree requires reasoning for a human to do, we can’t call that reasoning just because the mechanics are a bit different?

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#329

Post by fmbb »

pjs_ wrote: post #288

Not sure why everyone rates this. It’s full of very confidently made statements like “the AI has no ground truth” (obviously it does, it has ingested every paper ever), it “can’t reason logically” which seems like a stretch if you ever read the CoT of a frontier reasoning model and “can’t explain how they arrived at conclusions” where - I mean just try it yourself with o1, go as deep as you like asking how it arrived…

Training on all papers does not mean the model believes or knows the truth.

It is just a machine that spits out words.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#330

Post by amelius »

What is scientific about this?

Saying that LLMs are just guessing the next tokens therefore they are parrots, that doesn't bring anything to the table. You might as well say that humans guess the next key they type on their keyboards. Both are probably true, at some level, but you don't gain any insight from it. You also didn't say anything about the probability distribution of the guesswork, so you didn't say if the guessing was smart guessing or dumb guessing. To be honest, I think this view is just a way of sticking your head in the sand about the upcoming technological revolution.

Post reply on HN