Live data from Hacker News

Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

thebullshitmachines.com

361–370 of 652 posts

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#361

> LLMs are not capable of reflecting on and reporting about how or why they do what they do. i get the why, but about the how: deepseek has shown it's able to explain it's reasoning, how it reach a conclusion. That's quite a far stretch from the llm being only able to estimate statistically what's the right word to put after another one.

Their arguments are just one special-pleading fallacy after another. "But... but... but... it's different when we do it!" No, it's not different when we do it. The takeaway here isn't that the AI algorithms are so special and magical, it's that our brains are not. It takes some nerve (literally) for humans to throw around labels like "bullshit machine." The only advantage we really have is long-term memory. I'm sure…

> The only advantage we really have is long-term memory.

We also have short term memory that is associative to our long term memory. So, STM is about 7 items. But those 7 items are also pointers to LTM.

Oh, and the ability to learn new things.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#362

Earlier quoted context omitted.

fair, but "logically consistent thoughts" is a subject of deep investigation starting from the early euclidean geometry to the modern godel's theorems. ie, that logically consistent thinking starts from symbolization, axioms, proof procedures, world models. otherwise, you end up with persuasive words.

You just ruled out 99% of humans from having reasoning capabilities. The beautiful thing about reasoning models is that there is no need to overcomplicate it with all the things you've mentioned, you can literally read the model's reasoning and decide for yourself if it's bullshit or not.

>You just ruled out 99% of humans from having reasoning capabilities.

After a conversation with humans I think you'd agree 1% of them being able to reason deeply is a vast overestimation.

A good example to see how little people can reason is the following classic:

> Given the following premises derive a conclusion about your poems:

> 1) No interesting poems are unpopular among people of real taste.

> 2) No modern poetry is free from affectation.

> 3) All your poems are on the subject of soap bubbles.

> 4) No affected poetry is popular among people of taste.

> 5) Only a modern poem would be on the subject of soap bubbles.

The average person on the street won't even know where to start, the average philosophy student will fuck up the translation to first order logic, and a logic professor would need a proof assistant to get it right consistently.

Meanwhile o3-mini in 10 seconds:

We can derive a conclusion about your poems by following the logical implications of the given premises. Let’s rephrase each premise into a more formal form:

Premise 1: No interesting poems are unpopular among people of real taste. This can be reworded as: If a poem is interesting, then it is popular among people of real taste.

Premise 2: No modern poetry is free from affectation. This tells us: If a poem is modern, then it is affected (i.e., it shows affectation).

Premise 3: All your poems are on the subject of soap bubbles. In other words: Every one of your poems is about soap bubbles.

Premise 4: No affected poetry is popular among people of taste. This implies: If a poem is affected, then it is not popular among people of taste.

Premise 5: Only a modern poem would be on the subject of soap bubbles. This means: If a poem is about soap bubbles, then it is modern.

Now, let’s connect the dots step by step:

From Premise 3 and Premise 5:

All your poems are on the subject of soap bubbles.

Only modern poems can be about soap bubbles.

Conclusion: All your poems are modern.

From the conclusion above and Premise 2:

Since your poems are modern, and all modern poems are affected,

Conclusion: All your poems are affected.

From the conclusion above and Premise 4:

Since your poems are affected, and no affected poem is popular among people of taste,

Conclusion: Your poems are not popular among people of taste.

From Premise 1:

If a poem is interesting, it must be popular among people of taste.

Since your poems are not popular among people of taste (from step 3), it follows that:

Conclusion: Your poems cannot be interesting.

Final Conclusion: Your poems are not interesting.

Thus, by logically combining the premises, we conclude that your poems are not interesting.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#363
post #288

Not sure why everyone rates this. It’s full of very confidently made statements like “the AI has no ground truth” (obviously it does, it has ingested every paper ever), it “can’t reason logically” which seems like a stretch if you ever read the CoT of a frontier reasoning model and “can’t explain how they arrived at conclusions” where - I mean just try it yourself with o1, go as deep as you like asking how it arrived…

the machine is fooling you with a mimicry of reasoning. and you are falling for it.

I don't give a rat's ass about whether or not AI reasoning is "real" or a "mimicry". I care if machines are going to displace my economic value as a human-based general intelligence.

If a synthetic "mimicry" can displace human thinking, we've got serious problems, regardless of whether or not you believe that it's "real".

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#364
post #221

(while I work at OAI, the opinion below is strictly my own) I feel like the current version is fairly hazardous to students and might leave them worse off. If I offer help to nontechnical friends, I focus on: - look at rate of change, not current point - reliability substantially lags possibility, by maybe two years. - adversarial settings remain largely unsolved if you get enough shots, trends there are unclear - ig…

This seems like trying to offer help predicting the future or investing in companies, which is a different kind of help from how to coexist with these models, how to use them to do useful things, what their pitfalls are, etc.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#365

Earlier quoted context omitted.

> we don't think will readily be overturned I think that’s entirely the problem. You’re making linear predictions of the capabilities of non-linear processes. Eventually the predictions and the reality will diverge.

There's no evidence to support that's the case.

Said the turkey to the farmer

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#366

Earlier quoted context omitted.

Their arguments are just one special-pleading fallacy after another. "But... but... but... it's different when we do it!" No, it's not different when we do it. The takeaway here isn't that the AI algorithms are so special and magical, it's that our brains are not. It takes some nerve (literally) for humans to throw around labels like "bullshit machine." The only advantage we really have is long-term memory. I'm sure…

> The only advantage we really have is long-term memory. We also have short term memory that is associative to our long term memory. So, STM is about 7 items. But those 7 items are also pointers to LTM. Oh, and the ability to learn new things.

I think of the context as short-term memory. If there were a way to update the weights based on the context, such that the outcomes of future queries would be influenced, things would get interesting in a hurry.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#367

Earlier quoted context omitted.

No, it is distinguishable from real reasoning. Real reasoning, while flawed in various ways, goes through personal experience of the evaluator. LLMs don't have that capability at all. They're just sifting though tokens and associate statistical parameters to it with no skin in the game so to speak.

It seems like an arbitrary distinction. If an LLM can accomplish a task that we’d all agree requires reasoning for a human to do, we can’t call that reasoning just because the mechanics are a bit different?

Yes because it isn't an arbitrary distinction. My good old TI-83 can do calculations that I can't even do in my head but unlike me it isn't reasoning about them, that's actually why it's able to do them so fast, and it has some pretty big implications about what it can't do.

If you want to understand where a systems limitations are you need to understand not just what it does but how it does it, I feel like we need to start teaching classes on Behaviorism again.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#368

Earlier quoted context omitted.

Yes! First, thank you for the link about CoT misrepresentation. I've written a fair bit about this on Bluesky etc but I don't think much if any of that made it into the course yet. We should add this to lesson 6, "They're Not Doing That!" Your point about humanities courses is just right and encapsulates what we are trying to do. If someone takes the course and engages in the dialectical process and decides we are mu…

I'll give it a read. I must admit, the more I learn about the inner workings of LLM's the more I see them as simply the sum of their parts and nothing more. The rest is just anthropomorphism and marketing.

Funny, I feel the same way about humans.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#369

Earlier quoted context omitted.

> we don't think will readily be overturned I think that’s entirely the problem. You’re making linear predictions of the capabilities of non-linear processes. Eventually the predictions and the reality will diverge.

There's no evidence to support that's the case.

Every time someone claimed “emerging” behavior in LLMs it was exactly that. I can probably count more than 100 of these cases, many unpublished, but surely it is easy to find evidence by now.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#370

Earlier quoted context omitted.

It depends on your tolerance for error. When you have a machine that can only infer rules for reasoning from inputs [which are, more often than not, encoded in a very roundabout way within a language which is very ambiguous, like English], you have necessarily created something without "ground." That's obviously useful in certain situations (especially if you don't know the rules in some domain!), but it's categorica…

Are you contending that every human derives their reasoning from first principals rather than being taught rules in a natural language?

I'm contending that, like any good tool, there is a context where it is useful, and a context where it is not (and that we are at a stage where everything looks suspiciously like a nail).
Post reply on HN