Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

thebullshitmachines.com

Post reply on HN
Print view View original

571–580 of 652 posts

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#571

Post by habinero »

radlad wrote: post #479
habinero wrote: post #338

Earlier quoted context omitted.

Agree. It's also a virtue to point out the emperor has no clothes and the tailor peddling them is a bullshit artist. This is no different than the crypto people who insisted the blockchain would soon be revolutionary and used for everything, when in reality the only real use case for a blockchain is cryptocoins, and the only real use case for cryptocoins is crime. The only really good use case for LLMs is spam, becau…

> The only really good use case for LLMs is spam, because it's the only use case for generating a lot of human-like speech without meaning. As someone who's been writing code for nearly 20 years now, and who spent a few weeks rewriting a Flutter app in Jetpack Compose with some help from Claude ( https://play.google.com/store/apps/details?id=me.johnmaguire... ), I have to say I don't agree with this at all.

Ok? I too have been coding for over a decade and use Copilot as fancy autocomplete. I like it. It's not amazing.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#572

Post by daveguy »

ninetyninenine wrote: post #541
daveguy wrote: post #520

Earlier quoted context omitted.

If it's only reasoning randomly how do you know when anything has been reasoned properly vs just a generated simulation of reasonable text?

We use Probability. Find a prompt that has a large range aka codomain. If it arrived at the correct answer then that the only possibility here is reasoning because the codomain is so large it cannot arrive there by random chance. Of course make sure the prompt is unique such that it's not in the data and it's not doing any sort of "pattern matching". So like all science we prove it via probability. Observations match…

Pardon my ignorance -- assuming that range and codomain are approximately equivalent in this context, how do you specify a prompt with a large codomain? Is there a canonical example of a prompt with a large codomain?

It seems to me that, in natural language, the size of the codomain is related to the specificity of the prompt. For instance, if the prompt is "We are going to ..." then the codomain is enormous. But if the prompt is "2 times 2 is..." the codomain is, mathematically, {4, four}, some series of 4 symbols, eg IIII, or some other representation of the concept of "4" (ie different base or language representations: 0x04, 0b100, quatro, etc).

But if this is the case, a broad codomain is approximately synonymous with "no correct answer" or "result is widely interpretable". Which implies that the larger the codomain the easier it is to claim an answer "correct" in context of the prompt.

How do you reconcile loose interpretability with statistical rigor?

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#574

Post by Barrin92 »

Lerc wrote: post #433
Barrin92 wrote: post #367

Earlier quoted context omitted.

Yes because it isn't an arbitrary distinction. My good old TI-83 can do calculations that I can't even do in my head but unlike me it isn't reasoning about them, that's actually why it's able to do them so fast, and it has some pretty big implications about what it can't do. If you want to understand where a systems limitations are you need to understand not just what it does but how it does it, I feel like we need t…

Taking a different tack from others in this thread. I don't think you can say that a TI-83 is not reasoning if it is doing calculations. Certainly it is not aware of any concepts of numbers and has no meaningful sense of the operation, but those are attributes of sentience, not reasoning. The reasoning ability of a calculator is extremely limited but what make those capabilities that it does have, non reasoning. What…

>Do you consider sentience and reasoning to be one and the same?

No, but I think they share some similarities. You can be sentient without doing any reasoning, just through experience, there's probably a lot of simple life forms in that category. Where they overlap I think, is in that they require a degree of reflection. Reasoning I'd say is the capacity to distinguish between truth and falsehoods, to have mental content of the object you're reasoning about and as a consequence have a notion of understanding and an interior or subjective view.

The distinction I'd make is that calculation or memorization is not reasoning at all. My TI-83 or Stockfish can calculate math or chess but they have no notion of math or chess, they're basically Chinese rooms, they just perform mechanical operations. They can appear as if they reason, even a chess engine purely looking up results in a table base and with very simplistic brute force can play very strong chess but it doesn't know anything about chess. And with the LLMs you need to be careful because the "large" part does a lot of work. They often can sound like they reason but when they have to explain their reasoning they'll start to make up obvious falsehoods or contradictions. A good benchmark if something can reason is probably if it can.. reason about its reasoning coherently.

I do think the very new chain-of-thought models are more of a step into that direction, the further you get away from relying on data the more likely you're building something that reasons but we're probably very early into systems like that.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#575

Post by enragedcacti »

ninetyninenine wrote: post #539
enragedcacti wrote: post #510

Earlier quoted context omitted.

LLMs CAN read minds. Whether it can’t read minds is not provable. Literally I invite people to post prompts and correct answers to ChatGPT where it is trivially impossible for it to have known what number you were thinking of. Every one of those examples falsifies the claim that LLMs can’t read minds.

ok prove it. I'm thinking of a number right now between 1-10,000. Show me the number the LLM guesses. You can definitively prove this statement for me. It's a probability problem really. The range of a prompt has billions of possibilities. If it arrived at a correct answer within that range then the probability it got there without reasoning is miniscule. Same with this mind reading thing. Prove it.

Doesn't really seem fair that any one prompt proves your conclusion but it has to guess your exact number to prove my conclusion. Gemini guessed mine on the very first try (7) even though the range of numbers is infinite. Billions is small potatoes compared to what I've proven.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#576

Post by habinero »

TeMPOraL wrote: post #407
habinero wrote: post #332

Earlier quoted context omitted.

What? That's (1) not true and (2) says, uh, a lot of unintentional things about the way you approach the world. I'm not sure you realize quite how it makes you look. For one, it's not even internally consistent -- the people who built telescopes and satellites didn't "see" the moons, either. They got back a bunch of electrical signals and interpreted it to mean something. This worldview essentially boils down to the…

> I'm not sure you realize quite how it makes you look. I probably don't. > For one, it's not even internally consistent -- the people who built telescopes and satellites didn't "see" the moons, either. They got back a bunch of electrical signals and interpreted it to mean something. I'm not trying to go 100% reductionist here; I thought the point was clear. I was locking on the distinction between "learn from experi…

> Wait, what's wrong with this view? Wasn't exactly refuted in any way, despite proclamations by the more "embodied cognition" folks, whose beliefs are to me just a religion trying to retroactively fit to modern science to counter diminishing role of human soul at the center of it.

It's unfalsifiable, that's what's wrong with it. Sure, you could be a brain in a jar experiencing a simulated world, but there's nothing useful about that worldview. If the world feels real, you might as well treat it like it is.

> My point is to highlight that, for most of what we call today knowledge, which isn't tied to directly experiencing a phenomena in question, we're not learning in ways fundamentally different to what LLMs are doing

I get what you're trying to say -- nobody can derive everything from first principles, which is true -- but your conclusion is absolutely not true. Humans don't credulously accept what we're given in a true/false binary and spit out derived facts.

All knowledge is an approximation. There is very little absolute truth. And we're good at dealing with that.

Humans learn by building up mental models of how systems work, understanding when those models apply and when they don't, understanding how much they can trust the model and understanding how to test conclusions if they aren't sure.

LLMs can't do any of that.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#577

Post by radlad »

habinero wrote: post #571
radlad wrote: post #479

Earlier quoted context omitted.

> The only really good use case for LLMs is spam, because it's the only use case for generating a lot of human-like speech without meaning. As someone who's been writing code for nearly 20 years now, and who spent a few weeks rewriting a Flutter app in Jetpack Compose with some help from Claude ( https://play.google.com/store/apps/details?id=me.johnmaguire... ), I have to say I don't agree with this at all.

Ok? I too have been coding for over a decade and use Copilot as fancy autocomplete. I like it. It's not amazing.

Claude isn't Copilot, and I wasn't using it as autocomplete. I was using it to do things such as:

- Creating a migration from the old DB to the new DB, no modifications of the generated code necessary

- Refactoring state in a component out into a ViewModel, again no modifications necessary

- Creating all the classes necessary for interacting with a Room database (i.e. the data class, dao, and DI module) given a schema

- Creating the first iteration of a download worker, which I modified

Check out plugins like ClaudeMind for JetBrains! They can even intelligently (automatically) feed information from your current tab or other unopened but relevant-sounding files to the AI. It was an eye-opening experience.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#578

Post by azakai »

planb wrote: post #409
azakai wrote: post #403

Overall this is very good, but I have one specific note: Lesson 6 says "LLMs aren't conscious." I think I get what you're saying there - they are not conscious in the same way that humans are - but "consciousness" is a highly-debated term without a precise definition, and correspondingly philosophers have no consensus on whether machines in general are capable of it. Here is one of my favorite resources on that, the…

Philosophy aside - how can an LLM be conscious without a memory or manifestation in the real world? It is a function that, given an input, returns an output and stops existing afterwards. You wouldn't argue that f(x)=x^2 is conscious? I would maybe accept debates about whether for example ChatGPT (the whole system that stores old conversations and sends the history along with the current user entry) is conscious - bu…

It is true that human consciousness is continuous over time, but maybe some animals have very little of that?

Or, to look at it like Black Mirror, if you upload your consciousness into a machine, are you not conscious if it pauses the simulation for a moment? Perhaps you would have no memory of that time (like in Severance), but you could still be conscious at other times.

I do agree that a model at rest, just sitting on a hard drive, doesn't seem capable of consciousness. I also agree x^2 is not conscious. But the problem, philosophically, is actually separating those cases from things we know are conscious. The point of Searle's Chinese Room theorem is that he thinks no machine - not x^2, not a super-AI that passes the Turing Test - truly "thinks" (experiences, understands, feels, is conscious). But that position seems really hard to defend, even if it gives the "right" answer for x^2.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#579

Post by ninetyninenine »

enragedcacti wrote: post #575
ninetyninenine wrote: post #539

Earlier quoted context omitted.

ok prove it. I'm thinking of a number right now between 1-10,000. Show me the number the LLM guesses. You can definitively prove this statement for me. It's a probability problem really. The range of a prompt has billions of possibilities. If it arrived at a correct answer within that range then the probability it got there without reasoning is miniscule. Same with this mind reading thing. Prove it.

Doesn't really seem fair that any one prompt proves your conclusion but it has to guess your exact number to prove my conclusion. Gemini guessed mine on the very first try (7) even though the range of numbers is infinite. Billions is small potatoes compared to what I've proven.

I’ll pick a prompt such that the range is vast so that if it gets the answer right the probability is so small that it must have arrived there by reasoning.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#580

Post by corford »

ctbergstrom wrote: post #286
patja wrote: post #211

I'm uncomfortable with the use of profanity as a core element of this campaign branding, especially given that it seems to be an educational outreach effort. While this seems targeted at college age and above, I think it would be highly relevant content for a teen audience as well. While I swear in privacy on occasion I think it has no place in the classroom. I really don't care for it in the workplace either but I g…

This is something we've given serious consideration, having taught a course called "Calling Bullshit" ( http://callingbullshit.org ) for almost a decade and having authored a book by the same name that gets downranked on various Amazon features because of its title. But the bullshit is a term of art here, after the seminal 1986 article "On Bullshit" by Princeton philosopher Harry Frankfurt (later published as a littl…

FWIW, I don't think you should cave on this. For me, your choice to use it over "hallucination" instantly elevated the insight of the lessons. I also think the authenticity of the voice of the lessons benefits from owning the decision to use it fully rather than compromising with the shorter "bs" version.
Post reply on HN