Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

thebullshitmachines.com

Post reply on HN
Print view View original

641–650 of 652 posts

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#641

Post by radlad »

habinero wrote: post #636
radlad wrote: post #577

Earlier quoted context omitted.

Claude isn't Copilot, and I wasn't using it as autocomplete. I was using it to do things such as: - Creating a migration from the old DB to the new DB, no modifications of the generated code necessary - Refactoring state in a component out into a ViewModel, again no modifications necessary - Creating all the classes necessary for interacting with a Room database (i.e. the data class, dao, and DI module) given a schem…

I would consider all of that fancy autocomplete. If it's just helping you type boring boilerplate things you would have typed anyway, that's autocomplete. It's neat, but it's not the paradigm shift people seem to think it is. Copilot mostly just replaces StackOverflow searches for me.

I would've said the same thing a few weeks ago.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#642

Post by tripzilch »

consumer451 wrote: post #620
cristiancavalli wrote: post #611

Earlier quoted context omitted.

How does this prove reasoning? The thread you point to has several question in it that remain unanswered that ask the same question? How is this not entirely derivative too — there’s a huge number of these kind of 3-box “games” (although I don’t see this as a game really) so something very similar to this is probably in the training data a lot. Writing code to factor a number is definitely very common. Variation of t…

Here is GPT 4o in "reasoning" mode answering the question "Are LLMs capable of reasoning?" > It depends on how you define "reasoning." > LLMs like me can perform certain types of reasoning, such as: > Pattern recognition & analogy – Recognizing relationships between concepts and applying similar patterns in new contexts. > Deductive reasoning – Applying rules to specific cases (e.g., "If all humans are mortal and Soc…

It fails at deductive reasoning though. Pick a celebrity with non-famous children that don't obviously share their last name or something. If you ask it "who is the child of ", it will get it right, because this is in its training data, probably Wikipedia.

If you ask "who is the parent of ", it will often claim to have no knowledge about this person.

Yes sometimes it gets it right, but sometimes also not. Try a few celebrities.

Maybe the disagreement is about this?

Like if it gets it right a good amount of the time, you would say that means it's (in principle) capable of reasoning.

But I say, that if it gets it wrong a lot of the time, that means 1) it's not reasoning in situations when it gets it wrong, but also 2) it's most likely also not reasoning in situations when it gets it right.

And maybe you disagree with that, but then we don't agree on what "reasoning" means. Because I think that consistency is an important property of reasoning.

I think that if it gets "A is parent of B, implies B is child of A" wrong for some celebrity parents, but not for others, then it's not reasoning. Because reasoning would mean applying this logical construct as a rule, and if it's not consistent at that, it makes it hard to argue that it is in fact applying this logical rule instead of doing who-knows-what that happens to give the right answer, some of the time.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#643

Post by tripzilch »

ninetyninenine wrote: post #502
cristiancavalli wrote: post #493

Earlier quoted context omitted.

No I can “prove” it — look at any number of cases where LLMs can’t even do basic value comparisons despite being claimed as super intelligent. You can try and say well that’s a limitation of the technology and then I would reply — yes and that’s why I would say it’s not reasoning according the original human definition. Also you have yet to produce any evidence of reasoning and claiming you can over and over again do…

LLMs can reason they just don’t always reason. That’s the claim everyone makes. That is a human definition if it reasoned one time correctly. That is the colloquial definition. Someone who has brain damage can reason correctly on certain subjects and incorrectly on other subjects. This is an immensely reasonable definition. I’m not being pedantic or out of line here when I say LLMs can reason while using this definit…

No reasoning is about applying rules of logic consistently, so if you only do it some of the time, that's not reasoning.

If I roll a die and only _sometimes_ it returns the correct answer to a basic arithmetic question, this is the exact reason why we don't say a die is doing arithmetic.

Even worse in the case of LLMs, where it's not caused by pure chance, but also training bias and hallucinations.

You can claim nobody knows the exact definition of reasoning, maybe there are some edges which aren't clearly defined because they're part of Philosophy, but applying rules of logic consistently is not something you just don't always do and still call it reasoning.

Also, LLMs are generally incapable of saying they don't know something, cannot know something, can't do something, etc. They would rather try and hallucinate. When it does that, it's not reasoning. And you also can't explain to an LLM how to figure out it doesn't know something, and then actually say it doesn't know and not make stuff up. If it was capable of reasoning you should be able to convince it using _reason_, to do exactly that.

However, you

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#644

Post by yencabulator »

bccdee wrote: post #567
meroes wrote: post #481

Earlier quoted context omitted.

Teaching an LLM to solve a full sized Sudoku is not a goal right now. As an RLHF I’d estimate it would take 10-20 hours for a single RLHF’er to guide a model to the right answer for a single board. Then you’d need thousands of these for the model (or next model) to ingest. And each RLHF’s work needs checking which at least doubles the hours per task. It can’t do it because RLHF’ers haven’t taught models on large enou…

If it were actually reasoning generally, though, it wouldn't need to be trained on each game. It could be told the rules and figure things out from there.

Even worse, the LLM is supposed to already "know" Sudoku rules. Either that, or it doesn't "know" anything that was scraped from the web...

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#645

Post by yencabulator »

s2radhak wrote: post #342

Fascinating. The article repeatedly makes the claim that “LLMs work by predicting likely next words in a string of text”. Yet there’s the seemingly contradictory implication that we don’t know how LLMs work (ie we don’t know their secret sauce). How does one reconcile this? They’re either fancy autocompletes, or magic autocompletes (in which case the magic qualifier seems more important in understanding what they are…

For inference, we have hand crank that rotates a lot of gears, with a final gear making one token (word) appear in a slot. For learning, we even know how to feed a bunch of text into a complicated thing that tells us what gears to connect to each other and how. We have no idea why the gear ratios and placements are what they are.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#646

Post by interstice »

falcor84 wrote: post #423
interstice wrote: post #365

Earlier quoted context omitted.

Said the turkey to the farmer

I don't think that's how that metaphor works.

Not quite, but it was the closest pithy quote I could think of to convey the point that things can be false for a long time before they are suddenly true without warning.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#647

Post by hnfong »

cristiancavalli wrote: post #614
hnfong wrote: post #587

Earlier quoted context omitted.

IMHO, any argument against LLM intelligence should be validated by first applying them to humans. And then you'd realize that a lot of naïve arguments against LLMs would imply that a significant portion of homo-sapiens can't reason, are unable to really think, and are no more than stochastic parrots. It's actually a rather dangerous line of reasoning.

I’m curious what’s dangerous about it? How do your square the inability to play tic-tac toe or do value comparisons correctly with “we should compare this to a humans reasoning” If it can’t do things like basic value comparison correctly what business do we have saying it “reasons like a human?”

The danger is when LLMs start to outperform humans on many tasks (which they already have), claiming that LLMs are stochastic parrots could be seen to imply that less intelligent people are also no better than stochastic parrots.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#648

Post by cristiancavalli »

hnfong wrote: post #647
cristiancavalli wrote: post #614

Earlier quoted context omitted.

I’m curious what’s dangerous about it? How do your square the inability to play tic-tac toe or do value comparisons correctly with “we should compare this to a humans reasoning” If it can’t do things like basic value comparison correctly what business do we have saying it “reasons like a human?”

The danger is when LLMs start to outperform humans on many tasks (which they already have), claiming that LLMs are stochastic parrots could be seen to imply that less intelligent people are also no better than stochastic parrots.

Who is claiming that implications inevitability?Shutting down a valid line of discussion because of someone deciding to make a fallacious analogy seems like a proposition that would essentially stop all scientific discussion of intelligence more broadly. Also a great argument for limiting free speech/scientific discussion in general. Thoughts are not inherently dangerous unless acted upon in a dangerous way and supposing that some are so dangerous that we should simply not speak of them seems like an action that should be considered more thoroughly beyond “someone might do something rash”

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#649

Post by satiric »

Apologies in advance for nitpicking a small detail.

In lesson 9, you say "The CEO of the Perplexity.ai search platform argues that AI has made blue links obsolete.

"But without sources you are driving blind."

Isn't the entire point of perplexity that it gives you the sources?

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#650

Post by habinero »

interstice wrote: post #646
falcor84 wrote: post #423

Earlier quoted context omitted.

I don't think that's how that metaphor works.

Not quite, but it was the closest pithy quote I could think of to convey the point that things can be false for a long time before they are suddenly true without warning.

How about "Yes, they laughed at Galileo, but they also laughed at Bozo the Clown?"

We heard alllllll the same hype about how revolutionary the blockchain was going to be and look how that turned out.

It's a virtue to point out the emperor has no clothes. It's not a virtue to insist clothes tech is close to being revolutionary and if you just understand it harder, you'd see the space where the clothes go.

Post reply on HN