Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

thebullshitmachines.com

Post reply on HN
Print view View original

501–510 of 652 posts

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#501

Post by hennell »

ctbergstrom wrote: post #274
abzolv wrote: post #101

Your scroll-to-death user interface made me close the window before the end of the second page. Did you ask an LLM to recommend the most user-friendly UI to you?

We asked our target audience, 19-year-olds. They had a strong preference for this style. I know....

Aside from some of the long gaps between text I didn't think it was so bad. And I wholeheartedly approve of a process that checks the preferences of the target audience even if it's not what I (or they) would pick.

However I can't imagine anyone tests well with the video content. The discussion on teachers using AI generated slides (lesson 2) was really interesting, but it had to fight my desire to stop that awful audio. Clearly the sound recording didn't go well and you have what you have, but at least edit it so the three talkers are at some sort of consistent volume. I was raising and lowering trying to make out what was said from one speaker then being deafened by the next.

(To combat the poor sound, and make it more accessible, could be worth looking at adding subtitles. A fun opportunity to play with AI subtitling systems maybe ;) )

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#502

Post by ninetyninenine »

cristiancavalli wrote: post #493
ninetyninenine wrote: post #485

Earlier quoted context omitted.

I can prove LLMs can reason. You cannot prove LLMs can't reason. This is easily demonstrable. LLMs failing to reason is not proof LLMs can't reason, it's just proof that an LLM didn't reason for that prompt. All I have to do is show you one prompt with a correct answer that cannot be arrived at with pattern matching and the prompt can only be arrived at through reasoning. One. You have to demonstrate this for EVERY p…

No I can “prove” it — look at any number of cases where LLMs can’t even do basic value comparisons despite being claimed as super intelligent. You can try and say well that’s a limitation of the technology and then I would reply — yes and that’s why I would say it’s not reasoning according the original human definition. Also you have yet to produce any evidence of reasoning and claiming you can over and over again do…

LLMs can reason they just don’t always reason.

That’s the claim everyone makes. That is a human definition if it reasoned one time correctly. That is the colloquial definition.

Someone who has brain damage can reason correctly on certain subjects and incorrectly on other subjects. This is an immensely reasonable definition. I’m not being pedantic or out of line here when I say LLMs can reason while using this definition.

Nobody is making the claim that LLMs reason like humans or are human or reason perfectly every time. Again the claim is: LLMs are capable of reasoning.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#503

Post by ninetyninenine »

cristiancavalli wrote: post #494
ninetyninenine wrote: post #486

Earlier quoted context omitted.

Then I'll come up with a prompt such that the answer can only be arrived at via reasoning. I only have to demonstrate this once to prove LLMs CAN reason.

I don’t think this is the watertight case you think it is, furthermore good luck proving with closed models that your question that’s never been asked in any form or derivation (supposedly) is not in the training data.

It’s water tight if the claim is only LLMs CAN reason.

No one is making the claim that LLMs reason like humans or always reason correctly. Ask anyone who makes a claim similar to mine. We are all ONLY making the claim that LLMs can reason correctly. That is a small claim.

The counterclaim is LLMs can’t reason and that is a vastly expansive claim that is ludicrously unprovable.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#504

Post by ninetyninenine »

wruza wrote: post #471
ninetyninenine wrote: post #468

Earlier quoted context omitted.

It’s stupid. You can prove that LLMs can reason by simply giving it a novel problem where no data exists and having it solve that problem. LLMs CAN reason. Whether it can’t reason is not provable. To prove that you have to give the LLM every possible prompt that it has no data for and effectively show it never reasons and gets it wrong all the time. Not only is the proof impossible but it’s already been falsified as…

You can prove that LLMs can reason by simply giving it a novel problem where no data exists and having it solve that problem They scan a hyperdimensional problem space whose facetness and capacity a single human is unable to comprehend. But there potentially exist a slice that corresponds to a problem that is novel to a human. LLMs are completely alien to us both in capabilities and technicalities, so talking about w…

Reasoning is an abstract term. It doesn’t need to be similar to human reasoning. It just needs to be able to arrive at the answer through a process.

Clearly we used the term reasoning for many varied techniques. The term doesn’t narrow to specifically one form of “human” like reasoning only.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#505

Post by seliopou »

jurli wrote: post #500
bjourne wrote: post #452

What I find frightening is how many are willing to take LLM output at face value. An argument is won or lost not on its merits, but by whether the LLM say so. It was bad enough when people took whatever was written on Wikipedia at face value, trusting an LLM that may have hardcoded biases and is munging whatever data it comes across is so much worse.

This is what people said about the internet too. Remember the whole "do not ever use Wikipedia as a source". I mean sure, technically correct, but human beings are generally imprecise and having the correct info 95% of the time is fine. You learn to live with the 5% error

A buddy won a bet with me by editing the relevant Wikipedia article to agree with his side of the wager.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#506

Post by Loughla »

Mistletoe wrote: post #478
bjourne wrote: post #452

What I find frightening is how many are willing to take LLM output at face value. An argument is won or lost not on its merits, but by whether the LLM say so. It was bad enough when people took whatever was written on Wikipedia at face value, trusting an LLM that may have hardcoded biases and is munging whatever data it comes across is so much worse.

I’d take the Wikipedia answer any day. Millions of eyes on each article vs. a black box with no eyes on the outputs.

Even Wikipedia is a problem though. There are so many pages now that self-reference is almost impossible to detect. Meaning, the citation of a statement made on Wikipedia that uses an outside article for reference, which is an article that was originally written using that very Wikipedia article as its own citation.

It's all about trust. Trust the expert, or the crowd, or the machine.

They're all able to be gamed.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#507

Post by ninetyninenine »

more-nitor wrote: post #491
ninetyninenine wrote: post #489

Earlier quoted context omitted.

No. My claim is it can reason. So my claim is along the lines of it can make claims that are within bounds such as 1 to 5 or it can make claims not within those bounds. The opposing claim unbounded. It says LLMs can't reason period. They are making the claim that it is 100% for all possible prompts. No one is making the claim LLMs reason all the time and always. They don't. The claim is that they CAN reason. Versus t…

your claim (hypothesis): LLMs can reason your evidence: "it works with these inputs I tried!" ...hmm seems you're not quite versed in basic mathematical proofs?

Seems you’re not well versed in basic English.

If I can reason it doesn’t mean I’m always reasoning or constantly reasoning or if I know how to do reasoning for every prompt. It just means it’s possible. How narrow or how wide that possibility is, is orthogonal to the claim itself. Please employ logic here.

Ok math guy. Imagine I said numbers can be divided. The claim is true even though there is a number that can’t be divided. Zero.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#508

Post by AlienRobot »

wruza wrote: post #488
AlienRobot wrote: post #462

Earlier quoted context omitted.

I feel it's impossible for me to trust LLMs can reason when I don't know enough about LLMs to know how much of it is LLM and how much of it is sugarcoating. For example, I've always felt that having the whole thing being a single textbox is reductive and must create all sorts of problems. This thing must parse natural language and output natural language. This doesn't feel necessary. I think it should have some check…

it should have some checkboxes and numeric entries for some parameters, although I don't know what those parameters would be The only params they have are technical params. You may see these in various tgwebui tabs. Nothing really breathtaking, apart from high temperature (affects next token probability). Is generating natural language part of what an LLM is, or is this a separate program on top of what it does? They…

Thanks, that is very informative!

I have heard about the tokenization process before when I tried stable diffusion, but honestly I can't understand it. It sounds important but it also sounds like a very superficial layer whose only purpose is to remove ambiguity, the important work being done by the next layer in the process.

I believe part of the problem I have when discussing "AI" is that it's just not clear to me what "AI" is. There is a thing called "LLM," but when we talk about LLMs, are we talking about the concept in general or merely specific applications of the concept?

For example, in SEO often you hear the term "search engines" being used as a generic descriptor, but in practice we all know it's only about Google and nobody cares about Bing or the rest of the search engines nobody uses. Maybe they care a bit about AIs that are trying to replace traditional search engines like Perplexity, but that's about it. Similarly, if you talk about CMS's, chances are you are talking about Wordpress.

Am I right to assume that when people say "LLM" they really mean just ChatGPT/Copilot, Bard/Gemini, and now DeepSeek?

Are all these chatbots just locally run versions of ChatGPT, or they're just paying for ChatGPT as a service? It's hard to imagine everyone is just rolling their own "LLM" so I guess most jobs related to this field are merely about integrating with existing models rather than developing your own from scratch?

I had a feeling ChatGPT's "chat" would work like a text predictor as you said, but what I really wish I knew is whether you can say that about ALL LLMs. Because if that's true, then I don't think they are reasoning about anything. If, however, there was a way to make use of the LLM technology to tokenize formal logic, then that would be a different story. But if there is no attempt at this, then it's not the LLM doing the reasoning, it's humans who wrote the text that the LLM was trained on that did the reasoning, and the LLM is just parroting them without understanding what reasoning even is.

By the way, I find it interesting that "chat" is probably one of the most problematic applications the LLMs can have. Like if ChatGPT asked "what do you want me to autocomplete" instead of "how can I help you today" people would type "the mona lisa is" instead of "what is the mona lisa?" for example.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#509

Post by AlienRobot »

MetaWhirledPeas wrote: post #498
AlienRobot wrote: post #461

Earlier quoted context omitted.

I've seen someone use an LLM to summarize a paper to post it on reddit for people who haven't read the paper. Papers have abstracts...

Sounds fun, if only to compare it to the abstract.

You know, these days I think the abstracts are generated by LLMs too. And the paper. Or at least it uses something like Grammarly. If things keep going this ways typos are going to be a sign of academic integrity.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#510

Post by enragedcacti »

ninetyninenine wrote: post #468
cristiancavalli wrote: post #454

Earlier quoted context omitted.

Disagree — proponents of this point still have yet to prove reasoning and other studies agree about “reasoning” being potentially fake/simulated: https://the-decoder.com/apple-ai-researchers-question-openai... Just claiming a capability does not make it true and we have 0 “proof” of original reasoning that can be proved coming from these models. Especially given the potential cheating in current SOTA benchmarks

It’s stupid. You can prove that LLMs can reason by simply giving it a novel problem where no data exists and having it solve that problem. LLMs CAN reason. Whether it can’t reason is not provable. To prove that you have to give the LLM every possible prompt that it has no data for and effectively show it never reasons and gets it wrong all the time. Not only is the proof impossible but it’s already been falsified as…

LLMs CAN read minds. Whether it can’t read minds is not provable.

Literally I invite people to post prompts and correct answers to ChatGPT where it is trivially impossible for it to have known what number you were thinking of. Every one of those examples falsifies the claim that LLMs can’t read minds.

Post reply on HN