Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

thebullshitmachines.com

Post reply on HN
Print view View original

541–550 of 652 posts

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#541

Post by ninetyninenine »

daveguy wrote: post #520
ninetyninenine wrote: post #507

Earlier quoted context omitted.

Seems you’re not well versed in basic English. If I can reason it doesn’t mean I’m always reasoning or constantly reasoning or if I know how to do reasoning for every prompt. It just means it’s possible. How narrow or how wide that possibility is, is orthogonal to the claim itself. Please employ logic here. Ok math guy. Imagine I said numbers can be divided. The claim is true even though there is a number that can’t…

If it's only reasoning randomly how do you know when anything has been reasoned properly vs just a generated simulation of reasonable text?

We use Probability. Find a prompt that has a large range aka codomain. If it arrived at the correct answer then that the only possibility here is reasoning because the codomain is so large it cannot arrive there by random chance.

Of course make sure the prompt is unique such that it's not in the data and it's not doing any sort of "pattern matching".

So like all science we prove it via probability. Observations match with theory to a statistical degree.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#542

Post by K0balt »

lazide wrote: post #46
K0balt wrote: post #35

There is a bit of very important content missing from the explanation of the autocomplete analogy. The combination of encoding / tokenization of meanings and ideas, related concepts, and mapping these relationships in vector space makes LLMs not so much glorified text prediction engines as browsers/oracles of the sum total of cultural-linguistic knowledge as captured in the training corpus. Understanding how the impl…

It’s also why they can produce such hard to identify bullshit and harmful output. I’ve had some really convincing, yet fundamentally flawed, code output that if I hadn’t done about a million code reviews before I might have just used. And been totally screwed later. Near as I can tell, that the bullshit is so much more convincing with them is a huge detriment that society really won’t learn to appreciate until it’s g…

This is the big pain point to be sure. Subtly wrong but mostly excellent results.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#543

Post by williamcotton »

meroes wrote: post #515
williamcotton wrote: post #405

Earlier quoted context omitted.

LLMs are so far removed mechanically from brains the idea they reason is not even remotely worth considering. Jet planes are so far removed mechanically from a bird that the idea they fly is not even remotely worth considering.

You’re right that my argument depends upon there being a great physical distinction between brains and H100s or enough water flowing through troughs. But since we knew properties of wings were major comments to flight dating back to beyond the myths of Pegasus or Icarus, we rightly connected the similarities in the flight case. Yet while we have studied neurons and know the brain is apart of consciousness, we don’t k…

Motte? Consciousness.

Bailey? Reason.

How reasonable are the outputs of ANNs considering the inputs? This is a valid question and it has a useful response.

From ImageNet to LLMs we are finding these tools to give some scale of a reasonable response.

Recommended reading: Philosophical Investigations by Wittgenstein.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#544

Post by cristiancavalli »

ninetyninenine wrote: post #536
cristiancavalli wrote: post #512

Earlier quoted context omitted.

I still think the jury is out on this given that they seem to fail on obvious things which are trivially reasoned about by humans. Perhaps they reason differently at which point I would need to understand how this reasoning is different from a humans reasoning (perhaps biological reasoning more generally?) and then I would want to consider whether one ought to call it reasoning given its differences (if there are any…

They can fail at reasoning. But they can demonstrably succeed to. So the the statement that they CAN reason is demonstrably true. Ok if given a prompt where the solution can only be arrived at by reasoning and the LLM gets to the solution for that single prompt, then how can you say it can't reason?

Given your set of theoreticals then I would concede, yes the model is reasoning. At that point, though, the world would probably be far more concerned with your finding of a question that can only be met via reasoning and would be uninfluenced or paralleled by any empirical phenomenon including written knowledge as a medium of transference. The core issue I see here is you being able to prove that the model is actually reasoning in a concrete way that isn’t just a simulacrum like the Apple researchers et al. theorize it to be.

If you do find this question answer pair then it would be a massive breakthrough for science and philosophy more generally.

You say “demonstrably” but I still do not see a demonstration of these reasoning abilities that is not subject to the aforementioned criticisms.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#545

Post by superbatfish »

superbatfish wrote: post #453

The author makes this assertion about LLMs rather casually: >They don’t engage in logical reasoning. This is still a hotly debated question, but at this point the burden of proof is on the detractors. (To put it mildly, the famous "stochastic parrot" paper has not aged well.) The claim above is certainly not something that should be stated as fact to a naive audience (i.e. the authors' intended audience in this case)…

On the other hand, the authors make plenty of other great points -- about the fact that LLMs can produce bullshit, can be inaccurate, can be used for deception and other harms, are now a huge challenge for education.

The fact that they make many good points makes it all the more disappointing that they would taint their credibility with sloppy assertions!

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#546

Post by sgt101 »

fritzo wrote: post #474
sgt101 wrote: post #66

I wish the title wasn't so aggressively anti-tech though. The problem is that I would like to push this course at work, but doing so would be suicidal in career terms because I would be seen as negative and disruptive. So the good message here is likely to miss the mark where it may be most needed.

What would be a better title? "Hallucinating" seems inaccurate. Maybe "Untrustworthy machines"? "Critical thinking"? "Street smarts for humans"? "Social studies including robots"?

How about "How to thrive in a ChatGPT world"?

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#547

Post by pjs_ »

yathaid wrote: post #416
joenot443 wrote: post #396

Earlier quoted context omitted.

It's 1994. Larry Llyod Mayer has read the entire internet, hundreds of thousands of studies across every field, and can answer queries word for word the same as modern LLMs do. He speaks every major language. He's not perfect, he does occasionally make mistakes, but the sheer breadth of his knowledge makes him among the most employable individuals in America. The Pentagon, IBM, and Deloitte are begging to hire him. I…

Does his accuracy take a sudden precipitous fall when going from multiplying two three-digit numbers to two four-digit numbers?

This is a solved problem, ChatGPT uses a python prompt to do arithmetic now. Just like you would… all good. You Can Just Check Your Own Claims

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#548

Post by [deleted] »

jurli wrote: post #500
bjourne wrote: post #452

What I find frightening is how many are willing to take LLM output at face value. An argument is won or lost not on its merits, but by whether the LLM say so. It was bad enough when people took whatever was written on Wikipedia at face value, trusting an LLM that may have hardcoded biases and is munging whatever data it comes across is so much worse.

This is what people said about the internet too. Remember the whole "do not ever use Wikipedia as a source". I mean sure, technically correct, but human beings are generally imprecise and having the correct info 95% of the time is fine. You learn to live with the 5% error

[deleted]

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#549

Post by directevolve »

hunter-gatherer wrote: post #171
aqueueaqueue wrote: post #15

Earlier quoted context omitted.

I made the same sort of mistake with the internet being young back in 93! Having a machine do it for you can easily turn into brain switch off.

I keep telling everyone that the only reason I'm paid well to do "smart person stuff" is not because I'm smart, but because I've steadily watched everyone around me get more stupid over my life as a result of turning their brain switch off. I agree a course like this needs to exist, as I've seen people rely on chatGPT for a lot of information. Just yesterday I demonstrated with some neighbors about how easily it coul…

If men are more likely to die from flu if infected, and women more likely to be infected, an affirmative answer to both questions could be reasonable. When you take into account uncertainty about the goals, knowledge and cognitive capacity of the person asking the question, it's not obvious to me how the AI ought to react to an underspecified question like this.

Edit: When I plug this into a temporary chat on o3-mini, it gives plausible biochemical and behavioral mechanisms that might explain a gender difference in outcomes. Notably, the mechanisms it proposes are the same for both versions of the question, and the framing is consistent.

Specifically, for the "men worse than women" and "women worse than men" questions, it proposes hormone differences, X-linked immune regulatory genes, and medical care-seeking differences that all point toward men having worse outcomes than women. It describes these factors in both versions of the question, and in both versions, describes them as explaining why men have worse outcomes than women.

It doesn't specifically contradict the "women have worse outcomes than men" framing. But it reasons consistently with the idea that men have worse outcomes than women either way the question is posed.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#550

Post by JPLeRouzic »

Mistletoe wrote: post #478
bjourne wrote: post #452

What I find frightening is how many are willing to take LLM output at face value. An argument is won or lost not on its merits, but by whether the LLM say so. It was bad enough when people took whatever was written on Wikipedia at face value, trusting an LLM that may have hardcoded biases and is munging whatever data it comes across is so much worse.

I’d take the Wikipedia answer any day. Millions of eyes on each article vs. a black box with no eyes on the outputs.

> "Millions of eyes on each article"

Only a minority of users contribute regularly (126,301 have edited in the last 30 days):

https://en.wikipedia.org/wiki/Wikipedia:Wikipedians#Number_o...

And there are 6,952,556 articles in the English Wikipedia, so an average article is corrected every 55 months (more than 4 years).

It's hardly "Millions of eyes on each article"

Post reply on HN