Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

thebullshitmachines.com

Post reply on HN
Print view View original

411–420 of 652 posts

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#411

Post by rockskon »

williamcotton wrote: post #404
rockskon wrote: post #393

Earlier quoted context omitted.

Whenever I see someone confidently making a comparison between LLMs and people, I assume they are unserious individuals more interested in maintaining hype around technology than they are in actually discussing what it does.

Someone saying "they feel" something is not a confident remark. Also, there's plenty of neuroscience that is produced by very serious researchers that have no problems making comparisons between human brain function and statistical models. https://en.wikipedia.org/wiki/Bayesian_approaches_to_brain_f... https://en.wikipedia.org/wiki/Predictive_coding

Theories and approaches to study are not rational bases for making comparisons between LLMs and the human brain.

They're bases for studying the human brain - something which we are very much in our infancy of understanding.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#412

Post by cess11 »

pjs_ wrote: post #288

Not sure why everyone rates this. It’s full of very confidently made statements like “the AI has no ground truth” (obviously it does, it has ingested every paper ever), it “can’t reason logically” which seems like a stretch if you ever read the CoT of a frontier reasoning model and “can’t explain how they arrived at conclusions” where - I mean just try it yourself with o1, go as deep as you like asking how it arrived…

Computers are "reasoning" in the same sense they have a "heartbeat".

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#413

Post by beezlewax »

mr_toad wrote: post #318
ctbergstrom wrote: post #307

Earlier quoted context omitted.

Yes! First, thank you for the link about CoT misrepresentation. I've written a fair bit about this on Bluesky etc but I don't think much if any of that made it into the course yet. We should add this to lesson 6, "They're Not Doing That!" Your point about humanities courses is just right and encapsulates what we are trying to do. If someone takes the course and engages in the dialectical process and decides we are mu…

Current LLMs are not the end-all of LLMs, and chain of thought frontier models are not the end-all of AI. I’d be wary of confidently claiming what AI can and can’t do, at the risk of looking foolish in a decade, or a year, or at the pace things are moving, even a month.

That shouldn't give them any more merit that their current iteration deserves.

You could say the same thing about spaceships or self diving cars.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#414

Post by Borealid »

planb wrote: post #409
azakai wrote: post #403

Overall this is very good, but I have one specific note: Lesson 6 says "LLMs aren't conscious." I think I get what you're saying there - they are not conscious in the same way that humans are - but "consciousness" is a highly-debated term without a precise definition, and correspondingly philosophers have no consensus on whether machines in general are capable of it. Here is one of my favorite resources on that, the…

Philosophy aside - how can an LLM be conscious without a memory or manifestation in the real world? It is a function that, given an input, returns an output and stops existing afterwards. You wouldn't argue that f(x)=x^2 is conscious? I would maybe accept debates about whether for example ChatGPT (the whole system that stores old conversations and sends the history along with the current user entry) is conscious - bu…

There's a great exploration of this concept in Permutation City, a science fiction novel by Greg Egan. In the book, a deterministic human brain is simulated (perfectly) in random-access order. This thought experiment addresses all three of your arguments.

I don't see why something that doesn't exist some of the time inherently couldn't be conscious. Saying that something's output is a function of its inputs also doesn't seem to preclude consciousness. Some humans don't have persistent memory, and all humans (so far) don't exist for 99.99999999% of history.

I'm not trying to claim a particular definition of consciousness, but I find the counterarguments you're presenting uncompelling.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#415

Post by randomNumber7 »

pjs_ wrote: post #288

Not sure why everyone rates this. It’s full of very confidently made statements like “the AI has no ground truth” (obviously it does, it has ingested every paper ever), it “can’t reason logically” which seems like a stretch if you ever read the CoT of a frontier reasoning model and “can’t explain how they arrived at conclusions” where - I mean just try it yourself with o1, go as deep as you like asking how it arrived…

> “can’t explain how they arrived at conclusions”

Imagine I would tell my wife, that whenever we have a discussion, her opinion would only be valid when she can explain how she arrived at her conclusion.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#416

Post by yathaid »

joenot443 wrote: post #396
fmbb wrote: post #329

Earlier quoted context omitted.

Training on all papers does not mean the model believes or knows the truth. It is just a machine that spits out words.

It's 1994. Larry Llyod Mayer has read the entire internet, hundreds of thousands of studies across every field, and can answer queries word for word the same as modern LLMs do. He speaks every major language. He's not perfect, he does occasionally make mistakes, but the sheer breadth of his knowledge makes him among the most employable individuals in America. The Pentagon, IBM, and Deloitte are begging to hire him. I…

Does his accuracy take a sudden precipitous fall when going from multiplying two three-digit numbers to two four-digit numbers?

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#417

Post by sgt101 »

mpbart wrote: post #255
beepbooptheory wrote: post #253

Earlier quoted context omitted.

Really? I am curious how this could be disruptive in any meaningful sense. Whose feelings could possibly be hurt? It just feels like it would be getting offended from a course on libraries because the course talks about how sometimes the book is checked out.

Any executive who is fully bought in on the AI hype could see someone in their org recommending this as working against their interest and take action accordingly.

Yes. This is the issue.

"not on board", "anti-innovation", "not a team player", "disruptive", "unhelpful", "negative".

bye bye bye bye....

I see a lot of devs and IC's taking the attitude that "facts are facts" and then getting shocked by a) other people manipulating information to get their way and b) being fired for stating facts that are contrary to received wisdom without any regards to politics.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#418

Post by TeMPOraL »

anon373839 wrote: post #390
TeMPOraL wrote: post #226

Earlier quoted context omitted.

> It’s really an ideal term to describe what LLMs do. Only when you relax it to the point it also describes what most people do. To be specific: if by "truth" you mean objective, verifiable truth, and by "caring about truth" you mean caring about objective, empirically verifiable evidence, then by far most people are mostly only ever trying to be persuasive.

I hope you don't actually mean that. It's a cynical, sad, and - quite frankly - false thing to believe.

Why? I don't have a problem recognizing that "being convincing" to the right people is a very good way of learning and verifying knowledge about objective reality. Just because it's by proxy, doesn't mean it doesn't work. This is how every one of us learns most things, and what's really sad to me is deluding yourself into thinking we're more special than that.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#419

Post by eclecticfrank »

aidos wrote: post #13

This is amazing! I was speaking to a friend the other day who works in a team that influences government policy. One of the younger members of the team had been tasked with generating a report on a specific subject. They came back with a document filled with “facts”, including specific numbers they’d pulled from a LLM. Obviously it was inaccurate and unreliable. As someone who uses LLMs on a daily basis to help me bu…

This is not something only younger people are prone to. I work in a consulting role in IT and have observed multiple colleagues aged 30 and above use LLMs to generate content for reports and presentations without verifying the output.

Reminded me of wikipedia-sourced presentations in high school in the early 2000s.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#420

Post by input_sh »

tim333 wrote: post #188
input_sh wrote: post #126

Earlier quoted context omitted.

Fully agree, in recent weeks I've also started to consider LLMs in a wider context, which is to destroy all trust in the web. The enshittification of search engines, making social media verification meaningless, locking down APIs that used to be public, destroying public datasets, spreading lies about legacy media, the easiness of deploying bots that can sound human in short bursts of text... it's all leading towards…

>destroy all trust in the web Genuine question - how so? If I want to find stuff out I go to wikipedia, nyt, guardian, hn, linked sites and so on. I'm not aware of that lot being noticeably less trustable than in the past? If anything I find getting information more trustable than before in that there are a lot of long from interviews from all sorts of people on youtube where you can get their thoughts directly rathe…

Allow me quote their "prophet" Curtis Yarvin: "you can’t continue to have a Harvard or a New York Times past since perhaps the start of April." (https://www.theguardian.com/us-news/2024/dec/21/curtis-yarvi...)

Harvard's already under attack, Politico's already under attack, "Wokepedia" (as Musk has been calling it) is already under attack.

So... give it a couple of weeks from now.

Post reply on HN