Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

thebullshitmachines.com

Post reply on HN
Print view View original

331–340 of 652 posts

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#331

Post by quantified »

ZephyrBlu wrote: post #308
bwfan123 wrote: post #299

Earlier quoted context omitted.

fair, but "logically consistent thoughts" is a subject of deep investigation starting from the early euclidean geometry to the modern godel's theorems. ie, that logically consistent thinking starts from symbolization, axioms, proof procedures, world models. otherwise, you end up with persuasive words.

You just ruled out 99% of humans from having reasoning capabilities. The beautiful thing about reasoning models is that there is no need to overcomplicate it with all the things you've mentioned, you can literally read the model's reasoning and decide for yourself if it's bullshit or not.

That's sort of arrogant, Most of that 99 (if that many) % could learn if inspired to and provided resources. And does use reasoning and instinct in day-to-day life even if it's as simple as "I'll take go shopping before I take my car to the shop so I have the groceries" or "hide this money in a new place so my husband doesn't drink it away". Models will get better over time, and yes humans only use models too.

Humans rely in cues to tell when each other is fabricating or lying. Machines don't have those cues, and fabricate their reasoning too. So we have a complicatedly difficult time trusting them.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#332

Post by habinero »

TeMPOraL wrote: post #250
fancyfredbot wrote: post #245

Earlier quoted context omitted.

There's obviously nothing wrong with learning by reading, but the way you tell whether what you read is true is by seeing whether or not it fits in with observation of reality. That's the reason we're no longer reading the books about phlogiston.

> the way you tell whether what you read is true is by seeing whether or not it fits in with observation of reality The only way any of us ever gets to see "whether or not it fits in with observation of reality" is to see if they get an A or F on the test asking it. Seriously. The "moons of Jupiter" question is the only one of the above one gets to connect to an observation independent of humans, and then if they did…

What? That's (1) not true and (2) says, uh, a lot of unintentional things about the way you approach the world. I'm not sure you realize quite how it makes you look.

For one, it's not even internally consistent -- the people who built telescopes and satellites didn't "see" the moons, either. They got back a bunch of electrical signals and interpreted it to mean something. This worldview essentially boils down to the old "brain in a jar" which is fun to think about at 3am when you're 21 and stoned, but it's not otherwise useful so we discard it.

For another, "how many moons does Jupiter have" doesn't have a correct answer, because it doesn't have an answer. There is no objective definition of what a "moon" is. There's not even a precise IAU technical definition. Jupiter has rings that are constantly changing, every single particle of those could be considered a moon if you want to get pedantic enough.

I'm always a bit shocked and disappointed with people when they go "well, you learn it on a test and that's how you know" because ...no, no that's not at all how it works. The most essential part of learning is knowing how we know and knowing how certain we are in that conclusion.

"Jupiter has 95 moons" is not a useful or true fact. "Jupiter has 95 named moons and thousands of smaller objects orbiting it and the International Astronomical Union has decided it's not naming any more of them." is both useful and predictive [0] because you know there isn't going to be any more unless something really wild happens.

[0] https://science.nasa.gov/jupiter/jupiter-moons/

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#333

Post by throwaway4aday »

eigenform wrote: post #316
pjs_ wrote: post #288

Not sure why everyone rates this. It’s full of very confidently made statements like “the AI has no ground truth” (obviously it does, it has ingested every paper ever), it “can’t reason logically” which seems like a stretch if you ever read the CoT of a frontier reasoning model and “can’t explain how they arrived at conclusions” where - I mean just try it yourself with o1, go as deep as you like asking how it arrived…

It depends on your tolerance for error. When you have a machine that can only infer rules for reasoning from inputs [which are, more often than not, encoded in a very roundabout way within a language which is very ambiguous, like English], you have necessarily created something without "ground." That's obviously useful in certain situations (especially if you don't know the rules in some domain!), but it's categorica…

Are you contending that every human derives their reasoning from first principals rather than being taught rules in a natural language?

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#334

Post by mr_toad »

rcxdude wrote: post #167
TheOtherHobbes wrote: post #106

Earlier quoted context omitted.

Many claims don't stand up to scrutiny, and some look suspiciously like training to the test. The Apple study was clear about this. LLMs and their related modal models lack the ability to abstract information from noisy text inputs. This is really obvious if you play with any of the art generators. For example - the understanding of basic prepositions just isn't there. You can't say "Put this thing behind/over/in fro…

Part of this is that the art generators tend to use CLIP, whjch is not a particularly good text model, often only being slightly better than a bag of words, which makes many interactions and relationships pretty difficult to represent. Some of the newer ones have better frontends which improve this situation, though. I think color is fairly well abstracted, but most image generators are not good for edits, because th…

> I think color is fairly well abstracted, but most image generators are not good for edits, because the generator more or less starts from scratch

It’s unlikely that the models have been trained on “similarity”. Ask it to swap red boots for brown boots and it will happily generate an entirely different image because it was never trained on the concept of images being similar.

That doesn’t mean it’s impossible to train an LLM on the concept of similarity.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#335

Post by sabas123 »

hirenj wrote: post #86

This is a great resource, thanks. We (myself, a bioinformatician, and my co-cordinators, clinicians) are currently designing a course to hopefully arm medical students with the required basic knowledge they need to navigate the changing world of medicine in light of the ML and LLM advances. Our goal is to not only demystify medical ML, but also give them a sense of the possibilities with these technologies, and maybe…

> currently designing a course to hopefully arm medical students with the required basic knowledge they need to navigate the changing world of medicine in light of the ML and LLM advances

Could you share what you think would be some key basic points what they should learn? Personally I see this landscape changing so insanely much that I don't even know what to prepare for.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#336

Post by SpicyLemonZest »

timewizard wrote: post #251

It's not a "ChatGPT world." You can thrive just fine by entirely ignoring it and all the snake oil vendors living in it. 4 years and all they have to show for it is absurdly powerful video cards, sub 90% accuracy where it matters, and the only application is "chat bot." It's a fad. Wake up everyone.

You can entirely ignore it only in the sense that sticking your head in the sand is an option. A small but growing fraction of the text you read and images you see were generated by an LLM, and since 2023 at the latest they've been good enough that you cannot reliably tell which fraction it is.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#337

Post by mr_toad »

ctbergstrom wrote: post #322
mr_toad wrote: post #318

Earlier quoted context omitted.

Current LLMs are not the end-all of LLMs, and chain of thought frontier models are not the end-all of AI. I’d be wary of confidently claiming what AI can and can’t do, at the risk of looking foolish in a decade, or a year, or at the pace things are moving, even a month.

That's entirely true. We've tried hard to stick with general principles that we don't think will readily be overturned. But doubtless we've been too assertive for some people's taste and doubtless we'll be wrong in places. Hence the choice to develop not a static book but rather living document that will evolve with time. The field is developing too fast for anything else. With respect to what the future brings, we d…

> we don't think will readily be overturned

I think that’s entirely the problem. You’re making linear predictions of the capabilities of non-linear processes. Eventually the predictions and the reality will diverge.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#338

Post by habinero »

layoric wrote: post #292
nmca wrote: post #221

(while I work at OAI, the opinion below is strictly my own) I feel like the current version is fairly hazardous to students and might leave them worse off. If I offer help to nontechnical friends, I focus on: - look at rate of change, not current point - reliability substantially lags possibility, by maybe two years. - adversarial settings remain largely unsolved if you get enough shots, trends there are unclear - ig…

How does this help the students with their use of these tools in the now, to not be left worse off? Most of the points you list seem like defending against criticism rather than helping address the harm.

Agree. It's also a virtue to point out the emperor has no clothes and the tailor peddling them is a bullshit artist.

This is no different than the crypto people who insisted the blockchain would soon be revolutionary and used for everything, when in reality the only real use case for a blockchain is cryptocoins, and the only real use case for cryptocoins is crime.

The only really good use case for LLMs is spam, because it's the only use case for generating a lot of human-like speech without meaning.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#339

Post by nullc »

pjs_ wrote: post #288

Not sure why everyone rates this. It’s full of very confidently made statements like “the AI has no ground truth” (obviously it does, it has ingested every paper ever), it “can’t reason logically” which seems like a stretch if you ever read the CoT of a frontier reasoning model and “can’t explain how they arrived at conclusions” where - I mean just try it yourself with o1, go as deep as you like asking how it arrived…

> I mean just try it yourself with o1, go as deep as you like asking how it arrived at a conclusion

I don't mean to disagree overall, but on this point the LLM can post-facto rationalize its output but it has no introspection and has absolutely no idea why it made a given bit of output (except in so far as it was a result of COT which it could reiterate to you). The set of weights being activated could be nearly disjoint when answering and explaining the answer.

One can also make the same argument about humans -- that they can't introspect their own minds and are just posthoc rationalizing their explanations unless their thinking was a product of an internal monolog that they can recount. But humans have a lifetime of self-interaction that gives a good reason to hope that their explanations actually relate to their reasoning. LLM's do not.

And LLMs frequently give inconsistent results, it's easy to demonstrate the posthoc nature of LLM's rationalizations too: Edit the transcript to make the LLM say something it didn't say and wouldn't have said (very low probability), and then have it explain why it said that.

(Though again, split brain studies show humans unknowingly rationalizing actions in a similar way)

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#340

Post by SpicyLemonZest »

mr_toad wrote: post #334
rcxdude wrote: post #167

Earlier quoted context omitted.

Part of this is that the art generators tend to use CLIP, whjch is not a particularly good text model, often only being slightly better than a bag of words, which makes many interactions and relationships pretty difficult to represent. Some of the newer ones have better frontends which improve this situation, though. I think color is fairly well abstracted, but most image generators are not good for edits, because th…

> I think color is fairly well abstracted, but most image generators are not good for edits, because the generator more or less starts from scratch It’s unlikely that the models have been trained on “similarity”. Ask it to swap red boots for brown boots and it will happily generate an entirely different image because it was never trained on the concept of images being similar. That doesn’t mean it’s impossible to tra…

I just asked Midjourney to do precisely that, and it swapped the boots with no issue, although it didn't seem to quite understand what it meant for a cat to _wear_ boots.
Post reply on HN