Live data from Hacker News

Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

thebullshitmachines.com

221–230 of 652 posts

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#221
(while I work at OAI, the opinion below is strictly my own)

I feel like the current version is fairly hazardous to students and might leave them worse off.

If I offer help to nontechnical friends, I focus on:

- look at rate of change, not current point

- reliability substantially lags possibility, by maybe two years.

- adversarial settings remain largely unsolved if you get enough shots, trends there are unclear

- ignore the parrot people, they have an appalling track record prediction-wise

- autocorrect argument is typically (massively) overstated because RL exists

- doomers are probably wrong but those who belittle their claims typically understand less than the doomers do

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#222
post #110

People said the world wouldn't need more than 10 computers. Newyork times ridiculed space industry and etc.. AI is gonna disrupt all industries

People also didn't anticipate how social media and surveillance capitalism would shit up our lives.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#223
> After talking to literally hundreds of educators, employers, researchers, and policymakers, we have spent the last eight months developing the course on large language models (LLMs) that we think every college freshman needs to take.

Did you consider consulting with any college freshman or even college students? I know you're supposed to be guiding their education, but I think it's also good to check in and see what they care about learning.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#224
> LLMs are not capable of reflecting on and reporting about how or why they do what they do.

i get the why, but about the how:

deepseek has shown it's able to explain it's reasoning, how it reach a conclusion. That's quite a far stretch from the llm being only able to estimate statistically what's the right word to put after another one.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#225
Really enjoying this. Thank you for the great work. I'm currently on Lesson 11 and noticed a couple typos (missing words). I haven't found anywhere on the site itself where I could send feedback to report such a thing (maybe I missed it). Hopefully you aren't offended if I post them here.

I think the easiest way to point them out is to just have you search for the partial line of text while on Lesson 11 and you'll see the spots.

"No one is going to motivated by a robotic..." (missing the word "be")

"People who are given a possible solution to a problem tend to less creative at..." (again missing the word "be")

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#226

Do you feel that you may be being a bit provocative by calling LLM's 'bullshit machines'? I understand the frustration as I've been bullshitted by these models just as much as the next programmer, but surely with recent advancements in RAG and reasoning, they're not just 'bullshit machines' at this point, are they?

“Bullshit” actually means something: > In philosophy and psychology of cognition, the term "bullshit" is sometimes used to specifically refer to statements produced without particular concern for truth, clarity, or meaning, distinguishing "bullshit" from a deliberate, manipulative lie intended to subvert the truth. https://en.m.wikipedia.org/wiki/Bullshit It’s really an ideal term to describe what LLMs do.

> It’s really an ideal term to describe what LLMs do.

Only when you relax it to the point it also describes what most people do.

To be specific: if by "truth" you mean objective, verifiable truth, and by "caring about truth" you mean caring about objective, empirically verifiable evidence, then by far most people are mostly only ever trying to be persuasive.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#227

Earlier quoted context omitted.

Ground truth is possibly used in the sense that humans’ brains tie what they create to the properties of observed reality. Whatever new information comes in is compared to, or checked by, that. Whereas, LLM’s will believe anything you feed them in training no matter how unrealistic it is. I do think that, after much training data, they do have specific beliefs that are ingrained in them. That changing those is diffic…

> Ground truth is possibly used in the sense that humans’ brains tie what they create to the properties of observed reality. Whatever new information comes in is compared to, or checked by, that. What do you compare your knowledge of history to, other than what you expect other people will say ? Most knowledge we learn in life is tied only to our expectations of other people's reactions. Which works out fine, most of…

To start with, we know what a person is, rudimentary things about how they behave, our senses, how they commonly work, and can do mental comparisons (reality checks).

We know LLM’s don’t start with that because we initialize them with zero’d or random weights.

Then, their training data can be far more made up, even works of fiction, that the reality most humans observe with is almost always real. We could raise a human in VR or something where technically there would be comparisons. Most humans’ observations connect to expectations in their brain which was designed for the reality we operate in.

Finally, the brain has multiple components that each handle different jobs. They have different architectures to do those jobs well. Sections include language, numbers, reasoning, tiers of memory, hallucination prevention, mirroring, and even meta-level stuff like reward adjustment. We don’t just speculate that they do different things: damage to those regions shuts down those abilities. Tied to the physical realm are vision, motor, and spatial areas. We can feel objects, even temperature or pressure changes. That we can do a lot of that without supervised learning shows we’re tailor-made by God for success in this world.

LLM’s have one architecture that does one job which we try to get to do other things, like reasoning or ground truth. We pretend it’s something it’s not. The multimodal LLM’s are getting closer with specialized components. Even they aren’t all trained in a coherent way using real-world, observations in the senses. There’s usually a gap between systems like these and what the brain does in the real world just in how it gets reliable information about its operating environment.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#228

Do you feel that you may be being a bit provocative by calling LLM's 'bullshit machines'? I understand the frustration as I've been bullshitted by these models just as much as the next programmer, but surely with recent advancements in RAG and reasoning, they're not just 'bullshit machines' at this point, are they?

I wrote a post below about how AI hallucinated a whole regulation that didn’t exist which has been flagged for some reason. I have colleagues who have had arguments with clients who have asked AI questions about planning law and been given bullshit which they then insist is true and they can’t understand why thier architects won’t submit the appeal that they’re asking for. I think we’re in an era where any text, true…

It was flagged because it’s a copy paste of the same comment you made ten days ago.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#229

> LLMs are not capable of reflecting on and reporting about how or why they do what they do. i get the why, but about the how: deepseek has shown it's able to explain it's reasoning, how it reach a conclusion. That's quite a far stretch from the llm being only able to estimate statistically what's the right word to put after another one.

Their arguments are just one special-pleading fallacy after another. "But... but... but... it's different when we do it!"

No, it's not different when we do it. The takeaway here isn't that the AI algorithms are so special and magical, it's that our brains are not. It takes some nerve (literally) for humans to throw around labels like "bullshit machine."

The only advantage we really have is long-term memory. I'm sure that will be addressed soon enough. Someone will figure out the ANN analogue of memory consolidation during sleep, and that'll be the tipping point.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#230
post #84

Earlier quoted context omitted.

This is patently false. They are trained to generate correct responses.

Then comes the question of what is a correct response... ps: I fail to detect whether your comment was ironic or not.

There are different criteria in use for that. But sycophantic behavior is not the goal. It's something model builders actively try to prevent.
Post reply on HN