Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world • Hacker News Board

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#201

Post by somebehemoth » Sun, Feb 09, 2025, 4:14 PM UTC

tim333 wrote: post #193

roland35 wrote: post #130

I like the term "bullshit" over "hallucination". These AI machines have no perceptions, no concept of truth, so indeed they are just spewing out words with no regard for truth. And unfortunately the cost of spreading bullshit has gone down to almost 0.

I dislike the term bullshit because it's use regarding ChatGPT does not match the dictionary version "stupid or untrue talk or writing; nonsense". If your sentence "LLMs output is bullshit" is wrong you may be better off changing the sentence than rewriting the dictionary to fit your sentence. I mean you can redefine words if you like, like how young people use sick and bad to mean much the opposite of what they did,…

I think a hallucinated sentence embedded in a paragraph of truth fits the definition: stupid and nonsense. A bullshitter can be right sometimes or even most of the time. They are still a bullshitter.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#202

Post by api » Sun, Feb 09, 2025, 4:20 PM UTC

shaggie76 wrote: post #185

I was thinking how this article claims that people crave the authenticity of live music and that bullshit-generators will never be able to supply that. At first, I saw this as a reason for optimism, but then I got to thinking about evidence that people may not necessarily want authenticity after all. Organic produce was the first metaphor the came to mind: it's probably more healthy for you even if it isn't as pretty…

Organic produce isn’t a good example because it’s a dodgy poorly defined concept and it’s not clear that it’s either better for you or better tasting.

The thing about art is this: art is a message from a human to another human.

If I want art I want it to be that. I don’t want a numerical average of all past messages, which is what LLMs and diffusion models create. I also don’t want randomness or gimmickry, which is why I dislike a lot of pretentious modern art.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#203

Post by TeMPOraL » Sun, Feb 09, 2025, 4:21 PM UTC

ImPleadThe5th wrote: post #100

TeMPOraL wrote: post #95

"The LLMs have no ground truth" claim (around chapter 2) that's core to the "bullshit machines" argument is itself wrong. Of course LLMs have ground truth. What do the authors think here, that the text in training corpus is random ? Hint: it isn't. Real conversations are anything but random. There's a lot of information hidden in "statistical ordering of the words", because the distribution is not arbitrary . Statist…

So if I ask ChatGpt about bears and in the middle of explaining their diet it tells me something about how much they like porridge and in the middle of habitat it tells me they live in a quaint cabin in the woods, that's ... True? Statistically we certainly have a lot of words about 3 bears and their love for porridge. That doesn't mean it's true, it just means it's statistically significant. If I asked someone a sci…

ChatGPT has enough dimensions in its latent space to represent and distinguish between the various meanings of porridge and is able to be informed by the Goldilocks story without switching to it mid-sentence.

It's actually a good example of what I have in mind by saying human text isn't random. The Goldilocks story may not be scientific, but it's still highly correlated with scientific truth about matters like food, bears, or the daily lives of people. Put yourself in the shoes of an alien trying to make heads or tails of that story, you'll see just how many things in it are not arbitrary.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#204

Post by TeMPOraL » Sun, Feb 09, 2025, 4:23 PM UTC

vaidhy wrote: post #113

ImPleadThe5th wrote: post #100

Earlier quoted context omitted.

So if I ask ChatGpt about bears and in the middle of explaining their diet it tells me something about how much they like porridge and in the middle of habitat it tells me they live in a quaint cabin in the woods, that's ... True? Statistically we certainly have a lot of words about 3 bears and their love for porridge. That doesn't mean it's true, it just means it's statistically significant. If I asked someone a sci…

If that were the case, then you are right.. However, the current crop of LLMs seem to be good at understanding the context. A scientific data point about bears is unlikely to have Goldilocks in there (unless talking about evolution of life and Goldilocks zone). You can argue that there is meaning hidden in words that is not captured by words themselves in a given context - psychic knowledge as opposed to reasoned out…

Words don't carry meaning. Meaning exists in how words are or are not used together with other words. That is, in their.... statistical relationships to each other!

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#205

Post by empath75 » Sun, Feb 09, 2025, 4:32 PM UTC

fancyfredbot wrote: post #122

TeMPOraL wrote: post #95

"The LLMs have no ground truth" claim (around chapter 2) that's core to the "bullshit machines" argument is itself wrong. Of course LLMs have ground truth. What do the authors think here, that the text in training corpus is random ? Hint: it isn't. Real conversations are anything but random. There's a lot of information hidden in "statistical ordering of the words", because the distribution is not arbitrary . Statist…

Statistical truths based on observation of reality is the basis for science. Statistical truth based on the text on the internet is the basis for something else, and I would personally not like to call whatever that is science, or any truths established this way "ground truths".

What year did the Normans invade England?

What's Newton's second law?

Who was the last czar of Russia?

How many moons does Jupiter have?

I bet you "know" a lot of those facts not because you have observed them empirically, but because you read about them in books. And in fact, almost all scientists rely on reading for nearly everything they know about science, including nearly everything they know about their own specialties. Nobody has time to derive all of their knowledge of the world from personal observation, and most people who can read have probably learned almost all "facts" they know about the world from books.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#206

Post by cratermoon » Sun, Feb 09, 2025, 4:32 PM UTC

api wrote: post #202

shaggie76 wrote: post #185

I was thinking how this article claims that people crave the authenticity of live music and that bullshit-generators will never be able to supply that. At first, I saw this as a reason for optimism, but then I got to thinking about evidence that people may not necessarily want authenticity after all. Organic produce was the first metaphor the came to mind: it's probably more healthy for you even if it isn't as pretty…

Organic produce isn’t a good example because it’s a dodgy poorly defined concept and it’s not clear that it’s either better for you or better tasting. The thing about art is this: art is a message from a human to another human. If I want art I want it to be that. I don’t want a numerical average of all past messages, which is what LLMs and diffusion models create. I also don’t want randomness or gimmickry, which is w…

I didn't want to hijack the thread too much but yeah, organic farming was not originally about better for the consumer. The origin of organic farming methods was about taking care of the land and the ecosystems in which farming happens. The end products happened to be healthier, in some cases, because of reduced usage of chemicals known to be harmful resulted in less of them in the product at retail.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#207

Post by tim333 » Sun, Feb 09, 2025, 4:34 PM UTC

somebehemoth wrote: post #201

tim333 wrote: post #193

Earlier quoted context omitted.

I dislike the term bullshit because it's use regarding ChatGPT does not match the dictionary version "stupid or untrue talk or writing; nonsense". If your sentence "LLMs output is bullshit" is wrong you may be better off changing the sentence than rewriting the dictionary to fit your sentence. I mean you can redefine words if you like, like how young people use sick and bad to mean much the opposite of what they did,…

I think a hallucinated sentence embedded in a paragraph of truth fits the definition: stupid and nonsense. A bullshitter can be right sometimes or even most of the time. They are still a bullshitter.

Everyone gets things wrong sometimes.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#208

Post by empath75 » Sun, Feb 09, 2025, 4:34 PM UTC

ImPleadThe5th wrote: post #100

TeMPOraL wrote: post #95

"The LLMs have no ground truth" claim (around chapter 2) that's core to the "bullshit machines" argument is itself wrong. Of course LLMs have ground truth. What do the authors think here, that the text in training corpus is random ? Hint: it isn't. Real conversations are anything but random. There's a lot of information hidden in "statistical ordering of the words", because the distribution is not arbitrary . Statist…

So if I ask ChatGpt about bears and in the middle of explaining their diet it tells me something about how much they like porridge and in the middle of habitat it tells me they live in a quaint cabin in the woods, that's ... True? Statistically we certainly have a lot of words about 3 bears and their love for porridge. That doesn't mean it's true, it just means it's statistically significant. If I asked someone a sci…

LLM's demonstrably don't do this, nor do they say that they live in the hundred acre woods and love honey. Unless you ask about a specific bear.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#209

Post by tmnvdb » Sun, Feb 09, 2025, 4:39 PM UTC

TheOtherHobbes wrote: post #106

tmnvdb wrote: post #71

Hmm, it seems that the author takes very clear (and sometimes cynical) positions on some controversial questions. For example, "They don't have the capacity to think through problems logically." is an hotly debated claim, and I think with the advent of reasoning models this has at least become something one should not state in entry level material, which would hopefully reflect common understanding rather than the au…

Many claims don't stand up to scrutiny, and some look suspiciously like training to the test. The Apple study was clear about this. LLMs and their related modal models lack the ability to abstract information from noisy text inputs. This is really obvious if you play with any of the art generators. For example - the understanding of basic prepositions just isn't there. You can't say "Put this thing behind/over/in fro…

Regarding the apple paper: https://andrewmayne.com/2024/10/18/can-you-dramatically-impr...

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#210

Post by TeMPOraL » Sun, Feb 09, 2025, 4:43 PM UTC

nickpsecurity wrote: post #173

TeMPOraL wrote: post #95

"The LLMs have no ground truth" claim (around chapter 2) that's core to the "bullshit machines" argument is itself wrong. Of course LLMs have ground truth. What do the authors think here, that the text in training corpus is random ? Hint: it isn't. Real conversations are anything but random. There's a lot of information hidden in "statistical ordering of the words", because the distribution is not arbitrary . Statist…

Ground truth is possibly used in the sense that humans’ brains tie what they create to the properties of observed reality. Whatever new information comes in is compared to, or checked by, that. Whereas, LLM’s will believe anything you feed them in training no matter how unrealistic it is. I do think that, after much training data, they do have specific beliefs that are ingrained in them. That changing those is diffic…

> Ground truth is possibly used in the sense that humans’ brains tie what they create to the properties of observed reality. Whatever new information comes in is compared to, or checked by, that.

What do you compare your knowledge of history to, other than what you expect other people will say? Most knowledge we learn in life is tied only to our expectations of other people's reactions. Which works out fine, most of the time, because even with questions of scientific fact, it's enough some people are in position to ground information in empirical experiments, and then set everyone else's expectations based on that.