Since you are on Frankfurtian bullshit, why don't you consider Late G.A. Cohen's take on intellectual bullshit (bullshit perpetuated in the academia), as the latter's notion of bullshit is linked to knowledge, unlike the bullshit we hear from salesmen, showmen, etc.
Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world
301–310 of 652 posts
Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world
#302Not sure why everyone rates this. It’s full of very confidently made statements like “the AI has no ground truth” (obviously it does, it has ingested every paper ever), it “can’t reason logically” which seems like a stretch if you ever read the CoT of a frontier reasoning model and “can’t explain how they arrived at conclusions” where - I mean just try it yourself with o1, go as deep as you like asking how it arrived…
the machine is fooling you with a mimicry of reasoning. and you are falling for it.
If it's mimicry of reason is indistinguishable from real reasoning, how is it not reasoning?
Ultimately, an LLM models language and the process behind it's creation to some degree of accuracy or another. If that model includes a way to approximate the act of reasoning, then it is reasoning to some extent. The extent I am happy to agree is open for discussion, but that reasoning is taking place at all is a little harder to attack.
Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world
#303Not sure why everyone rates this. It’s full of very confidently made statements like “the AI has no ground truth” (obviously it does, it has ingested every paper ever), it “can’t reason logically” which seems like a stretch if you ever read the CoT of a frontier reasoning model and “can’t explain how they arrived at conclusions” where - I mean just try it yourself with o1, go as deep as you like asking how it arrived…
LLMs that use Chain of Thought sequences have been demonstrated to misrepresent their own reasoning [1]. The CoT sequence is another dimension for hallucination.
So, I would say that an LLM capable of explaining its reasoning doesn't guarantee that the reasoning is grounded in logic or some absolute ground truth.
I do think it's interesting that LLMs demonstrate the same fallibility of low quality human experts (i.e. confident bullshitting), which is the whole point of the OP course.
I love the goal of the course: get the audience thinking more critically, both about the output of LLMs and the content of the course. It's a humanities course, not a technical one.
(Good) Humanities courses invite the students to question/argue the value and validity of course content itself. The point isn't to impart some absolute truth on the student - it's to set the student up to practice defining truth and communicating/arguing their definition to other people.
Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world
#304Really well done. It is really a challenge for students to navigate their way around the AI landscape. I am definitely considering sharing that with my students. Have you noticed a difference in how your students approach LLMs after taking your course? A possible issue I see is that it is preaching to the choir; a student who is enclined to use LLMs for everything is less likely to engage with the material in the fir…
This is an excellent point about scientific writing. We'll add something to that effect.
We have not taught this course from the web-based materials yet, but it distills much of the two-week unit that we covered in our "Calling Bullshit" course this past autumn. We find that our students are generally very interested to better understand the LLMs that they are using — and almost every one of them does, to vary degree. (Of course there may be some selection bias in that the 180 students who sign up to take a course on data reasoning may be more curious and more skeptical than the average.)
Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world
#305Earlier quoted context omitted.
Yes but people have culpability and responsibility. In places of power or influence, this culpability can lead to being fired, legal action, disbarment, loss of money, etc. so there is a real pressure to be coherent and aligned with reality.
You say this but I have a ton of experience that "can lead" and "there is" come with a gigantic pile of caveats to the extent that they appear to be more false than true. Or they're technically true but practically meaningless. The world has been utterly awash in mass-perpetuated misinformation for at least all of recorded history without any real ability to stem the onslaught. This is not a modern problem just becau…
This is a sidestep imo. People _can_ be held accountable, though they will not always be. Machines add a layer of complexity - money is lost or a life is lost because AI made the call, who bears the burden? Machines _can't_ be held accountable.
Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world
#306> Moreover, a hallucination is a pathology. It's something that happens when systems are not working properly. > When an LLM fabricates a falsehood, that is not a malfunction at all. The machine is doing exactly what it has been designed to do: guess, and sound confident while doing it. > When LLMs get things wrong they aren't hallucinating. They are bullshitting. Very important distinction and again, shows the marke…
If we want to be pedantic about language, they aren't bullshitting. Bulshitting implies an intent to deceive, whereas LLMs are simply trying their best to predict text. Nobody gains anything from using terms closely related to human agency and intentions.
The authors have a specific definition of bullshit that they contrast with lying. In their definition, lying involves intent to deceive; bullshitting involves not caring if you’re deceiving.
Lesson 2, The Nature of Bullshit: “BULLSHIT involves language or other forms of communication intended to appear authoritative or persuasive without regard to its actual truth or logical consistency.”
Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world
#307Not sure why everyone rates this. It’s full of very confidently made statements like “the AI has no ground truth” (obviously it does, it has ingested every paper ever), it “can’t reason logically” which seems like a stretch if you ever read the CoT of a frontier reasoning model and “can’t explain how they arrived at conclusions” where - I mean just try it yourself with o1, go as deep as you like asking how it arrived…
LLMs that use Chain of Thought sequences have been demonstrated to misrepresent their own reasoning [1]. The CoT sequence is another dimension for hallucination. So, I would say that an LLM capable of explaining its reasoning doesn't guarantee that the reasoning is grounded in logic or some absolute ground truth. I do think it's interesting that LLMs demonstrate the same fallibility of low quality human experts (i.e.…
Yes!
First, thank you for the link about CoT misrepresentation. I've written a fair bit about this on Bluesky etc but I don't think much if any of that made it into the course yet. We should add this to lesson 6, "They're Not Doing That!"
Your point about humanities courses is just right and encapsulates what we are trying to do. If someone takes the course and engages in the dialectical process and decides we are much too skeptical, great! If they decide we aren't skeptical enough, also great. As we say in the instructor guide:
"We view this as a course in the humanities, because it is a course about what it means to be human in a world where LLMs are becoming ubiquitous, and it is a course about how to live and thrive in such a world. This is not a how-to course for using generative AI. It's a when-to course, and perhaps more importantly a why-not-to course.
"We think that the way to teach these lessons is through a dialectical approach.
"Students have a first-hand appreciation for the power of AI chatbots; they use them daily.
"Students also carry a lot of anxiety. Many students feel conflicted about using AI in their schoolwork. Their teachers have probably scolded them about doing so, or prohibited it entirely. Some students have an intuition that these machines don't have the integrity of human writers.
"Our aim is to provide a framework in which students can explore the benefits and the harms of ChatGPT and other LLM assistants. We want to help them grapple with the contradictions inherent in this new technology, and allow them to forge their own understanding of what it means to be a student, a thinker, and a scholar in a generative AI world."
Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world
#308Earlier quoted context omitted.
What is reasoning if not a chain of logically consistent thoughts?
fair, but "logically consistent thoughts" is a subject of deep investigation starting from the early euclidean geometry to the modern godel's theorems. ie, that logically consistent thinking starts from symbolization, axioms, proof procedures, world models. otherwise, you end up with persuasive words.
You just ruled out 99% of humans from having reasoning capabilities.
The beautiful thing about reasoning models is that there is no need to overcomplicate it with all the things you've mentioned, you can literally read the model's reasoning and decide for yourself if it's bullshit or not.
Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world
#309Not sure why everyone rates this. It’s full of very confidently made statements like “the AI has no ground truth” (obviously it does, it has ingested every paper ever), it “can’t reason logically” which seems like a stretch if you ever read the CoT of a frontier reasoning model and “can’t explain how they arrived at conclusions” where - I mean just try it yourself with o1, go as deep as you like asking how it arrived…
the machine is fooling you with a mimicry of reasoning. and you are falling for it.
If it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck.
Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world
#310> Moreover, a hallucination is a pathology. It's something that happens when systems are not working properly. > When an LLM fabricates a falsehood, that is not a malfunction at all. The machine is doing exactly what it has been designed to do: guess, and sound confident while doing it. > When LLMs get things wrong they aren't hallucinating. They are bullshitting. Very important distinction and again, shows the marke…
If we want to be pedantic about language, they aren't bullshitting. Bulshitting implies an intent to deceive, whereas LLMs are simply trying their best to predict text. Nobody gains anything from using terms closely related to human agency and intentions.
The authors of this website have published one of the famous books on the topic[0] (along with a course), and their definition is as follows:
"Bullshit involves language, statistical figures, data graphics, and other forms of presentation intended to persuade by impressing and overwhelming a reader or listener, with a blatant disregard for truth and logical coherence."
It does not imply an intent to deceive, just disregard for whether the BS is truth or not. In this case, I see how the definition can apply to LLMs in the sense that they are just doing their best to predict the most likely response.
If you provided them with training data where the majority inputs agree on a common misconception, they will output similar content as well.