Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

thebullshitmachines.com

Post reply on HN
Print view View original

161–170 of 652 posts

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#161

Post by rcxdude »

ta8645 wrote: post #77
Freak_NL wrote: post #74

Earlier quoted context omitted.

They address this in lesson 2: > According to philosopher Harry Frankfurt, a liar knows the truth and is trying to lead us in the opposite direction. > A bullshitter either doesn't know the truth, or doesn't care. They are just trying to be persuasive. Being persuasive (i.e., churn out convincing prose) is how LLMs were designed to be.

> Being persuasive (i.e., churn out convincing prose) is how LLMs were designed to be. No. They were designed to churn out accurate prose that accurately reflects their model of reality. They're just imperfect. You're being cynical and emotional to use the term bullshit. And again, it anthropomorphizes the LLM, it implies agency.

Where in the loss function of LLM training is the relationship between their model of reality and their predicted tokens? Any internal model an LLM has is an emergent property of their underlying training.

(And, given the way instruct/chat models are finetuned, I would say convincing/persuasive is very much the direction they are biased)

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#162

Post by ta8645 »

dijksterhuis wrote: post #151
ta8645 wrote: post #145

Earlier quoted context omitted.

You're really missing the point and getting lost in definitions. The entire point of human language is to model reality. Just because it is limited, inexact, and imperfect does not disqualify it as a model of reality. Since LLMs are directly based on that language, they are definitely based on and are a model of reality. Are they perfect? No. Are they limited? Yes. Are they "bullshit"? Only to someone who is judging…

and herein lies the rub. > The entire point of human language is to model reality. is it? are you absolutely certain of that fact? is language not something that actually has a variety of purposes? fiction novels usually do not describe our reality, but imagined realities . they use language to convey ideas and concepts that do not necessarily exist in the real world. ref: Philip k dick. > Since LLMs are directly bas…

> imagined realities.

Imagined realities are a real part of reality.

> so LLMs are an approximation of an approximate model of reality?

Yes, and we as humans have a mental model that is just an approximation of reality. And we read books that are just an approximation of another human's approximation of reality. Does that mean that we are bullshit because we rely on approximations of approximations?

You're being way too pedantic and dismissive. Models are models, regardless of how limited and imperfect they are.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#163

Post by trimethylpurine »

JKCalhoun wrote: post #128
einrealist wrote: post #49

This website is so important! Now ask yourself why AI companies don't want to be regulated or scrutinized. So many companies (users and providers) jump on the AI hype train because of FOMO. The end result might be just as destructive as this mythical "AGI". Edit: I am not saying to not use the technology. I am just on the side of caution and constant validation. The technology has to serve society. But I fear this hy…

I generally take issue when "FOMO" is used. Could go with: FOBBWIIBM - Fear of being blindsided when it, inevitably, becomes mainstream. Or drop the "fear" altogether: JOENT - Joy of exploring new territory.

>being blindsided when it, inevitably, becomes mainstream.

I don't see how this could happen. This is not a limited resource. It's not a real estate opportunity. There is enough AI for everyone to buy when it becomes useful to do so.

I think FOMO correctly identifies the irrational effort of many companies to jump in without any idea of what the utility might be in any practical sense.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#164

Post by nonrandomstring »

uh_uh wrote: post #139
nonrandomstring wrote: post #76

Earlier quoted context omitted.

> cynical () positions on some controversial questions. I feel "cynical" is an inappropriate word here. We may have to, for the same (ecumenical) reasons that thinkers like Churchland, Hofstadter, Dennet, Penrose and company have all struggled with, eventually accept the impossibility of proof of (existence or non-existence) on any hypothesis of "machine mind". The pragmatic response is, "does it offer utility for me…

The choice unfortunately seems to correlate with the person's age. Younger generations will have no trouble treating LLMs as actually intelligent. Yet another example of "Science progresses one funeral at a time.”

> correlates with age

Definitely a "citation needed" moment I think. Friday, I was with a lot of 12 year olds all firmly of the opinion that it's a "way to get intelligence/information" but it's not actually intelligent. (FWIW in UK they say "for real life intelligent") I noted this distinction. Or rather, I noted because that's what they're taught. So teachers, naturally pass on the commonsense position that "it's still just a computer". That means waiting for funerals will not settle the matter either. That's not to say a significant sect of more credulous "AI worshippers" will not emerge.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#165

Post by _heimdall »

JKCalhoun wrote: post #132
_heimdall wrote: post #116

Earlier quoted context omitted.

Any new technology given this much attention and money will disrupt, that's not really a question. The question is whether we're going to be better off for it, and if people want all that change in the first place.

It could be argued that the adoption of the automobile was a bad move. You and I debating either its efficacy or social good though is irrelevant if it marches on regardless.

I didn't explain my point there, let me try with the automobile example.

There's a key difference in the how the impact of the automobile happened - consumers got to choose to buy them and the impact was driven in large part by market demand.

The fact that LLMs are going to have a big impact seems obvious because a comparatively few number of people are making a huge deal out of them, both with attention and money. LLMs will be big but that says nothing of their usefulness or even consumer demand, it says more about how the industry is being financed.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#166

Post by einrealist »

JKCalhoun wrote: post #128
einrealist wrote: post #49

This website is so important! Now ask yourself why AI companies don't want to be regulated or scrutinized. So many companies (users and providers) jump on the AI hype train because of FOMO. The end result might be just as destructive as this mythical "AGI". Edit: I am not saying to not use the technology. I am just on the side of caution and constant validation. The technology has to serve society. But I fear this hy…

I generally take issue when "FOMO" is used. Could go with: FOBBWIIBM - Fear of being blindsided when it, inevitably, becomes mainstream. Or drop the "fear" altogether: JOENT - Joy of exploring new territory.

You are right. But these are different different types of motivations of the same thing. And there is always context for these motivations.

Its a different thing to sell Trump that LLMs should take over crucial decisions within a government than just using it for some prototyping, code completion at work or to create cat pictures at home.

Take Copilot for example. It was rolled out in different companies, I worked with. Aside of warnings and maybe trainings, I doubt the companies are really able to measure the impact that has. Students are already using the technology to do homework. Schools and universities are sending mixed signals about the results. And then those students enter the workforce with Copilot enabled by default.

At least with companies, its the "free market" that will regulate (unless some company is too big to fail...)

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#167

Post by rcxdude »

TheOtherHobbes wrote: post #106
tmnvdb wrote: post #71

Hmm, it seems that the author takes very clear (and sometimes cynical) positions on some controversial questions. For example, "They don't have the capacity to think through problems logically." is an hotly debated claim, and I think with the advent of reasoning models this has at least become something one should not state in entry level material, which would hopefully reflect common understanding rather than the au…

Many claims don't stand up to scrutiny, and some look suspiciously like training to the test. The Apple study was clear about this. LLMs and their related modal models lack the ability to abstract information from noisy text inputs. This is really obvious if you play with any of the art generators. For example - the understanding of basic prepositions just isn't there. You can't say "Put this thing behind/over/in fro…

Part of this is that the art generators tend to use CLIP, whjch is not a particularly good text model, often only being slightly better than a bag of words, which makes many interactions and relationships pretty difficult to represent. Some of the newer ones have better frontends which improve this situation, though.

I think color is fairly well abstracted, but most image generators are not good for edits, because the generator more or less starts from scratch, and from a new random seed each time (and even if the seed is fixed, the initial stages of the generation, where things like the rough image composition form, tend to be quite chaotic and so sensitive to small changes in prompt). There are tools that can make far more controlled adjustments of an image, but they tend to be a bit less user-friendly.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#168

Post by leoc »

tialaramex wrote: post #59

This course mentions the famous Apple advertisement. Unfortunately it slightly oversells and while I'm sure that's not because this fragment was written by an LLM it is exactly the sort of over-simplification which leads to LLMs generating wild bullshit when they interpolate this "fact" with other "facts" they've been fed, and we ought to strive to do better when writing for humans. "Describe how prior to 1984, there…

Jimmy Carter installed a Xerox Alto in the White House in 1978! https://www.ourmidland.com/news/article/Check-out-the-first-... Never mind the Xerox Star, or Apple shipping the Lisa in 1983 ...

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#169

Post by jcgrillo »

tmnvdb wrote: post #153
jcgrillo wrote: post #146

Earlier quoted context omitted.

As far as I can tell, those charts are merely describing the price per token that LLM hosting companies are charging, not what running the model actually costs. The distinction is important for two reasons: 1. These companies are heavily subsidized by huge amounts of venture investment 2. If I'm integrating this technology into my web product there's absolutely no way I'll be adding a 3rd party company as a dependenc…

This is incorrect, there are real effiency gains. Slightly old by the standards of this field, but a good overview: https://arxiv.org/abs/2403.05812

I'm sorry, what is incorrect? And the paper you linked appears to be about training, not inference? I wouldn't train a model in a user's session, instead I'd run the model. That's the cost that seems a blocker, all the models are already trained, I dont need to invest a dime in that.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#170

Post by 1970-01-01 »

TeMPOraL wrote: post #95

"The LLMs have no ground truth" claim (around chapter 2) that's core to the "bullshit machines" argument is itself wrong. Of course LLMs have ground truth. What do the authors think here, that the text in training corpus is random ? Hint: it isn't. Real conversations are anything but random. There's a lot of information hidden in "statistical ordering of the words", because the distribution is not arbitrary . Statist…

>Of course LLMs have ground truth.

It is my understanding that LLMs have no such thing, as empiric truth is weighted. For example, if Newton's laws are in conflict with another fact, the LLM will defer to the fact that it finds more probable in context. It will then require human resources to undo and unfold it's core error, else you receive bewildering and untrue remarks or outputs.

Post reply on HN