Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

thebullshitmachines.com

Post reply on HN
Print view View original

231–240 of 652 posts

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#231

Post by tmnvdb »

nonrandomstring wrote: post #164
uh_uh wrote: post #139

Earlier quoted context omitted.

The choice unfortunately seems to correlate with the person's age. Younger generations will have no trouble treating LLMs as actually intelligent. Yet another example of "Science progresses one funeral at a time.”

> correlates with age Definitely a "citation needed" moment I think. Friday, I was with a lot of 12 year olds all firmly of the opinion that it's a "way to get intelligence/information" but it's not actually intelligent. (FWIW in UK they say "for real life intelligent") I noted this distinction. Or rather, I noted because that's what they're taught . So teachers, naturally pass on the commonsense position that "it's…

There was a paper a while back on AI usage at work among engineers and it was very strongly correlated to age. This is not surprising, technology adoption is always very dependent on age. (None of this tells you if the technology is a net good)

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#232

Post by krupan »

ta8645 wrote: post #154
krupan wrote: post #152

Earlier quoted context omitted.

If not bullshit then what would you call it?

As the technology exists today: imperfect, often prone to mistakes, and unable to relay confidence levels. These problems may be addressed in future implementations. That's the same message, without any emotional baggage, or overly dismissive tone.

That would be great if those who are selling the technology described it that way. I, and apparently others, feel like maybe "bullshit" is a better counter to the current marketing for LLMs

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#234

Post by cratermoon »

TeMPOraL wrote: post #216
cratermoon wrote: post #206

Earlier quoted context omitted.

I didn't want to hijack the thread too much but yeah, organic farming was not originally about better for the consumer. The origin of organic farming methods was about taking care of the land and the ecosystems in which farming happens. The end products happened to be healthier, in some cases, because of reduced usage of chemicals known to be harmful resulted in less of them in the product at retail.

Perhaps, but then the meaning of the word is in how it's used (something that, if more people truly understood, would cut the amount of dismissing LLMs as "bullshit machines" and "stochastic parrots" by half). "Organic farming" may have initially been about sustainability, but the result correlated well enough with healthy food - and even more so with the naturalistic fallacy-fueled "healthy food" fad, that the latte…

Marketing and greed do ruin everything they touch, yes.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#235

Post by _heimdall »

JKCalhoun wrote: post #189
_heimdall wrote: post #165

Earlier quoted context omitted.

I didn't explain my point there, let me try with the automobile example. There's a key difference in the how the impact of the automobile happened - consumers got to choose to buy them and the impact was driven in large part by market demand. The fact that LLMs are going to have a big impact seems obvious because a comparatively few number of people are making a huge deal out of them, both with attention and money. L…

I'm not too worried about Big Corporation trying to push a rope. In the end it really is only going to succeed if "we" want it — find value in it.

> only going to succeed if "we" want it — find value in it.

I'll be pleasantly surprised if that's how it turns out.

At least so far market dynamics haven't really been much of a driver for LLMs. Those with the money think its the next big thing and are pouring cash both into the LLMs themselves and any product that slaps a "powered by AI" sticker on the box.

That's not to say people aren't also actively choosing to use LLMs, but in my opinion the market demand doesn't account for the massive amount of hype and funding, or the pervasiveness of LLMs being added to so many products.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#236

Post by tmnvdb »

jcgrillo wrote: post #199
tmnvdb wrote: post #194

Earlier quoted context omitted.

https://a16z.com/llmflation-llm-inference-cost/

This is getting awfully tedious. Are you trolling or do you genuinely think responding to a question about actual compute cost trends with a venture fund's puff piece about sticker price trends is helpful?

You are correct to note 'real' costs of the leading labs are not public. It is surely true that the labs are operating at below cost (we are definitely not paying for the full R&D), but it seems unlikely that this fully explains the reduction in inference costs over the last years. We also know from open models like deepseek that the cost per inference token at a fixed performance level is going down very quickly matching the curve of leading labs inference cost decreases. You can even test it yourself on your own pc if you want.

I would add that inference cost decreases is what we should expect, it stands to reason there will be algorithmic improvements in inference, and compute cost is still going down just because of (a somewhat sloped) Moore's law.

Maybe you could also be a bit friendlier and forthcoming in your responses.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#237

Post by TeMPOraL »

ssivark wrote: post #9

Kudos; feels very timely! I feel that one underappreciated nuance is why we cannot use human examinations to judge AI. I haven't seen this satisfactorily spelt out anywhere, so I recently wrote a Twitter thread [1], including an example with running -vs- biking. It might be worth making sure your students understand this. Happy to expand on any aspects if you seek. [1] : https://x.com/ergodicthought/status/1887774722…

Perhaps it's no longer being spelled out because it's getting outdated?

In your thread you argue we can't assume AI models generalize the same way we do (which is technically true except maybe not in the limit), but you seem to be worried about the extent of generalization ability (like learning to run vs. bike example, in terms of generalizing from either to climbing stairs).

Thing is, people made these objections a lot until the last year or two - this is what we're now calling a narrow AI problem. A "hot dog or not?" classifier ins't going to generalize into open-ended visual classifier of arbitrary images; a sentiment analysis bot isn't going to generalize into an universal translator; a code completion model isn't going to be giving good personal advice while speaking in pirate poetry. Specialized models fundamentally couldn't do that. But we went past that very rapidly, and for the past half a year or so, we've already seen models excelling at every single task listed above simultaneously. Same architecture, same basic training approach, few extra modalities, ever growing capabilities.

Between that and both successes and failures being eerily similar to how humans succeed or fail at these tasks, it's understandable that people are perhaps no longer convinced this class of models can't generalize in a similar way to how humans do.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#238

Post by foobiekr »

joshdavham wrote: post #16

Do you feel that you may be being a bit provocative by calling LLM's 'bullshit machines'? I understand the frustration as I've been bullshitted by these models just as much as the next programmer, but surely with recent advancements in RAG and reasoning, they're not just 'bullshit machines' at this point, are they?

they are just bullshit machines. bullshitters can cite wikipedia and are still bullshitters who are bullshitting.

Re: Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

#239

Post by sesm »

TeMPOraL wrote: post #186
sesm wrote: post #104

Earlier quoted context omitted.

> Statistical ground truth isn't any worse than explicitly given one There are multiple kinds of truths. 'Statistical truth' is at best 'consensus truth', and that's only when LLM doesn't hallucinate.

That's the only one that's available, though. When a kid at school is being taught, say, Newton's laws of motion, or what happened in 476 CE, they're not experiencing the empirical truth about either. They're only learning the consensus truth, i.e. the correct answer to give to the teacher, so they get good grade instead of bad grade, and so their parents praise them instead of punishing them, etc. This covers pretty…

I've heard about at least 4 theories of truth: Correspondence, Coherence, Consensus and Pragmatic (as described, for example, here https://commoncog.com/four-theories-of-truth/).

If we look at Newtonian mechanics, then various independently verifiable experiments are examples of Correspondence truth, and the minimal mathematical framework that describes them is an example of Coherence truth.

Post reply on HN