Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Recent AI model progress feels mostly like bullshit

lesswrong.com

Post reply on HN
Print view View original

311–320 of 478 posts

Re: Recent AI model progress feels mostly like bullshit

#311

Post by Tepix »

leonidasv wrote: post #262
AIPedant wrote: post #243

Earlier quoted context omitted.

The "ferrying animals across a river" problem has definitely not been solved, they still don't understand the problem at all, overcomplicating it because they're using an off-the-shelf solution instead of actual reasoning: o1 screwing up a trivially easy variation: https://xcancel.com/colin_fraser/status/1864787124320387202 Claude 3.7, utterly incoherent: https://xcancel.com/colin_fraser/status/1898158943962271876 De…

Gemini 2.5 Pro got the farmer problem variation right: https://aistudio.google.com/app/prompts?state=%7B%22ids%22:%...

That can't be viewed without logging into Google first.

Re: Recent AI model progress feels mostly like bullshit

#312

Post by wg0 »

Unlike many - I find author's complaints on the dot.

Once all the AI batch startups have sold subscriptions to the cohort and there's no more further market growth because businesses outside don't want to roll the dice on a probabilistic model that doesn't have an understanding of pretty much anything rather is a clever imitation machine on the content it has seen, the AI bubble will burst when more statups would start packing up by end of 2026 or max 2027.

Re: Recent AI model progress feels mostly like bullshit

#315

Post by katsura »

aezart wrote: post #294
billforsternz wrote: post #238

Earlier quoted context omitted.

I asked Google "how many golf balls can fit in a Boeing 737 cabin" last week. The "AI" answer helpfully broke the solution into 4 stages; 1) A Boeing 737 cabin is about 3000 cubic metres [wrong, about 4x2x40 ~ 300 cubic metres] 2) A golf ball is about 0.000004 cubic metres [wrong, it's about 40cc = 0.00004 cubic metres] 3) 3000 / 0.000004 = 750,000 [wrong, it's 750,000,000] 4) We have to make an adjustment because se…

> I have seen much AI output which is extraordinary it's funny how one serious fail can impact my point of view so dramatically. I feel the same way. It's like discovering for the first time that magicians aren't doing "real" magic, just sleight of hand and psychological tricks. From that point on, it's impossible to be convinced that a future trick is real magic, no matter how impressive it seems. You know it's fake…

To be fair, I love that magicians can pull tricks on me even though I know it is fake.

Re: Recent AI model progress feels mostly like bullshit

#316

Post by dragandj »

maccard wrote: post #37
grey-area wrote: post #35

Earlier quoted context omitted.

That’s not a business model, it’s a pipe dream. This bubble will be burst by the Trump tariffs and the end of the zirp era. When inflation and a recession hit together hope and dream business models and valuations no longer work.

Which one? Nvidia are doing pretty ok selling GPU's, and OpenAI and Anthropic are doing ok selling their models. They're not _viable_ business models, but they could be.

They are doing OK in the sense that they are spending a dollar to earn a dime I suppose.

Re: Recent AI model progress feels mostly like bullshit

#317

Post by InDubioProRubio »

greenmartian wrote: post #268
billforsternz wrote: post #238

Earlier quoted context omitted.

I asked Google "how many golf balls can fit in a Boeing 737 cabin" last week. The "AI" answer helpfully broke the solution into 4 stages; 1) A Boeing 737 cabin is about 3000 cubic metres [wrong, about 4x2x40 ~ 300 cubic metres] 2) A golf ball is about 0.000004 cubic metres [wrong, it's about 40cc = 0.00004 cubic metres] 3) 3000 / 0.000004 = 750,000 [wrong, it's 750,000,000] 4) We have to make an adjustment because se…

Weird thing is, in Google AI Studio all their models—from the state-of-the-art Gemini 2.5Pro, to the lightweight Gemma 2—gave a roughly correct answer. Most even recognised the packing efficiency of spheres. But Google search gave me the exact same slop you mentioned. So whatever Search is using, they must be using their crappiest, cheapest model. It's nowhere near state of the art.

Its most likely one giant ["input token close enough question hash"] = answer_with_params_replay? It doesent missunderstands the question, it tries to squeeze the input to something close enough?

Re: Recent AI model progress feels mostly like bullshit

#318

Post by signa11 »

signa11 wrote: post #286
iambateman wrote: post #39

The core point in this article is that the LLM wants to report _something_, and so it tends to exaggerate. It’s not very good at saying “no” or not as good as a programmer would hope. When you ask it a question, it tends to say yes. So while the LLM arms race is incrementally increasing benchmark scores, those improvements are illusory. The real challenge is that the LLM’s fundamentally want to seem agreeable, and th…

> The core point in this article is that the LLM wants to report _something_, and so it tends to exaggerate. It’s not very good at saying “no” or not as good as a programmer would hope. umm, it seems to me that it is this (tfa): But I would nevertheless like to submit, based off of internal benchmarks, and my own and colleagues' perceptions using these models, that whatever gains these companies are reporting to the…

[this went way outside the edit-window and hence a separate comment] imho, state of varying experience with llm's can aptly summed in this poem by Mr. Longfellow

     There was a little girl,
        Who had a little curl,
     Right in the middle of her forehead.
        When she was good,
        She was very good indeed,
     But when she was bad she was horrid.

Re: Recent AI model progress feels mostly like bullshit

#319

Post by otabdeveloper4 »

AIPedant wrote: post #118
InkCanon wrote: post #75

The biggest story in AI was released a few weeks ago but was given little attention: on the recent USAMO, SOTA models scored on average 5% (IIRC, it was some abysmal number). This is despite them supposedly having gotten 50%, 60% etc performance on IMO questions. This massively suggests AI models simply remember the past results, instead of actually solving these questions. I'm incredibly surprised no one mentions th…

Yes, here's the link: https://arxiv.org/abs/2503.21934v1 Anecdotally, I've been playing around with o3-mini on undergraduate math questions: it is much better at "plug-and-chug" proofs than GPT-4, but those problems aren't independently interesting, they are explicitly pedagogical. For anything requiring insight, it's either: 1) A very good answer that reveals the LLM has seen the problem before (e.g. naming the theo…

Anecdotally: schoolkids are at the leading edge of LLM innovation, and nowadays all homework assignments are explicitly made to be LLM-proof. (Well, at least in my son's school. Yours might be different.)

This effectively makes LLMs useless for education. (Also sours the next generation on LLMs in general, these things are extremely lame to the proverbial "kids these days".)

Re: Recent AI model progress feels mostly like bullshit

#320

Post by heresie-dabord »

joelthelion wrote: post #25

I've used gemini 2.5 this weekend with aider and it was frighteningly good. It probably depends a lot on what you are using them for, and in general, I think it's still too early to say exactly where LLMs will lead us.

> It probably depends a lot on what you are using them for, and in general, I think it's still too early to say exactly where LLMs will lead us.

Even approximations must be right to be meaningful. If information is wrong, it's rubbish.

Presorting/labelling various data has value. Humans have done the real work there.

What is "leading" us at present are the exaggerated valuations of corporations. You/we are in a bubble, working to justify the bubble.

Until a tool is reliable, it is not installed where people can get hurt. Unless we have revised our concern for people.

Post reply on HN