Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Recent AI model progress feels mostly like bullshit

lesswrong.com

Post reply on HN
Print view View original

101–110 of 478 posts

Re: Recent AI model progress feels mostly like bullshit

#101

Post by stavros »

more_corn wrote: post #87
stavros wrote: post #64

Earlier quoted context omitted.

LLMs aren't good at being search engines, they're good at understanding things. Put an LLM on top of a search engine, and that's the appropriate tool for this use case. I guess the problem with LLMs is that they're too usable for their own good, so people don't realizing that they can't perfectly know all the trivia in the world, exactly the same as any human.

Ironically though an LLM powered search engine (some word about being perplexed) is becoming way better than the undisputed king of traditional search engines (something oogle)

That's because they put an LLM over a traditional search engine.

Re: Recent AI model progress feels mostly like bullshit

#102

Post by lukev »

mmcnl wrote: post #83
lukev wrote: post #45

Earlier quoted context omitted.

This seems like a probable end state, but we're going to have to stop calling LLMs "artificial intelligence" in order to get there.

Why not? Objectively speaking LLMs are artificial intelligent. Just because it's not human level intelligence doesn't mean it's not intelligent.

Objectively speaking a chess engine is artificially intelligent. Just because it's not human level doesn't mean it's not intelligent. Repeat for any N of 100s of different technologies we've built. We've been calling this stuff "thinking machines" since Turing and it's honestly just not useful at this point.

The fact is, the phrase "artificial intelligence" is a memetic hazard: it immediately positions the subject of conversation as "default capable", and then forces the conversation into trying to describe what it can't do, which is rarely a useful way to approach it.

Whereas with LLMs (and chess engines and every other tech advancement) it would be more useful to start with what the tech _can_ do and go from there.

Re: Recent AI model progress feels mostly like bullshit

#103

Post by fnordpiglet »

mvdtnz wrote: post #85
fnordpiglet wrote: post #63

Earlier quoted context omitted.

This is less an LLM thing than an information retrieval question. If you choose a model and tell it to “Search,” you find citation based analysis that discusses that he indeed had problems with alcohol. I do find it interesting it quibbles whether he was an alcoholic or not - it seems pretty clear from the rest that he was - but regardless. This is indicative of something crucial when placing LLMs into a toolkit. The…

Any information found in a web search about Newman will be available in the training set (more or less). It's almost certainly a problem of alignment / "safety" causing this issue.

There’s a simpler explanation than that’s that the model weights aren’t an information retrieval system and other sequences of tokens are more likely given the totality of training data. This is why for an information retrieval task you use an information retrieval tool similarly to how for driving nails you use a hammer rather than a screw driver. It may very well be you could drive the nail with the screw driver, but why?

Re: Recent AI model progress feels mostly like bullshit

#104

Post by Avicebron »

john2x wrote: post #48
drooby wrote: post #38

Earlier quoted context omitted.

I asked GPT-4.5 and it searched the web and immediately gave me a "yes" with paragraphs of sources cited.

Truth is a probability game. Just keep trying until you arrive.

"man puts prompt into LLM" > "LLM ships bad code" > "machine fails" > "person dies" > "fire man" > "man puts prompt into LLM"

Re: Recent AI model progress feels mostly like bullshit

#105

Post by InkCanon »

HarHarVeryFunny wrote: post #43

The disconnect between improved benchmark results and lack of improvement on real world tasks doesn't have to imply cheating - it's just a reflection of the nature of LLMs, which at the end of the day are just prediction systems - these are language models, not cognitive architectures built for generality. Of course, if you train an LLM heavily on narrow benchmark domains then its prediction performance will improve…

That's fair. But look up the recent experiment on SOTA models on the then just released USAMO 2025 questions. Highest score was 5%, supposedly SOTA last year was IMO silver level. There could be some methodological differences - ie USAMO paper required correct proofs and not just numerical answers. But it really strongly suggests even within limited domains, it's cheating. I'd wager a significant amount that if you tested SOTA models on a new ICPC set of questions, actual performance would be far, far worse than their supposed benchmarks.

Re: Recent AI model progress feels mostly like bullshit

#106

Post by mitthrowaway2 »

MegaButts wrote: post #68
stavros wrote: post #64

Earlier quoted context omitted.

LLMs aren't good at being search engines, they're good at understanding things. Put an LLM on top of a search engine, and that's the appropriate tool for this use case. I guess the problem with LLMs is that they're too usable for their own good, so people don't realizing that they can't perfectly know all the trivia in the world, exactly the same as any human.

> LLMs aren't good at being search engines, they're good at understanding things. LLMs are literally fundamentally incapable of understanding things. They are stochastic parrots and you've been fooled.

What does the word "understand" mean to you?

Re: Recent AI model progress feels mostly like bullshit

#107

Post by ohgr »

maccard wrote: post #28
ohgr wrote: post #8

It’s not even approaching the asymptotic line of promises made at any achievable rate for the amount of cash being thrown at it. Where’s the business model? Suck investors dry at the start of a financial collapse? Yeah that’s going to end well…

> where’s the business model? For who? Nvidia sell GPUs, OpenAI and co sell proprietary models and API access, and the startups resell GPT and Claude with custom prompts. Each one is hoping that the layer above has a breakthrough that makes their current spend viable. If they do, then you don’t want to be left behind, because _everything_ changes. It probably won’t, but it might. That’s the business model

You missed the end of the supply chain. Paying users. Who magically disappear below market sustaining levels of sales when asked to pay.

Re: Recent AI model progress feels mostly like bullshit

#108

Post by the8472 »

MegaButts wrote: post #68
stavros wrote: post #64

Earlier quoted context omitted.

LLMs aren't good at being search engines, they're good at understanding things. Put an LLM on top of a search engine, and that's the appropriate tool for this use case. I guess the problem with LLMs is that they're too usable for their own good, so people don't realizing that they can't perfectly know all the trivia in the world, exactly the same as any human.

> LLMs aren't good at being search engines, they're good at understanding things. LLMs are literally fundamentally incapable of understanding things. They are stochastic parrots and you've been fooled.

A stochastic parrot with a sufficiently tiny residual error rate needs a stochastic model so precisely compressing the world and sophisticated decompression algorithms that it could be called reasoning.

Take two 4K frames of a falling vase, ask a model to predict the next token... I mean the following images. Your model now needs include some approximations of physics - and the ability to apply it correctly - to produce a realistic outcome. I'm not aware of any model capable of doing that, but that's what it would mean to predict the unseen with high enough fidelity.

Re: Recent AI model progress feels mostly like bullshit

#109

Post by [deleted] »

lukev wrote: post #40

This is a bit of a meta-comment, but reading through the responses to a post like this is really interesting because it demonstrates how our collective response to this stuff is (a) wildly divergent and (b) entirely anecdote-driven. I have my own opinions, but I can't really say that they're not also based on anecdotes and personal decision-making heuristics. But some of us are going to end up right and some of us ar…

[deleted]

Re: Recent AI model progress feels mostly like bullshit

#110

Post by MegaButts »

mitthrowaway2 wrote: post #106
MegaButts wrote: post #68

Earlier quoted context omitted.

> LLMs aren't good at being search engines, they're good at understanding things. LLMs are literally fundamentally incapable of understanding things. They are stochastic parrots and you've been fooled.

What does the word "understand" mean to you?

An ability to answer questions with a train of thought showing how the answer was derived, or the self-awareness to recognize you do not have the ability to answer the question and declare as much. More than half the time I've used LLMs they will simply make answers up, and when I point out the answer is wrong it simply regurgitates another incorrect answer ad nauseum (regularly cycling through answers I've already pointed out are incorrect).

Rather than give you a technical answer - if I ever feel like an LLM can recognize its limitations rather than make something up, I would say it understands. In my experience LLMs are just algorithmic bullshitters. I would consider a function that just returns "I do not understand" to be an improvement, since most of the time I get confidently incorrect answers instead.

Yes, I read Anthropic's paper from a few days ago. I remain unimpressed until talking to an LLM isn't a profoundly frustrating experience.

Post reply on HN