Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I genuinely don't understand why some people are still bullish about LLMs

twitter.com

Post reply on HN
Print view View original

551–560 of 1001 posts

Re: I genuinely don't understand why some people are still bullish about LLMs

#551

Post by mdp2021 »

worthless-trash wrote: post #366
cogman10 wrote: post #161

Earlier quoted context omitted.

> Wah, it can't write code like a Senior engineer with 20 years of experience! No, that's not my problem with it. My problem with it is that inbuilt into the models of all LLMs is that they'll fabricate a lot. What's worse, people are treating them as authoritative. Sure, sometimes it produces useful code. And often, it'll simply call the "doTheHardPart()" method. I've even caught it literally writing the wrong algor…

I saw someone saying 80% of doctors believe that LLM's are trustworthy consultation partners. Code created by LLM's doesnt compile, hallucinated API's.. invalid syntax and completely broken logic, why would you trust it with someones life !

Apologies, but have you noticed that if your entrusted (the "doctor") trusted the unentrustable (the "LLM"), then your entrusted is not trustworthy?

Re: I genuinely don't understand why some people are still bullish about LLMs

#552

Post by chgs »

Animats wrote: post #178
cogman10 wrote: post #161

Earlier quoted context omitted.

> Wah, it can't write code like a Senior engineer with 20 years of experience! No, that's not my problem with it. My problem with it is that inbuilt into the models of all LLMs is that they'll fabricate a lot. What's worse, people are treating them as authoritative. Sure, sometimes it produces useful code. And often, it'll simply call the "doTheHardPart()" method. I've even caught it literally writing the wrong algor…

> inbuilt into the models of all LLMs is that they'll fabricate a lot. Still the elephant in the room. We need an AI technology that can output "don't know" when appropriate. How's that coming along?

> It's impossible to predict with certainty who will be the U.S. President in 2046. The political landscape can change significantly over time, and many factors, including elections, candidates, and events, will influence the outcome. The next U.S. presidential election will take place in 2028, so it would be difficult to know for sure who will hold office nearly two decades from now.

So it can say “I don’t know”

Re: I genuinely don't understand why some people are still bullish about LLMs

#554

Post by somenameforme »

gilbetron wrote: post #109

I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…

It's mostly because of how they were initially marketed. In an effort to drive hype 'we' were promised the world. Remember the "leaks" from Google about an engineer trying to get the word out that they had created a sentient intelligence? In reality Bard, let alone whatever early version he was using, is about as sentient as my left asscheek.

OpenAI did similar things by focusing to the point of absurdity on 'safety' for what was basically a natural language search engine that has a habit of inventing nonsensical stuff. But on that same note (and also as you alluded to) - I do agree that LLMs have a lot of use as natural language search engines in spite of their proclivity to hallucinate. Being able to describe a e.g. function call (or some esoteric piece of history) by description and then often get the precise term/event that I'm looking for is just incredibly useful.

But LLMs obviously are not sentient, are not setting us on the path to AGI, or any other such nonsense. They're arguably what search engines should have been 10 or 15 years ago, but anti-competitive monopolization of the industry meant that search engine technology progress basically stalled out, if not regressed for the sake of ads (and individual 'entrepreneurs' becoming better at SEO), about the time Google fully established itself.

Re: I genuinely don't understand why some people are still bullish about LLMs

#555

Post by hellzbellz123 »

so many people have shown me these stupid ass AI summaries for random things and if you have even a basic understanding of the relevent issue then the answers just seem bizare. this feels like cheating on my homework and not understanding. like using photomath on homework does.

Re: I genuinely don't understand why some people are still bullish about LLMs

#556

Post by genevra »

gilbetron wrote: post #109

I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…

Absolutely, as someone with far less experience than you I feel quite spoiled I'll have this for help coding for the rest of time.

Re: I genuinely don't understand why some people are still bullish about LLMs

#557

Post by noufalibrahim »

gloosx wrote: post #545
gilbetron wrote: post #109

I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…

The problem Sabine tries to communicate is that reality is different from what the cash-heads behind main commercial models are trying to portray. They push the narrative that they’ve created something akin to human cognition, when in reality, they’ve just optimised prediction algorithms on an unprecedented scale. They are trying to say that they created Intelligence, which is the ability to acquire and apply knowled…

Much as i agree with the point about overhyping from companies, I'd be more sympathetic to this point of view if she acknowledged the merits of the technology.

Yes, it hallucinates and if you replace your brain with one of these things, you won't last too long. However, it can do things which, in the hands of someone experienced, are very empowering. And it doesn't take an expert to see the potential.

As it stands, it sounds like a case of "it's great in practice but the important question is how good it is in theory."

Re: I genuinely don't understand why some people are still bullish about LLMs

#558

Post by emsign »

gilbetron wrote: post #109

I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…

LLMs can't be trusted. They aé like an overconfident idiot who is pretending quite impressively, butif you check on the result it's just a bit too much bullshit in the result. So there's practically zero gain in using LLMs except WHEN you actually need a text that's nice and eloquent bullshit.

Almost everytime I've tried using LLMs I've fallen into thepattern on calling out, correcting and argueing with the LLMs which is of course in itself sillyto do, because they don't learn, they don't really "get it" when they are wrong. There's no benefit to talking to a human.

Re: I genuinely don't understand why some people are still bullish about LLMs

#559

Post by Terr_ »

crazygringo wrote: post #42

If there's one common thread across LLM criticisms, it's that they're not perfect. These critics don't seem to have learned the lesson that the perfect is the enemy of the good . I use ChatGPT all the time for academic research. Does it fabricate references? Absolutely, maybe about a third of the time. But has it pointed me to important research papers I might never have found otherwise? Absolutely . The rate of inac…

> Verifying the accuracy of its statements is easy.

For single statements, sometimes, but not always. For all of the many statements, no. Having the human attention and discipline to mindfully verify every single one without fail? Impossible.

Every software product/process that assumes the user has superhuman vigilance is doomed to fail badly.

> Automation centaurs are great: they relieve humans of drudgework and let them focus on the creative and satisfying parts of their jobs. That's how AI-assisted coding is pitched [...]

> But a hallucinating AI is a terrible co-pilot. It's just good enough to get the job done much of the time, but it also sneakily inserts booby-traps that are statistically guaranteed to look as plausible as the good code (that's what a next-word-guessing program does: guesses the statistically most likely word).

> This turns AI-"assisted" coders into reverse centaurs. The AI can churn out code at superhuman speed, and you, the human in the loop, must maintain perfect vigilance and attention as you review that code, spotting the cleverly disguised hooks for malicious code that the AI can't be prevented from inserting into its code. As qntm writes, "code review [is] difficult relative to writing new code":

-- https://pluralistic.net/2025/03/18/asbestos-in-the-walls/

Re: I genuinely don't understand why some people are still bullish about LLMs

#560

Post by harvey9 »

kristiandupont wrote: post #477
worthless-trash wrote: post #366

Earlier quoted context omitted.

I saw someone saying 80% of doctors believe that LLM's are trustworthy consultation partners. Code created by LLM's doesnt compile, hallucinated API's.. invalid syntax and completely broken logic, why would you trust it with someones life !

>I saw someone saying 80% of doctors believe that LLM's are trustworthy consultation partners. See, now that is something I don't know why I should trust: a random person on the internet citing a statistics that they saw someone else say.

The claim seems plausible because it doesn't say there was any formal evaluation, just that some doctors (who may or may not understand how LLMs work) hold an opinion.
Post reply on HN