Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I genuinely don't understand why some people are still bullish about LLMs

twitter.com

Post reply on HN
Print view View original

371–380 of 1001 posts

Re: I genuinely don't understand why some people are still bullish about LLMs

#371

Post by mythz »

I understand how developers can come to this conclusion if they're only using local models that can run on consumer GPUs since there's a time cost to prompting and the output is fairly low quality with a higher probability of errors and hallucinations.

But I don't understand how you can come to this conclusion when using SOTA models like Claude Sonnet 3.7, it's response has always been useful and when it doesn't get it right first time you can keep prompting it with clarifications and error responses. On the rare occasion it's unable to get it right, I'm still left with a bulk of useful code that I can manually fix and refactor.

Either way my interactions with Sonnet is always beneficial. Maybe it's a prompt issue? I only ask it to perform small, specific deterministic tasks and provide the necessary context (with examples when possible) to achieve it.

I don't vibe code or unleash an LLM on an entire code base since the context is not large enough and I don't want it to refactor/break working code.

Re: I genuinely don't understand why some people are still bullish about LLMs

#372

Post by jjani »

selcuka wrote: post #157
taberiand wrote: post #108

Earlier quoted context omitted.

I have been saying this about llms for a while - if you know what you want, how to ask for it, and what the correct output will look like, LLMs are fantastic (at least Claude Sonnet is). And I mean that seriously, they are a highly effective tool for productive development for senior developers. I use it to produce whole classes, large sql queries, terraform scripts, etc etc. I then look over that output, iterate on…

> they are a highly effective tool for productive development for senior developers I think this is the most important bit many people miss. It is advertised as an autonomous software developer, or something that can take a junior to senior levels, but that's just advertising. It is actually most useful for senior developers, as it does the grunt work for them, while grunt work is actually useful work for a junior de…

> It is advertised as an autonomous software developer

By a few currently niche VC players, I guess. I don't see Anthropic, the overwhelming revenue leader in dollars spent on LLM-related tools for SWE, claiming that.

Re: I genuinely don't understand why some people are still bullish about LLMs

#373

Post by Aeolun »

gilbetron wrote: post #109

I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…

> Wah, it can't write code like a Senior engineer with 20 years of experience!

Thank goodness for that too. I want it to help me with my job, not replace me.

Re: I genuinely don't understand why some people are still bullish about LLMs

#374

Post by Aeolun »

worthless-trash wrote: post #366
cogman10 wrote: post #161

Earlier quoted context omitted.

> Wah, it can't write code like a Senior engineer with 20 years of experience! No, that's not my problem with it. My problem with it is that inbuilt into the models of all LLMs is that they'll fabricate a lot. What's worse, people are treating them as authoritative. Sure, sometimes it produces useful code. And often, it'll simply call the "doTheHardPart()" method. I've even caught it literally writing the wrong algor…

I saw someone saying 80% of doctors believe that LLM's are trustworthy consultation partners. Code created by LLM's doesnt compile, hallucinated API's.. invalid syntax and completely broken logic, why would you trust it with someones life !

Because LLM’s, with like 20% hallucination rate, are more reliable than overworked, tired doctors that can spend only one ounce of their brainpower on the patient they’re currently helping?

Re: I genuinely don't understand why some people are still bullish about LLMs

#375

Post by GolDDranks »

GolDDranks wrote: post #354
gilbetron wrote: post #109

I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…

I am so confused too. I hold these beliefs at the same time, and I don't feel they don't contradict each other, but apparently for many people some of these do: - LLMs are a miraculous technology that are capable of tasks far beyond what we believed would be achievable with AI/ML in the near future. Playing with them makes me constantly feel like "this is like sci-fi, this shouldn't be possible with 2025's technology…

I guess that Sabine's beef with LLM's that they are hyped as a legit "human level assistant" -kind of thing by the business people, which they clearly aren't yet. Maybe I've just managed to... manage my expectations?

Re: I genuinely don't understand why some people are still bullish about LLMs

#377

Post by jasonjmcghee »

jandrewrogers wrote: post #297
gilbetron wrote: post #109

I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…

TBH, they produce trash results for almost any question I might want to ask them. This is consistently the case. I must use them differently than other people. LLMs produce midwit answers. If you are an expert in your domain, the results are kind of what you would expect for someone who isn’t an expert. That is occasionally useful but if I wanted a mediocre solution in software I’d use the average library. No LLM I h…

> No LLM I have ever used has delivered an expert answer...and that's where all the value is.

If this were true, no one would hire junior employees and assistants. There's a huge amount of work that requires more time than expertise.

Re: I genuinely don't understand why some people are still bullish about LLMs

#378

Post by nyarlathotep_ »

nonethewiser wrote: post #355
cogman10 wrote: post #161

Earlier quoted context omitted.

> Wah, it can't write code like a Senior engineer with 20 years of experience! No, that's not my problem with it. My problem with it is that inbuilt into the models of all LLMs is that they'll fabricate a lot. What's worse, people are treating them as authoritative. Sure, sometimes it produces useful code. And often, it'll simply call the "doTheHardPart()" method. I've even caught it literally writing the wrong algor…

> What's worse, people are treating them as authoritative. … I've both seen online and heard people quote LLM output as if it were authoritative. Thats not an LLM problem. But indeed quite bothersome. Dont tell me what Chatgpt told you. Tell me what you know. Maybe you got it from ChatGPT and verified it. Great. But my jaw kind of drops when people cite an LLM and just assume it’s correct.

> But my jaw kind of drops when people cite an LLM and just assume it’s correct.

Yes but they're literally told by allegedly authoritative sources that it's going to change everything and eliminate intellectual labor, so is it totally their fault?

They've heard about the uncountable sums of money spent on creating such software, why would they assume it was anything short of advertised?

Re: I genuinely don't understand why some people are still bullish about LLMs

#379

Post by Aeolun »

cogman10 wrote: post #203
MattGaiser wrote: post #169

Earlier quoted context omitted.

Humans bullshit and hallucinate and claim authority without citation or knowledge. They will believe all manner of things. They frequently misunderstand. The LLM doesn’t need to be perfect. Just needs to beat a typical human. LLM opponents aren’t wrong about the limits of LLMs. They vastly overestimate humans.

> LLM opponents aren’t wrong about the limits of LLMs. They vastly overestimate humans. On the contrary. Humans can earn trust, learn, and can admit to being wrong or not knowing something. Further, humans are capable of independent research to figure out what it is they don't know. My problem isn't that humans are doing similar things to LLMs, my problem is that humans can understand consequences of bullshitting at…

> On the contrary. Humans can earn trust, learn, and can admit to being wrong or not knowing something. Further, humans are capable of independent research to figure out what it is they don't know.

Some humans can, certainly. Humans as a race? Maybe, ish.

Re: I genuinely don't understand why some people are still bullish about LLMs

#380

Post by financypants »

anon291 wrote: post #327
Animats wrote: post #178

Earlier quoted context omitted.

> inbuilt into the models of all LLMs is that they'll fabricate a lot. Still the elephant in the room. We need an AI technology that can output "don't know" when appropriate. How's that coming along?

How many companies train on data that contains 'i don't know' responses. Have you ever talked with a toddler / young child? You need to explicitly teach children to not bull shit. At least I needed to teach mine.

I never thought about this but I have experienced this with my children.
Post reply on HN