Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I genuinely don't understand why some people are still bullish about LLMs

twitter.com

Post reply on HN
Print view View original

521–530 of 1001 posts

Re: I genuinely don't understand why some people are still bullish about LLMs

#521

Post by vidarh »

mdp2021 wrote: post #511
vidarh wrote: post #502

Earlier quoted context omitted.

People fabricated a lot too. Yesterday I spent far less time fixing issues in the far more complex and larger changes Claude Code managed to churn out than what the junior developer I worked with needed. Sometimes it's the reverse. But with my time factored in, working with Claude Code is generally more productive for me than working with a junior. The only reason I still work with a junior dev is as an investment in…

You are mixing a point and the issue, largely heterogeneous: Claude being a champion in producing good code vs LLMs in general being delirious. If your junior developer is just "junior", that is one matter; if your junior developer hallucinates documentation details, that's different.

Every developer I've ever worked with have gotten things wrong. Whether you call that hallucinating or not is irrelevant. What matters is the effort it takes to fix.

Re: I genuinely don't understand why some people are still bullish about LLMs

#523

Post by torginus »

Should we listen to Sabine in this case? Isn't this another manifestation of a generally intelligent person, who happens to be an expert in her field weighing in on something she's not an expert on, thinking her expertise transfers?

This is the most common 'smart person' fallacy out there.

As for my 2 cents, LLMs can do sequence modeling and prediction tasks, so as long as a problem can be reduced to sequence modeling (which is a lot of them!), they can do the job.

This is like saying that the Fourier Transform is played out because you can only do so much with manipulating signal frequencies.

Re: I genuinely don't understand why some people are still bullish about LLMs

#524

Post by bjornsing »

sc68cal wrote: post #208
crazygringo wrote: post #42

If there's one common thread across LLM criticisms, it's that they're not perfect. These critics don't seem to have learned the lesson that the perfect is the enemy of the good . I use ChatGPT all the time for academic research. Does it fabricate references? Absolutely, maybe about a third of the time. But has it pointed me to important research papers I might never have found otherwise? Absolutely . The rate of inac…

> Does it fabricate references? Absolutely, maybe about a third of the time And you don't have concerns about that? What kind of damage is that doing to our society, long term, if we have a system that _everyone_ uses and it's just accepted that a third of the time it is just making shit up?

> And you don't have concerns about that? What kind of damage is that doing to our society, long term, if we have a system that _everyone_ uses and it's just accepted that a third of the time it is just making shit up?

Main problem with our society is that two thirds of what _everyone_ says is made up shit / motivated reasoning. The random errors LLMs make are relatively benign, because there is no motivation behind them. They are just noise. Look through them.

Re: I genuinely don't understand why some people are still bullish about LLMs

#525

Post by mattmanser »

bjornsing wrote: post #520

I’ve been using Claude a lot lately, and I must say I very much disagree. For example, the other day I was chatting with it about the health risks associated with my high consumption of grown salmon. It then generated a small program to simulate the accumulation of PCB in my body. I could review the program, ask questions about the assumptions, etc. It all seemed very reasonable. A toxicokinetic analysis it called it…

Google it.

There's a bunch of scientific papers talking about that.

Bet that plenty of those researchers have written python programs they've uploaded to GitHub and you just got one of their programs regurgitated

It's not intelligence mate, it's just copying an existing program.

Re: I genuinely don't understand why some people are still bullish about LLMs

#526

Post by bjornsing »

starchild3001 wrote: post #353

I have a bunch of friends who don't get along well with each other, but I tend to get along with all of them. I believe this is about focusing on the good in people, and being able to ignore the bad. I think it's the same with tools. To me AI is an out-of-this-world, OP tool. Is it perfect? No. But it's amazing! The good I get out of it far surpasses its mistakes. Almost like people. People "hallucinate" and say wron…

Agree. Had the same thought when I saw her complaint about getting the publication year of the article wrong. If she had a grad student who could read, understand and summarize the article well, but inadvertently said it was from 2023 instead of 2025, (hopefully) she wouldn’t call that grad student unintelligent.

Re: I genuinely don't understand why some people are still bullish about LLMs

#527

Post by refurb »

cogman10 wrote: post #161
gilbetron wrote: post #109

I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…

> Wah, it can't write code like a Senior engineer with 20 years of experience! No, that's not my problem with it. My problem with it is that inbuilt into the models of all LLMs is that they'll fabricate a lot. What's worse, people are treating them as authoritative. Sure, sometimes it produces useful code. And often, it'll simply call the "doTheHardPart()" method. I've even caught it literally writing the wrong algor…

My company just broadly adopted AI. It’s not a tech company and usually late to the game when it comes to tech adoption.

I’m counting down the days when some AI hallucination makes its way all the way to the C-suite. People will get way too comfortable with AI and don’t understand just how wrong it can be.

Some assumption will come from AI, no one will check it and it’ll become a basic business input. Then suddenly one day someone smart will say “thats not true” and someone will trace it back to AI. I know it.

I assume at that point in time there will be some general directive on using AI and not assuming it’s correct. And then AI will slowly go out of favor.

Re: I genuinely don't understand why some people are still bullish about LLMs

#528

Post by loxs »

Davidzheng wrote: post #393
tootie wrote: post #235

Earlier quoted context omitted.

Possibly a reaction to Bill Gates recent statements that it will begin replacing doctors and teachers. It's ridiculous to say LLMs are incredibly useful and valuable. It's highly dubious to think they can be trusted with actual critical tasks without careful supervision.

Capability today and next year will probably be very different in reliability

As someone who uses LLMs to write code every day, I don't see a huge progress since last year, so I'm also not that sure about next year.

Re: I genuinely don't understand why some people are still bullish about LLMs

#529

Post by bjornsing »

mattmanser wrote: post #525
bjornsing wrote: post #520

I’ve been using Claude a lot lately, and I must say I very much disagree. For example, the other day I was chatting with it about the health risks associated with my high consumption of grown salmon. It then generated a small program to simulate the accumulation of PCB in my body. I could review the program, ask questions about the assumptions, etc. It all seemed very reasonable. A toxicokinetic analysis it called it…

Google it. There's a bunch of scientific papers talking about that. Bet that plenty of those researchers have written python programs they've uploaded to GitHub and you just got one of their programs regurgitated It's not intelligence mate, it's just copying an existing program.

I don’t mind to be honest. I don’t expect more intelligence than that from my doctor either. I want them to identify the relevant science and regurgitate / apply it.

Re: I genuinely don't understand why some people are still bullish about LLMs

#530

Post by simoncion »

simonw wrote: post #64
cool_dude85 wrote: post #47

Earlier quoted context omitted.

It seems hard to be bullish on LLMs as a generally useful tool if the solution to problems people have is "use trial and error to improve how you write your prompts, no, it's not obvious how to do so, yes, it depends heavily on the exact model you use."

You could say that about any power tool. A Mitre Saw is an amazing thing to have in a woodshop, but if you don't learn how to use it you're probably going to cut off a finger. The problem is that LLMs are power tools that are sold as being so easy to use that you don't need to invest any effort in learning them at all. That's extremely misleading.

Unlike the manufacturers of LLMs, the manufacturers of real-world power tools

* Know how they work

* Are legally liable for defects in design or manufacture that cause injury, death, or property damage

* Provide manuals that instruct the operator how to effectively and safely use the power tool

Post reply on HN