Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Unreliability of LLMs and What Lies Ahead

verissimo.substack.com

Post reply on HN
Print view View original

31–40 of 164 posts

Re: The Unreliability of LLMs and What Lies Ahead

#31

Post by mastercheif »

kenjackson wrote: post #14
nomel wrote: post #9

Earlier quoted context omitted.

From what I can tell, rather than a simple difference in expectation (which could explain your positive experience vs others), it seems to be a "comfort within uncertainty" difference that, from what I can tell, is a personality trait! You're comfortable with the uncertainty, and accommodate it in your use and expectations. You're left feeling good about the experience, within that uncertainty. Others are repelled by…

I wonder if this is like dishwasher usage. As a kid growing up we never used the dishwasher. It was just the drying rack. The reason was you had to rinse off the big stuff anyways, and then the resulting quality of dishwashing was poor in it. You'd often get a fork with rice stuck between it still, which was unacceptable. As a grown up now I use a dishwasher for everything that is permitted to go in it. I still have…

You don’t need to pre-wash dishes before they go in the sink, beyond a basic scrapping of the plate into the garbage.

Pre-washing dishes degrades the performance of the dishwasher. This is due to the use of enzymes in modern detergent formulations.

I’ve sent dozens of people the Technology Connections video on this topic to rave reviews: https://youtu.be/jHP942Livy0

Re: The Unreliability of LLMs and What Lies Ahead

#32

Post by koakuma-chan »

thorum wrote: post #13

Good article. Agree that general unreliability will continue to be an issue since it's fundamental to how LLMs work. However, it would surprise me if there was still a significant gap between single-turn and multi-turn performance in 18 months. Judging by improvements in the last few frontier model releases, I think the top AI labs have finally figured out how to train for multi-turn and agentic capabilities (likely…

They’re reliable enough for many use cases

Re: The Unreliability of LLMs and What Lies Ahead

#33

Post by TheOtherHobbes »

nomel wrote: post #9
jmathai wrote: post #3

My experience with LLm-based chat is so different from what the article (and some friends) describe. I use LLM chat for a wide range of tasks including coding, writing, brainstorming, learning, etc. It’s mostly right enough. And so my usage of it has only increased and expanded. I don’t know how less right it needs to be or how often to reduce my usage. Honestly, I think it’s hard to change habits and LLM chat, at it…

From what I can tell, rather than a simple difference in expectation (which could explain your positive experience vs others), it seems to be a "comfort within uncertainty" difference that, from what I can tell, is a personality trait! You're comfortable with the uncertainty, and accommodate it in your use and expectations. You're left feeling good about the experience, within that uncertainty. Others are repelled by…

It's not comfort with uncertainty, it's discomfort with the predictable effects of uncertainty.

I don't want to have to waste time tidying up after an unreliable software tool which is being sold as saving me time. I don't want to be misled by hallucinated fantasies that have no relationship to reality. (See also - lawyers getting laughed out of courtrooms because of this.)

I don't want to have to cancel a travel booking because an AI agent booked me a holiday in Angkor Wat when I wanted a train ticket to Crystal Palace in South London.

Hypotheticals? Not even slightly. Ask anyone who's lost their KDP author account on Amazon or been locked out of Meta because of AI moderation errors.

This is common sense, not some kind of personality flaw.

I'm happy using LLMs for coding and research, but it's also clear the technology is in perpetual beta - at best - and is being wildly oversold.

Normal software operating with this level of reliability would be called "very buggy."

But apparently LLMs get a pass because one day they might not be as buggy as they are today.

Which - if you think about it - is ridiculous, even by the usual standards of the software industry.

Re: The Unreliability of LLMs and What Lies Ahead

#34

Post by coliveira »

AlienRobot wrote: post #23

I'm no AI fan, but articles talking about the shortcomings of LLM's seem to have to be complaining that forks aren't good for drinking soup. Don't use LLM's to do 2 + 2. Don't use LLM's to ask how many r's are in strawberry. For the love of God. It's not actual intelligence. This isn't hard. It just randomly spits out text. Use it for what it's good at instead. Text. Instead of hunting for how to do things in program…

The problem is exactly how the public will learn "not to ask 2+2". When you have a well trained professional using an LLM it's all great. They know how to separate hallucination from actually good results as you do. The problem lies with the general public and new workers who will, no questions about it, use the AI generated results as some sort of truth.

Re: The Unreliability of LLMs and What Lies Ahead

#35

Post by fellowniusmonk »

CuriouslyC wrote: post #30
foobiekr wrote: post #18

Earlier quoted context omitted.

Charitably, your low expectations are probably the source of your finding them acceptable. It’s also possible - and you should not take this as an insult, it’s just the way it is - you may not know enough about the subjects of your interactions to really spot how wrong they are. However the cases you list - brainstorming - don’t really care about wrong answers. Coding is in the eye of the beholder, but for anything t…

The code that the best frontier models produce is definitely good if you prompt it with what you believe "good" means, with the caveat that code quality depends heavily on the language -- Python, Typescript/Javascript, Java and C are quite good, Rust, C++ and Go tend to be decent to weak depending on the specific model, and other languages are poor.

Yeah, I've had mixed results with Rust. Oddly it's been most helpful for me so far in getting Rust code running in WASM without having to know anything about WASM, which I have found delightful.

Re: The Unreliability of LLMs and What Lies Ahead

#36

Post by yongjik »

bluefirebrand wrote: post #27
AlienRobot wrote: post #23

I'm no AI fan, but articles talking about the shortcomings of LLM's seem to have to be complaining that forks aren't good for drinking soup. Don't use LLM's to do 2 + 2. Don't use LLM's to ask how many r's are in strawberry. For the love of God. It's not actual intelligence. This isn't hard. It just randomly spits out text. Use it for what it's good at instead. Text. Instead of hunting for how to do things in program…

> Don't use LLM's to do 2 + 2. Don't use LLM's to ask how many r's are in strawberry But use them to do more important things that require more precision and accuracy? No thanks

When you employ your developers at $200K/yr you won't trust them to tell you the first one hundred digits of pi, but you'll trust them with your business logic, which is much more important and mission-critical to you.

Same thing.

Re: The Unreliability of LLMs and What Lies Ahead

#37

Post by bluefirebrand »

mjr00 wrote: post #28
bluefirebrand wrote: post #15

Earlier quoted context omitted.

Sure, but in both cases you are running a real risk of producing incorrect data If you're a product lead and you ask an LLM to produce a script that gets that output, you still should verify the output is correct Otherwise you run a real risk of seeming like an idiot later when you give a report on "tickets closed in the past week" and your data is completely wrong. "Why hasn't John closed any tickets this week? Is h…

You're absolutely right. You need to verify the script works, and you need to be able to read the code to see what it's actually doing and if it passes the smell test (as a sibling commenter said, the same way you would for a code snippet off StackOverflow). But ultimately for these bits which are largely rote "take data from API, transform into data format X" tasks, LLMs do a great job getting at least 95% of the wa…

> so why not outsource that to an LLM and verify the output?

I mean sure, why not. My argument isn't that it doesn't work, it's that it doesn't really save time

If you try to have it do big changes you will be swamped reviewing those changes for correctness for a long time while you build a mental model of the work

If you have it do small changes, the actual performance improvement is marginal at best, because small changes already don't take much time or effort to create

I really think that LLM-coding has largely just shifted "time spent typing" to "time spent reviewing"

Yes, past a certain size reviewing is faster than typing. But LLMs are not producing terribly good output for large amounts of code still

Re: The Unreliability of LLMs and What Lies Ahead

#38

Post by karn97 »

thorum wrote: post #13

Good article. Agree that general unreliability will continue to be an issue since it's fundamental to how LLMs work. However, it would surprise me if there was still a significant gap between single-turn and multi-turn performance in 18 months. Judging by improvements in the last few frontier model releases, I think the top AI labs have finally figured out how to train for multi-turn and agentic capabilities (likely…

Reasoning is just the worst kind of stop gap measure. The state that should emerge internally is forced through automating prompts. And you can clearly see this because the models rarely follow their own "reasoning". Its just auto self prompting

Re: The Unreliability of LLMs and What Lies Ahead

#39

Post by AlienRobot »

coliveira wrote: post #34
AlienRobot wrote: post #23

I'm no AI fan, but articles talking about the shortcomings of LLM's seem to have to be complaining that forks aren't good for drinking soup. Don't use LLM's to do 2 + 2. Don't use LLM's to ask how many r's are in strawberry. For the love of God. It's not actual intelligence. This isn't hard. It just randomly spits out text. Use it for what it's good at instead. Text. Instead of hunting for how to do things in program…

The problem is exactly how the public will learn "not to ask 2+2". When you have a well trained professional using an LLM it's all great. They know how to separate hallucination from actually good results as you do. The problem lies with the general public and new workers who will, no questions about it, use the AI generated results as some sort of truth.

Maybe use an LLM to detect when the public is asking the wrong question and display a message saying "As a large language model, I don't know how to count."

Re: The Unreliability of LLMs and What Lies Ahead

#40

Post by lcfcjs6 »

dist-epoch wrote: post #4
josefritzishere wrote: post #2

It's hard to say "never" in technology. History isn't really on your side. However, LLMs have largely proven to be good at things computers were are already good at: repetitive tasks, parallel processing, and data analysis. There's nothing magical about an LLM that seems to be defeating the traditional paradigm. Increasingly I lean toward an implosion of the hype cycle for AI.

Funny, I don't remember any computer program in the past being able to explain a news article through the lens of one particular philosopher. Or being able to explain the static physical forces in a picture that are keeping a structure from collapsing. Or recommend me a python library which does X, Y and Z with constraints A, B and C. But I guess you can file all the above under "data analysis".

[dead]
Post reply on HN