Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Unreliability of LLMs and What Lies Ahead

verissimo.substack.com

Post reply on HN
Print view View original

101–110 of 164 posts

Re: The Unreliability of LLMs and What Lies Ahead

#101

Post by empath75 »

bluefirebrand wrote: post #96
empath75 wrote: post #76

Earlier quoted context omitted.

They save me a tremendous amount of time, you just need to be smart about what you try to get them to do. _Busy work_ is what you want to focus on, not anything that takes a ton of domain knowledge and intelligence. Just as an example from today, i had a huge pile of yaml documents that needed to have some transformations done to them -- they were pretty simple and obvious, but I just went into cursor, give it a befo…

> _Busy work_ is what you want to focus on, not anything that takes a ton of domain knowledge and intelligence Eh.. Maybe that's more of a sign that we shouldn't be doing busywork in the first place

You are in a magical place if you never have to do busy work.

Re: The Unreliability of LLMs and What Lies Ahead

#102

Post by nomel »

TheOtherHobbes wrote: post #33
nomel wrote: post #9

Earlier quoted context omitted.

From what I can tell, rather than a simple difference in expectation (which could explain your positive experience vs others), it seems to be a "comfort within uncertainty" difference that, from what I can tell, is a personality trait! You're comfortable with the uncertainty, and accommodate it in your use and expectations. You're left feeling good about the experience, within that uncertainty. Others are repelled by…

It's not comfort with uncertainty, it's discomfort with the predictable effects of uncertainty. I don't want to have to waste time tidying up after an unreliable software tool which is being sold as saving me time. I don't want to be misled by hallucinated fantasies that have no relationship to reality. (See also - lawyers getting laughed out of courtrooms because of this.) I don't want to have to cancel a travel boo…

These apply:

> comfortable with the uncertainty, and accommodate it in your use

Many of the tasks you listed are require absolute determinism.

> regardless of how well it may work for a subset of tasks they try

You're using examples of absolute determinism, even though, with certainty, it has worked for some tasks you've throw at it.

Re: The Unreliability of LLMs and What Lies Ahead

#103

Post by rini17 »

ToucanLoucan wrote: post #52
josefritzishere wrote: post #2

It's hard to say "never" in technology. History isn't really on your side. However, LLMs have largely proven to be good at things computers were are already good at: repetitive tasks, parallel processing, and data analysis. There's nothing magical about an LLM that seems to be defeating the traditional paradigm. Increasingly I lean toward an implosion of the hype cycle for AI.

LLMs are a legitimate technology with legitimate applications. However in a desperate bid for a new iPhone moment to assure Wall Street that the fantasy of infinite growth in a finite world is possible, they have utterly lost the plot regarding what statistical analysis of words at scale is capable of doing. Useless? Far from it. The basis for a 300 billion company with no meaningful products after almost a decade wo…

You seriously underestimate the appeal of burning cycles on GPUs to get something cool, if barely useful, out. Cryptocurrencies are still very much alive, too.

Re: The Unreliability of LLMs and What Lies Ahead

#104

Post by whyowhy3484939 »

fellowniusmonk wrote: post #63
whyowhy3484939 wrote: post #43

Earlier quoted context omitted.

There is this interesting thing called the Paradox of Automation where increasing automation increases the importance of human intervention. We are trying this out on a societal level. It will be.. interesting, to say the least. Also, congratulations on becoming a team. I sure hope you have the mental bandwidth to check all that output carefully. If so, doubly congrats, because you might be the smartest human that ev…

I appreciate you're incredulity and snark! Dismissing without engagement is a fun ability to exercise. I look forward to talking past each other going forward :-) HackerNews typically doesn't appreciate and will ban accounts for that type of engagement as it is just personal and not a factual wrestling with the point of discussion, I see you are new here and I would encourage you to not continue to engage in the patt…

Ah, I met my match it seems.

I confess I might be showing signs of unlawful thought patterns. I will correct that, fellowniusmonk. Thanks for pointing that out.

I am in the "code is not an asset, it's a liability"-camp and our recently acquired ability to swiftly defecate metric tons of it is not something I am particularly thrilled about. In fact, I find "senior" engineers using LoC as a productivity metric highly suspect - at best. I thought we passed that phase a decade or two ago. Not saying you are one, but in the spirit of talking past each other I thought it prudent to put up a good straw man.

All in all to be completely honest I find it hard to parse your original point so I concur I wasn't engaging properly. To be fair you opened with "in terms of code output" so that's what triggered me I guess.

Re: The Unreliability of LLMs and What Lies Ahead

#105

Post by brentm »

andrewmutz wrote: post #98
brentm wrote: post #87

This is a good articulation of what is a real concern around the AI bull thesis. If a calculator works great 99% of the time you could not use that calculator to build a bridge. Using AI for more than code generation is still very difficult and requires a human in the loop to verify the results. Sometimes using AI ends up being less productive because you're spending all your time debugging it's outputs. It's great b…

What we are seeing with our customers is that LLM errors are a very manageable problem. End users adapt pretty quickly to the idea that AI systems aren't perfect. In many cases AI products are doing tasks that used to be done by humans and these humans were making mistakes too, so the end user is used to the idea that the task will get accomplished with some non-zero error rate. You just need to build your products i…

Yea I just think the true unlock in productivity will come from not requiring a human in the loop.

Re: The Unreliability of LLMs and What Lies Ahead

#106

Post by glial »

johnea wrote: post #5

> Internally, it uses a sophisticated, multi-path strategy, approximating the sum with one heuristic while precisely determining the final digit with another. Yet, if asked to explain its calculation, the LLM describes the standard 'carry the one' algorithm taught to humans. So, the LLM isn't just wrong, it also lies...

Talking about "truth" or "lies" with LLMs isn't helpful.

Re: The Unreliability of LLMs and What Lies Ahead

#107

Post by smeeger »

hallucinations are essentially the only thing keeping all knowledge workers from being made permanently redundant. if that doesnt make you a little concerned then you are a fool. and the predictions of all the experts in 2010 is that what is currently happening right in front of us could never happen within a hundred years. why are the predictions of experts more reliable now? anyone who dismisses the risks is just a sorry fool

Re: The Unreliability of LLMs and What Lies Ahead

#108

Post by mjburgess »

asadotzler wrote: post #88
mjburgess wrote: post #65

I think I'm settling on a "Gell-mann Amnesia" explanation of why people are so rabidly committed to the "acceptable veracity" of LLM output. When you don't know the facts, you're easily mislead by plausible-sounding analysis, and having been mislead -- a certain default prejudice to existing beliefs takes over. There's a significant asymmetry of effort in belief change vs. acquisition. I think there's also an ego-pro…

Bullshit works on lots of people. Seeming to be true, or even just plausible, is enough for most people. This is why powerful bullshit machines are dangerous tools.

If people were easy enough to convince that they had been deceived, then I'd not mind so much. It's the extraordinary lengths people will go to in order to protect the bullshit they acquired with far less scepticism. Genuinely wild leaps of logic, shallowness of reasoning, on-the-face-of-it non-sequiturs, claims offered as great defeaters which require only a single moment of reflection to see through.

This is the problem. The problem is how bullshit conscripts its dupes into this self-degradation and bad faith dialogue with others.

And of course, how there are mechanisms in society (LLMs now one of them) which correlate this self-degrading shallowness of reasoning -- so that all at once an expert is faced with millions of people with half-baked notions and a great desire to preserve them.

Re: The Unreliability of LLMs and What Lies Ahead

#109

Post by GuinansEyebrows »

dist-epoch wrote: post #4
josefritzishere wrote: post #2

It's hard to say "never" in technology. History isn't really on your side. However, LLMs have largely proven to be good at things computers were are already good at: repetitive tasks, parallel processing, and data analysis. There's nothing magical about an LLM that seems to be defeating the traditional paradigm. Increasingly I lean toward an implosion of the hype cycle for AI.

Funny, I don't remember any computer program in the past being able to explain a news article through the lens of one particular philosopher. Or being able to explain the static physical forces in a picture that are keeping a structure from collapsing. Or recommend me a python library which does X, Y and Z with constraints A, B and C. But I guess you can file all the above under "data analysis".

it is the result of data analysis. the computer program isn't explaining anything, or recommending anything. it's simply presenting the results of querying data analyzed at scale and returning the "most likely" result (as determined by the system prompt and human input from developers and users of the program). "most likely" is still a super-fuzzy grey area.

https://www.plough.com/en/topics/life/technology/computers-c...

Re: The Unreliability of LLMs and What Lies Ahead

#110

Post by tveita »

brentm wrote: post #87

This is a good articulation of what is a real concern around the AI bull thesis. If a calculator works great 99% of the time you could not use that calculator to build a bridge. Using AI for more than code generation is still very difficult and requires a human in the loop to verify the results. Sometimes using AI ends up being less productive because you're spending all your time debugging it's outputs. It's great b…

> If a calculator works great 99% of the time you could not use that calculator to build a bridge.

We know for certain that certified lawyers have committed malpractice by using ChatGPT, in part because the made-up citations are relatively easy to spot. Malpractice by engineers might take a little more time to discover.

Post reply on HN