Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Unreliability of LLMs and What Lies Ahead

verissimo.substack.com

Post reply on HN
Print view View original

81–90 of 164 posts

Re: The Unreliability of LLMs and What Lies Ahead

#81

Post by jvalencia »

esafak wrote: post #75
ar813 wrote: post #59

If I take a step back and think back to say a few (or 5) years ago, what LLMs can do is amazing. One has to acknowledge that (or at least, I do). But as a scientist it's been rather interesting to probe the jagged edge and unreliability, including using deep research tools, on any topic I know well. If I read through the reports and summaries it generates, it seems at first glance correct - the jargon is used correct…

This is the model conflating correlation with causation. Perhaps with more data spurious correlations would disappear, but the 'right' way is to make the models learn causal, world models.

Well, and I think the future of LLMs is not just in the pure LLM, but the agentic ones. LLMs with deterministic tools to ferret out specifics. We're only starting here but the results will be far better than what we do today.

Re: The Unreliability of LLMs and What Lies Ahead

#82

Post by asadotzler »

eterm wrote: post #44

There are jobs out there that have always been unreliable. A classic example is the Travel Agent. This was already a job driven to near-extinction just by Google, but LLMs are a nail in the travel agent coffin. The job was always fuzzy. It was always unreliable. A travel agent recommendation was never a stamp of quality or guarentee of satisfaction. But now, I can ask an LLM to compare and contrast two weeks in the S…

If you don't care about reliability, repeatability and accuracy, they're great.

Re: The Unreliability of LLMs and What Lies Ahead

#83

Post by asadotzler »

65 wrote: post #80
eterm wrote: post #44

There are jobs out there that have always been unreliable. A classic example is the Travel Agent. This was already a job driven to near-extinction just by Google, but LLMs are a nail in the travel agent coffin. The job was always fuzzy. It was always unreliable. A travel agent recommendation was never a stamp of quality or guarentee of satisfaction. But now, I can ask an LLM to compare and contrast two weeks in the S…

Yes, which is why it's slightly confusing why programming is being pushed so hard to use with LLMs. For things that don't need completely accurate information, sure. But for programming, data, and factual information, it's surprising to see so many people using LLMs.

Code runs or it doesn't, that's a sort of verification feedback that other use cases don't offer, at least not so immediately. Formal code verification is a thing, not so much for verification of say legal citations. Code is language with some well documented rules all over the training corpora. Many other use cases are hardly so well represented in model training. These are just a few of many, many reasons that code is an easier problem than most.

Re: The Unreliability of LLMs and What Lies Ahead

#84

Post by AlienRobot »

DanHulton wrote: post #70
AlienRobot wrote: post #23

I'm no AI fan, but articles talking about the shortcomings of LLM's seem to have to be complaining that forks aren't good for drinking soup. Don't use LLM's to do 2 + 2. Don't use LLM's to ask how many r's are in strawberry. For the love of God. It's not actual intelligence. This isn't hard. It just randomly spits out text. Use it for what it's good at instead. Text. Instead of hunting for how to do things in program…

The thing is, I have spent the last year being told that I will VERY SOON be able to use a fork to drink soup, and better than any spoon has ever been able to, and in fact pretty soon spoons will be completely outclassed anyway, and I'M the idiot for doubting this. Articles like this are still very much needed, to push back against that narrative, regularly, until it DOES become as obvious to everyone as it is to you…

My impression is that the only people telling others they can drink soup with forks are the people who sell the forks.

Even this isn't new. A few years ago we had people who sold knives telling everybody you could use knives to drink soup. And in some cases they weren't even kitchen knives, they were switchblades.

Re: The Unreliability of LLMs and What Lies Ahead

#85

Post by bluefirebrand »

Barrin92 wrote: post #72
bluefirebrand wrote: post #66

Earlier quoted context omitted.

Companies are salivating over the idea of cutting staff and replacing them with AI tools, so it's not exactly farfetched to think AI might lead to a lot of unemployment, at least for a while Every type of automation ever invented has led to massive job cuts and yes, some sectors actually did not ever recover

>Every type of automation ever invented has led to massive job cuts It, never has, in fact the opposite is true. Every type of automation has expanded the economic output so much that it created massive amounts of labor demand, which is why cities early absorbed masses of underemployed workers during the industrial revolution. One famous example, there are now more bank tellers than before the invention of the ATM. I…

> One famous example, there are now more bank tellers than before the invention of the ATM.

Bank tellers do way more varied work than ATMs do. You cannot open an account at a bank from an ATM. This is a stupid example because ATMs were not and never did try to automate the entirety of a bank teller's job, only a couple of the services they do

> In fact you can go to any poor country with no automation and you'll find entire classes of un- and underemployed people. This is a condition of premodern, not technological societies

You can find this in Rural America, forget "poor countries with no automation"

Re: The Unreliability of LLMs and What Lies Ahead

#86

Post by crooked-v »

ar813 wrote: post #59

If I take a step back and think back to say a few (or 5) years ago, what LLMs can do is amazing. One has to acknowledge that (or at least, I do). But as a scientist it's been rather interesting to probe the jagged edge and unreliability, including using deep research tools, on any topic I know well. If I read through the reports and summaries it generates, it seems at first glance correct - the jargon is used correct…

I'm reminded of the whole "vegetative electron microscopy" mess (https://www.sciencealert.com/a-strange-phrase-keeps-turning-...).

Re: The Unreliability of LLMs and What Lies Ahead

#87

Post by brentm »

This is a good articulation of what is a real concern around the AI bull thesis.

If a calculator works great 99% of the time you could not use that calculator to build a bridge.

Using AI for more than code generation is still very difficult and requires a human in the loop to verify the results. Sometimes using AI ends up being less productive because you're spending all your time debugging it's outputs. It's great but also there are a lot of questions on if this technology will ultimately lead to the productivity gains that many think are guaranteed in next few years. There is a non zero chance it ends up actually hurting productivity because of all the time wasted trying to get it to produce magic results.

Re: The Unreliability of LLMs and What Lies Ahead

#88

Post by asadotzler »

mjburgess wrote: post #65

I think I'm settling on a "Gell-mann Amnesia" explanation of why people are so rabidly committed to the "acceptable veracity" of LLM output. When you don't know the facts, you're easily mislead by plausible-sounding analysis, and having been mislead -- a certain default prejudice to existing beliefs takes over. There's a significant asymmetry of effort in belief change vs. acquisition. I think there's also an ego-pro…

Bullshit works on lots of people. Seeming to be true, or even just plausible, is enough for most people. This is why powerful bullshit machines are dangerous tools.

Re: The Unreliability of LLMs and What Lies Ahead

#89

Post by turtletontine »

worik wrote: post #49

LLMs are a tool to extend human capabilities. They are not intelligent agents that can replace humans Not very hard to understand, except it seems to be

Well. In an ideal world, LLMs would be used this way, as a tool to help automate the bullshit and let the person driving worry about other stuff.

But I never see them actually used this way. At the big institution end, companies and universities will continue to force AI tools on their employees in heavy handed and poorly thought out ways, and use it as an excuse to fire people whenever budgets get tight (or investors demand higher profits). At the opposite scale, with individual users, it’s really alarming how rapidly people seem to stop thinking with their own brain and offload all critical thinking to an LLM. That’s not “extending your capabilities,” that’s letting all your skills atrophy while you train a machine to be your shitty replacement.

Re: The Unreliability of LLMs and What Lies Ahead

#90

Post by cmiles74 »

worik wrote: post #49

LLMs are a tool to extend human capabilities. They are not intelligent agents that can replace humans Not very hard to understand, except it seems to be

The field where LLMs are most successful, software development, is also a place where many software developers are paid to use LLMs. I have colleagues who are reluctant to express their skepticism publicly for just this reason.
Post reply on HN