Live data from Hacker News

The Unreliability of LLMs and What Lies Ahead

verissimo.substack.com

41–50 of 164 posts

Re: The Unreliability of LLMs and What Lies Ahead

#41

It's hard to say "never" in technology. History isn't really on your side. However, LLMs have largely proven to be good at things computers were are already good at: repetitive tasks, parallel processing, and data analysis. There's nothing magical about an LLM that seems to be defeating the traditional paradigm. Increasingly I lean toward an implosion of the hype cycle for AI.

[dead]

Re: The Unreliability of LLMs and What Lies Ahead

#42
LLMs can't evaluate their own output. LLMs suggest possibilities, but can't evaluate them. Imagine an insane man who is rumbling something smart, but doesn't self-reflect. The evaluation is done against some framework of values that are considered true: the rules of a board game, the language syntax or something else. LLMs also can't fabricate evaluation because the latter is a rather rigid and precise model, a unlike natural language. Otherwise you could set up two LLMs questioning each other.

Re: The Unreliability of LLMs and What Lies Ahead

#43
post #3

My experience with LLm-based chat is so different from what the article (and some friends) describe. I use LLM chat for a wide range of tasks including coding, writing, brainstorming, learning, etc. It’s mostly right enough. And so my usage of it has only increased and expanded. I don’t know how less right it needs to be or how often to reduce my usage. Honestly, I think it’s hard to change habits and LLM chat, at it…

I really don't understand people who are down on LLM. In terms of code output. I have gone from the productivity of being a Sr. Engineer to a team with .8 of a Sr. Engineer, 5 Jr. Engineers and One dude solely dedicated to reading/creating documentation. Unlike a lot of my fellow engineers who are also from traditional CS backgrounds and haven't worked in revenue restricted startup environments, I also have been VERY…

There is this interesting thing called the Paradox of Automation where increasing automation increases the importance of human intervention. We are trying this out on a societal level. It will be.. interesting, to say the least.

Also, congratulations on becoming a team. I sure hope you have the mental bandwidth to check all that output carefully. If so, doubly congrats, because you might be the smartest human that ever lived.

Re: The Unreliability of LLMs and What Lies Ahead

#44
There are jobs out there that have always been unreliable.

A classic example is the Travel Agent. This was already a job driven to near-extinction just by Google, but LLMs are a nail in the travel agent coffin.

The job was always fuzzy. It was always unreliable. A travel agent recommendation was never a stamp of quality or guarentee of satisfaction.

But now, I can ask an LLM to compare and contrast two weeks in the Seychelles with two weeks in the Caribbean, have it then come up with sample itineraries and sample budgets.

Is it going to be accurate? No, it'll be messy and inaccurate, but sometimes a vibe check is all you ever wanted to confirm that yeah, you should blow your money on the Seychelles, or to confirm that actually, you were right to pick the Caribbean.

Or that actually, both are twice the amount you'd prefer to spend, where dear ChatGPT would be more suitable?

etc.

When it comes down to the nitty-gritty, does it start hallucinating hotels and prices? Sure, at that point you break out trip-advisor, etc.

But as a basic "I don't even know where I want to go on holiday ( vacation ), please help?" it's fantastic.

Re: The Unreliability of LLMs and What Lies Ahead

#45
post #36

Earlier quoted context omitted.

> Don't use LLM's to do 2 + 2. Don't use LLM's to ask how many r's are in strawberry But use them to do more important things that require more precision and accuracy? No thanks

When you employ your developers at $200K/yr you won't trust them to tell you the first one hundred digits of pi, but you'll trust them with your business logic, which is much more important and mission-critical to you. Same thing.

The difference is that (hopefully) your employee is honest enough to say "I do not know the first 100 digits of Pi offhand but I can find out"

An LLM will happily produce a string of 100 digits that might be the first 100 digits of Pi, might be some known sequence of 100 digits in Pi but not the first 100, or might be 100 random digits that have nothing to do with Pi

Re: The Unreliability of LLMs and What Lies Ahead

#47

I'm no AI fan, but articles talking about the shortcomings of LLM's seem to have to be complaining that forks aren't good for drinking soup. Don't use LLM's to do 2 + 2. Don't use LLM's to ask how many r's are in strawberry. For the love of God. It's not actual intelligence. This isn't hard. It just randomly spits out text. Use it for what it's good at instead. Text. Instead of hunting for how to do things in program…

People need to stop recommending forks to drink soup with.

Re: The Unreliability of LLMs and What Lies Ahead

#48
post #3

My experience with LLm-based chat is so different from what the article (and some friends) describe. I use LLM chat for a wide range of tasks including coding, writing, brainstorming, learning, etc. It’s mostly right enough. And so my usage of it has only increased and expanded. I don’t know how less right it needs to be or how often to reduce my usage. Honestly, I think it’s hard to change habits and LLM chat, at it…

> It’s mostly right enough. Honestly this is why your experience is different: your expectations are different (and likely lower). I never find they are "mostly right enough", I find they are "mostly wrong in ways that range from subtle mistakes to extremely incorrect". The more subtly they are wrong, the worse I rate their output actually, because that is what costs me more time when I try to use them I want tools t…

This. I use LLMs for some tasks, but for more complex issues, I do it myself. I tried to use it for a project by defining each task as clearly as possible, and I spent weeks trying to come up with something useful. Mind you, I achieved 80% of what I wanted after iterating and "telling" the chat that their answers were wrong, and going over the code to double-check if everything was okay. Now I use it for specific, simple tasks if these are work-related, and then use it for random kinds of stuff that I can verify by going to the actual source.

Re: The Unreliability of LLMs and What Lies Ahead

#50

It's hard to say "never" in technology. History isn't really on your side. However, LLMs have largely proven to be good at things computers were are already good at: repetitive tasks, parallel processing, and data analysis. There's nothing magical about an LLM that seems to be defeating the traditional paradigm. Increasingly I lean toward an implosion of the hype cycle for AI.

What I don’t understand is, how can a liar be good at data analysis?
Post reply on HN