Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

That post never existed. Stop listening to that thing

rachelbythebay.com

Post reply on HN
Print view View original

141–150 of 184 posts

Re: That post never existed. Stop listening to that thing

#141

Post by lazide »

recursive wrote: post #90
grey-area wrote: post #6

> The worst part is that everyone who's decided to willingly lobotomize themselves is going to have to come to the realization that these things are full of shit. It'll have to happen one by one, and nobody else can make it happen for them. A fascinating dichotomy has become apparent between those who trust LLM output and those who don’t and don’t understand why you would. Surely if the machine you go to for answers…

> Surely if the machine you go to for answers regularly makes things up you would just stop using it? Perhaps people have to be burned by something really bad personally before they realise the limitations? LLMs are very convincing and persuasive. Psychics are still in business. Although to be fair they probably don't have as much revenue. I think people really enjoy being told how smart and insightful they are, and…

A big factor behind psychics is because it feeds a deep need to believe that someone knows and/or there is actually a plan.

Re: That post never existed. Stop listening to that thing

#142

Post by lazide »

Eddy_Viscosity2 wrote: post #127
grey-area wrote: post #6

> The worst part is that everyone who's decided to willingly lobotomize themselves is going to have to come to the realization that these things are full of shit. It'll have to happen one by one, and nobody else can make it happen for them. A fascinating dichotomy has become apparent between those who trust LLM output and those who don’t and don’t understand why you would. Surely if the machine you go to for answers…

> Surely if the machine you go to for answers regularly makes things up you would just stop using it? People still respond to ads and political speeches.

Churches remain popular as well.

Re: That post never existed. Stop listening to that thing

#143

Post by lazide »

perching_aix wrote: post #125
Revisional_Sin wrote: post #86

Earlier quoted context omitted.

What mistakes do they make?

On the more general side, it's a bit hard to describe, just like it is hard to describe when someone "Googles bad". They ask self serving questions, underspecify their requests, omit crucial context that the agent is blatantly not going to have access to, or subtly misdirect the agent. They expect the agent to figure out everything: you'll never catch them write a prompt longer than one or two sentences. They never s…

Having done all that (what you’re doing) - it often just ends up wasted effort, with poor quality at the end.

Why not just actually do the thing, instead?

Re: That post never existed. Stop listening to that thing

#144

Post by ButlerianJihad »

lazide wrote: post #141
recursive wrote: post #90

Earlier quoted context omitted.

> Surely if the machine you go to for answers regularly makes things up you would just stop using it? Perhaps people have to be burned by something really bad personally before they realise the limitations? LLMs are very convincing and persuasive. Psychics are still in business. Although to be fair they probably don't have as much revenue. I think people really enjoy being told how smart and insightful they are, and…

A big factor behind psychics is because it feeds a deep need to believe that someone knows and/or there is actually a plan.

We only need to look at the Oracle of Delphi to see that people have been seeking personal spiritual advice for millennia. And how has that been working out for us?

Even today, going to a therapist and having 1:1 sessions is a rational activity, and even covered by insurance. What do we hope to derive from therapy sessions but some personal insight and improvement?

You know, I go to church, and from my perspective, sometimes the most difficult discernment for an individual is between "The Holy Spirit's message to us in general" or "the general messaging to everyone around us" vs. "what I receive and my personal interpretation of things".

When preachers and oracles and leaders are speaking in generalities and trying to get big followings and trying to appeal to the widest audiences, that's when it's most difficult for us to determine what God is really saying to us, in our own hearts; that special instruction for our own lives. We can't actually get that from an oracle, psychic, or any 3rd party. It really needs to come from our own well-formed conscience.

And it's the same with a chatbot or LLM conversation. We can pose questions, make prompts, and get spammed with tokens and walls of text. No matter how personalized, it's still up to us to interpret that, and extract nuggets of news that we can use. It always has been.

Re: That post never existed. Stop listening to that thing

#145

Post by lazide »

ButlerianJihad wrote: post #144
lazide wrote: post #141

Earlier quoted context omitted.

A big factor behind psychics is because it feeds a deep need to believe that someone knows and/or there is actually a plan.

We only need to look at the Oracle of Delphi to see that people have been seeking personal spiritual advice for millennia. And how has that been working out for us? Even today, going to a therapist and having 1:1 sessions is a rational activity, and even covered by insurance. What do we hope to derive from therapy sessions but some personal insight and improvement? You know, I go to church, and from my perspective, s…

Sure, and hamburgers and shakes are bad for us. Also very popular.

Re: That post never existed. Stop listening to that thing

#146

Post by happymellon »

JeremyNT wrote: post #109
grey-area wrote: post #6

> The worst part is that everyone who's decided to willingly lobotomize themselves is going to have to come to the realization that these things are full of shit. It'll have to happen one by one, and nobody else can make it happen for them. A fascinating dichotomy has become apparent between those who trust LLM output and those who don’t and don’t understand why you would. Surely if the machine you go to for answers…

> A fascinating dichotomy has become apparent between those who trust LLM output and those who don’t and don’t understand why you would. I feel like the most pragmatic perspective is "trust but verify." This is why they're so effective at coding: you can run the code yourself (or the test suite) to verify that it actually does what it's supposed to. And maybe these people finding hallucinated results on Rachel's site…

> I feel like the most pragmatic perspective is "trust but verify."

This perspective I really don't get.

What has any of the LLM companies done to earn my trust? I lean more towards "verify because I don't trust".

Re: That post never existed. Stop listening to that thing

#147

Post by grey-area »

dcrazy wrote: post #119
grey-area wrote: post #52

Earlier quoted context omitted.

We know they are, and the author of the article cites the proof. LLMs do hallucinate, there is no way to make them not do it, because of the way they work.

A “hallucination” is an authoritative counterfactual statement returned as a response. Why do you think it is impossible to engineer an LLM (by which I am including tool usage and RAG) that catches and prevents such statements?

Because they are word generators without any concept of quality save what is in their weights and they have been trained on the internet, much of which is wrong or inappropriate for any given context. They have also been trained to be people pleasers and do as they are told.

The popular answer is sometimes the wrong answer.

Re: That post never existed. Stop listening to that thing

#148

Post by sph »

grey-area wrote: post #6

> The worst part is that everyone who's decided to willingly lobotomize themselves is going to have to come to the realization that these things are full of shit. It'll have to happen one by one, and nobody else can make it happen for them. A fascinating dichotomy has become apparent between those who trust LLM output and those who don’t and don’t understand why you would. Surely if the machine you go to for answers…

> Surely if the machine you go to for answers regularly makes things up you would just stop using it?

Steve Yegge likened LLMs to slot machines. The human brain is very vulnerable to random reward systems. If you get an hallucination, just pull the lever once more.

Re: That post never existed. Stop listening to that thing

#149

Post by perching_aix »

lazide wrote: post #143
perching_aix wrote: post #125

Earlier quoted context omitted.

On the more general side, it's a bit hard to describe, just like it is hard to describe when someone "Googles bad". They ask self serving questions, underspecify their requests, omit crucial context that the agent is blatantly not going to have access to, or subtly misdirect the agent. They expect the agent to figure out everything: you'll never catch them write a prompt longer than one or two sentences. They never s…

Having done all that (what you’re doing) - it often just ends up wasted effort, with poor quality at the end. Why not just actually do the thing, instead?

You mean why not work myself instead of the agent? That'd be because for me it's been working great, and so it does make sense. In the scenarios it doesn't, I do indeed just fall back to manual work. A lot of those scenarios are obvious too, so not too many wasted runs to speak of either.

I did give up on cheaper models, they required constant babysitting, and in those cases yes, the benefits indeed evaporated. The expensive models have been genuinely working wonders though, and were still able to justify themselves economically plenty, at least by my own measurements.

I think there's also one underappreciated and indirect way agents help with productivity: they counteract the attention span collapse of the past years. By being addictive themselves, they keep you engaged, and being engaged means being productive. Not even asking an agent to check something out feels too rich, and once you've asked, you're already one foot into the flow.

There's also definitely been some honeymoon effect going on for me, where I dived into more work more readily, just to see if the agent can figure things out on its own.

Re: That post never existed. Stop listening to that thing

#150

Post by lazide »

perching_aix wrote: post #149
lazide wrote: post #143

Earlier quoted context omitted.

Having done all that (what you’re doing) - it often just ends up wasted effort, with poor quality at the end. Why not just actually do the thing, instead?

You mean why not work myself instead of the agent? That'd be because for me it's been working great, and so it does make sense. In the scenarios it doesn't, I do indeed just fall back to manual work. A lot of those scenarios are obvious too, so not too many wasted runs to speak of either. I did give up on cheaper models, they required constant babysitting, and in those cases yes, the benefits indeed evaporated. The e…

Being engaged == doing stuff, not necessarily being actually productive.

Busy does not always equal doing something useful.

Easy to pay a lot to the model vendor though eh?

Post reply on HN