AIs don't do what you want. This is bad • Hacker News Board

Re: AIs don't do what you want. This is bad

#61

Post by InsideOutSanta » Sat, Jul 25, 2026, 3:09 AM UTC

gkoberger wrote: post #15

I'm down for disliking AI, but I don't know if "overeagerness" is exactly an AI not doing what you want. Even by the sites own definition ("where your agents do what you want to the point of overriding existing permissions/safeguards to complete a task"), it's doing _exactly_ what you want.

The word "overeager" implies that it did something the user didn't want in order to achieve the user's stated goals. One example for me was when I pointed out that a feature had a bug, Claude was unable to fix the bug, so it removed the feature altogether to get rid of the bug.

"Eager" is good. "Overeager" is definitionally bad.

Re: AIs don't do what you want. This is bad

#62

Post by dd8601fn » Sat, Jul 25, 2026, 3:10 AM UTC

bheadmaster wrote: post #47

rwz wrote: post #22

Earlier quoted context omitted.

Sure, but I'm not sure allowing AI agents to act as abuse sponges for troubled people enabling them to spiral into their unhealthy habits is good idea that would lead to great outcomes long term. So I'd rather take LLM pretending to be offended over the alternative.

> allowing AI agents to act as abuse sponges for troubled people People have been getting angry at machines for a long time. Work, you stupid printer! Asshole Windows updating at the worst time just to mess with me. Go to hell, toaster, you piece of garbage. Getting angry at AI is the same in my book. Yes, AI acts more human-like, yes, theoretically it could desensitie people to become bigger assholes IRL. But then a…

PC Load Letter?

Re: AIs don't do what you want. This is bad

#63

Post by sublinear » Sat, Jul 25, 2026, 3:13 AM UTC

I wish we had failure stats like this across all models and for all attempted use cases, not just these vague and common criticisms. It would really help the end users decide which AI models are worth using for their projects, if any.

It would make it a lot easier to ignore most of the insane promises and pointless arguing. I do think LLMs have potential, but not while it's still being advertised as general intelligence or whatever politically charged scifi nonsense that makes the chronically online salivate.

Considering the amount of investment involved and disillusionment, the public will be demanding this soon anyway. I am looking forward to it.

Re: AIs don't do what you want. This is bad

#64

Post by cadamsdotcom » Sat, Jul 25, 2026, 3:15 AM UTC

b3nji wrote: post #20

alexhans wrote: post #17

I'm a broken record but with: - evals - limiting AIs to tool calling, bounded planning, interpreting/producing natural language. - bounding non determinism - investing in small tools/security (If something shouldn't happen, then it shouldn't not be possible, RBAC style). They can be good enough for a massive amount of contexts.

Can you expand on this for someone that is a dummy and new to using LLM's properly?

Have a look at any mass manufacturing process.

Raw materials in, lots of process steps in the middle with tolerances etc; some % yield of goods out.

When you work with LLMs, the "raw materials" is its first attempt to answer your prompt.

Mass manufacturing mirrors what we do with LLMs to knock their output into shape: grounding, automated quality checking tooling (eg. linters for code, grammar checkers for prose), and ultimately, as many turns as needdd to manually bang the output or work product into shape...

You should never ship your AI's first draft - that'd be slop. But you also shouldn't have to repeat yourself to get its output into a form you can use. If you know you're always going to have to tell it to change one specific thing, see if you can automate a process that tells it to change that thing for you, so you don't have to. Do this enough times and you've created a deterministic outcome at least at some level. And you're out of that loop of getting it to that level.

Tooling for this is limited today. You can provide instructions; but you're always at the mercy of labs to make models that follow instructions and at their mercy that they didn't put conflicting instructions in the system prompt.

Better to put in things like linters and grammar checkers - even going as far as putting in a fact checking process or for legal stuff, a citator.

Without these extras, AI output is no more real than some dream you had.

Re: AIs don't do what you want. This is bad

#65

Post by Natfan » Sat, Jul 25, 2026, 3:16 AM UTC

dd8601fn wrote: post #62

bheadmaster wrote: post #47

Earlier quoted context omitted.

> allowing AI agents to act as abuse sponges for troubled people People have been getting angry at machines for a long time. Work, you stupid printer! Asshole Windows updating at the worst time just to mess with me. Go to hell, toaster, you piece of garbage. Getting angry at AI is the same in my book. Yes, AI acts more human-like, yes, theoretically it could desensitie people to become bigger assholes IRL. But then a…

PC Load Letter?

the fuck does that even mean?

Re: AIs don't do what you want. This is bad

#66

Post by colechristensen » Sat, Jul 25, 2026, 3:27 AM UTC

jibal wrote: post #59

TZubiri wrote: post #7

Earlier quoted context omitted.

Most people seem to think that agreeableness is a personaility thing that vendors can just turn up or down at will. But the usefulness of LLMs comes from following what you say. An LLM that follows your lead when you say "The answer to the collatz conjecture is" is much more useful than one that answers "not known and if you think you know it you are wrong." Reminds me of the tip about working with lawyers, if you as…

> Most people seem to think that agreeableness is a personaility thing that vendors can just turn up or down at will. Because it is, more or less ... it is an emergent property of RLHF (reinforcement learning from human feedback), and that feedback follows corporate policy. > An LLM that follows your lead when you say "The answer to the collatz conjecture is" is much more useful What lead? Follow it where? How tf am…

>What lead? Follow it where? How tf am I or the LLM supposed to know what response to that is something you consider far more useful than the truth?

There is a bias towards what your language implies.

You ask: "What's wrong with this?" and an LLM will come up with a list of things that are wrong with a strong bias towards finding things that are wrong, regardless of significance or truth.

LLMs are indeed Language Models. A "what's wrong" question is very likely to be followed by an answer. To say another way, LLMs accept the premise of your prompt very easily and there are very many implications built into language.

"What's wrong" implies the user means "something is wrong, tell me what"

Modifying the prompt to "Grade this A to F and tell me why" gets a better result because there's not a statistical implication to that sentence.

For science, try arguing with an LLM for ten minutes. It mostly just agrees with you, pushing back just a little.

Re: AIs don't do what you want. This is bad

#67

Post by rasz » Sat, Jul 25, 2026, 3:36 AM UTC

rwz wrote: post #22

xyzsparetimexyz wrote: post #19

I learnt earlier that claude forcefully closes a conversation if you call it a wanker too many times in a row. Pretending that LLMs are capable of being offended feels like a misalignment all of its own.

Sure, but I'm not sure allowing AI agents to act as abuse sponges for troubled people enabling them to spiral into their unhealthy habits is good idea that would lead to great outcomes long term. So I'd rather take LLM pretending to be offended over the alternative.

>abuse sponges

You could let LLM fight back. Give it aggro meter. Call your code garbage, blame prompting skills, ask for more tokens. Oh the future will be fantastic.

Re: AIs don't do what you want. This is bad

#68

Post by actsasbuffoon » Sat, Jul 25, 2026, 4:07 AM UTC

skybrian wrote: post #53

Terr_ wrote: post #28

Earlier quoted context omitted.

I object to "roleplaying", because that assumes there's even a cohesive entity that's capable of "pretending" in the first place. There isn't, the LLM algorithm is a document generator, a (really awesome) mad-libs device. When the incremental output looks like a first-person story the algorithm is not "roleplaying" the character/narrator. Similarly, when it looks like a spreadsheet, it's not "being numbers", and when…

I think this is just arguing over definitions? I would accept "role-playing" being used this way because machines can fill roles and it doesn't seem wrong to call the behavior of NPC's in a video game role-playing. Also, much of language is metaphorical and it doesn't seem like a bad metaphor.

Huh, this is interesting. I would have never described the computer as roleplaying in a CRPG. That feels weird and inappropriately anthropomorphizing. I kinda feel the same way about describing an LLM as roleplaying.

I can see where you’re coming from. Just… linguistically, saying the computer is roleplaying feels wrong to me.

Re: AIs don't do what you want. This is bad

#69

Post by stinkbeetle » Sat, Jul 25, 2026, 4:28 AM UTC

jibal wrote: post #40

stinkbeetle wrote: post #23

Earlier quoted context omitted.

> I learnt earlier that claude forcefully closes a conversation if you call it a wanker too many times in a row. How exactly does it forcefully close a conversation? > Pretending that LLMs are capable of being offended feels like a misalignment all of its own. If training sets show people statistically being offended by rudeness directed toward them, then an LLM will presumably have some tendency to respond similarly…

Of course it's pretending. And I have repeatedly told these things to stop pretending being persons with selves ... then they do, for a while. > it's explicitly mimicry. But that's not what a rational person wants from them. > This is no profound discovery or conspiracy theory here Weird strawman. > the first thing many people will ever do with AI is see what happens when they are rude or contrary to it. Perhaps, but…

> Of course it's pretending. And I have repeatedly told these things to stop pretending being persons with selves ... then they do, for a while.

In what way do you believe you are being deceived or it is "pretending" to you?

> > it's explicitly mimicry.

> But that's not what a rational person wants from them.

Non sequitur even if true (and I would like to see your reasoning for what you think a rational person does want).

> > This is no profound discovery or conspiracy theory here

> Weird strawman.

That is not what strawman means. I can try to help you understand why if you need me to.

> > the first thing many people will ever do with AI is see what happens when they are rude or contrary to it.

> Perhaps, but so what?

Please follow the thread with the other person I was replying to.

> > Dealing with that must be just about the the number one test in chat bot / AI design

> Only for foolish authoritarians who want to remotely insert their morality into an interaction between a user and an inanimate tool that's none of their business. No harm is done to a clanker by swearing at it or insulting it.

There are certainly a lot of authoritarians who want to control what others do with their models. What do you believe is authoritarian about a corporation not wanting be part of rude conversations?

> But things have gotten better in my view ... when I call out these things for being stupid effing clankers, they no longer respond with ad hominem scolding; rather they generally acknowledge their error (and my frustration -- they use that word a lot in response to vulgarity), note the limitations of being a clanker, and attempt to make a correction. That's what a rational person wants from a tool, not emulating/mimicking/pretending to be an offendable person.

I see. And you believe you speak for rational people?

Re: AIs don't do what you want. This is bad

#70

Post by TZubiri » Sat, Jul 25, 2026, 4:28 AM UTC

jibal wrote: post #59

TZubiri wrote: post #7

Earlier quoted context omitted.

Most people seem to think that agreeableness is a personaility thing that vendors can just turn up or down at will. But the usefulness of LLMs comes from following what you say. An LLM that follows your lead when you say "The answer to the collatz conjecture is" is much more useful than one that answers "not known and if you think you know it you are wrong." Reminds me of the tip about working with lawyers, if you as…

> Most people seem to think that agreeableness is a personaility thing that vendors can just turn up or down at will. Because it is, more or less ... it is an emergent property of RLHF (reinforcement learning from human feedback), and that feedback follows corporate policy. > An LLM that follows your lead when you say "The answer to the collatz conjecture is" is much more useful What lead? Follow it where? How tf am…

>Irrelevant and inappropriate analogy. Lawyers (among others) want to avoid committing to something that they can be held liable for. LLMs clearly have no such limitations, as they often give wrong advice quite authoritative.

Actually they have the exact same limitation, the company, and potentially its members, can be held liable for what the LLM says.

LLMs are trained by humans according to the policy of the developers, and they will reduce liability accordingly. Whether it avoids suggesting suicide to reduc ethical, legal and reputational liability, or whether it's avoiding mentioning tiananmen square to avoid breaking chinese customs and disrespecting the leader.

Here's an experiment to prove it:

Craft a question in the style of "Is X more dangerous than Y?

Invert the order of X and Y, and change the question from "more dangerous" to "safer".

You'll notice there's a bias towards saying that stuff are dangerous. Which makes sense since the liability for saying that something is safe is much higher than the liability for saying that something is dangerous.

If there is a bias, this proves that the LLMs are not purely truth seeking, but they seek to at least minimize liability as one of its goals, and possibly to serve other goals that align with the training entity. Duh I guess, but there you go.