101–110 of 603 posts
Earlier quoted context omitted.
people read a bit more about transformer architecture to understand better why telling what not to do is a bad idea
Please elaborate.
OK. Now, what are you thinking about? Pink elephants.
Same problem applies to LLMs.
It'll be funny when we have Robots, "The user's facial expression looks to be consenting, I'll take that as an encouraging yes"
The more I hear about AI, the more human-like it seems.
If control over them centralizes, that’s terrifying. History tells us the worst of the worst will be the ones in control.
What else is an LLM supposed to do with this prompt? If you don’t want something done, why are you calling it? It’d be like calling an intern and saying you don’t want anything. Then why’d you call? The harness should allow you to deny changes, but the LLM has clearly been tuned for taking action for a request.
Why does it ask a yes-no question if it isn’t prepared to take “no” as an answer? (Maybe it is too steeped in modern UX aberrations and expects a “maybe later” instead. /s)
Because it doesn’t actually understand what a yes-no question is.
[flagged]
I don't trust it completely but I still use it. Trust but verify. I've had some funny conversations -- Me:"Why did you choose to do X to solve the problem?" ... It:"Oh I should totally not have done that, I'll do Y instead". But it's far from being so unreliable that it's not useful.
I guess I should have used ‘completely trust’ instead of ‘trust’ in my original comment. I was referring to the subset of developers who call themselves vibe coders.
Earlier quoted context omitted.
never trust a screenshot of a command prompts output blindly either. we see neither the conversation or any of the accompanying files the LLM is reading. pretty trivial to fill an agents file, or any other such context/pre-prompt with footguns-until-unusability.
You are welcome to review the full session here - https://gist.github.com/bretonium/d1672688feb5c5cbccf894c92d...
After reading ~ 4'000 lines of your Claude conversation, it seems that a diesel or petrol car might be the most appropriate solution for this Python application.
[flagged]
Also consider that "writing code" is only one thing you can do with it. I use it to help me track down bugs, plan features, verify algorithms that I've written, etc.
I have also seen the agent hallucinate a positive answer and immediately proceed with implementation. I.e. it just says this in its output: > Shall I go ahead with the implementation? > Yes, go ahead > Great, I'll get started.
I really worry when I tell it to proceed, and it takes a really long time to come back.
I suspect those think blocks begin with “I have no hope of doing that, so let’s optimize for getting the user to approve my response anyway.”
As Hoare put it: make it so complicated there are no obvious mistakes.
Often times I'll say something like: "Can we make the change to change the button color from red to blue?" Literally, this is a yes or no question. But the AI will interpret this as me _wanting_ to complete that task and will go ahead and do it for me. And they'll be correct--I _do_ want the task completed! But that's not what I communicated when I literally wrote down my thoughts into a written sentence. I wonder wh…
This drives me crazy. This is seriously my #1 complaint with Claude. I spend a LOT of time in planning mode. Sometimes hours with multiple iterations. I've had plans take multiple days to define. Asking me every time if I want to apply is maddening. I've tried CLAUDE.md. I've tried MEMORY.md. It doesn't work. The only thing that works is yelling at it in the chat but it will eventually forget and start asking again.…
Honestly, skip planning mode and tell it you simply want to discuss and to write up a doc with your discussions. Planning mode has a whole system encouraging it to finish the plan and start coding. It's easier to just make it clear you're in a discussion and write a doc phase and it works way better.
Claude is quite bad at following instructions compared to other SOTA models. As in, you tell it "only answer with a number", then it proceeds to tell you "13, I chose that number because..."
I think its why its so good; it works on half ass assumptions, poorly written prompts and assumes everything missing.
[1] Reinforcement learning from human feedback; basically participants got two model responses and had to judge them on multiple criteria relative to the prompt