Things like this make me sad because they make obvious that most people don’t understand a bit about how LLM work. The “answer before reasoning” is a good evidence for it. It misses the most fundamental concept of tranaformers: the are autoregressive. Also, the reinforcement learning is what make the model behave like what you are trying to avoid. So the model output is actually what performs best in the kind of soft…
Universal Claude.md – cut Claude output tokens
41–50 of 192 posts
Re: Universal Claude.md – cut Claude output tokens
#42Things like this make me sad because they make obvious that most people don’t understand a bit about how LLM work. The “answer before reasoning” is a good evidence for it. It misses the most fundamental concept of tranaformers: the are autoregressive. Also, the reinforcement learning is what make the model behave like what you are trying to avoid. So the model output is actually what performs best in the kind of soft…
For complex tasks this is not a useful prompt.
Re: Universal Claude.md – cut Claude output tokens
#43Things like this make me sad because they make obvious that most people don’t understand a bit about how LLM work. The “answer before reasoning” is a good evidence for it. It misses the most fundamental concept of tranaformers: the are autoregressive. Also, the reinforcement learning is what make the model behave like what you are trying to avoid. So the model output is actually what performs best in the kind of soft…
This doesn't stop it from reasoning before answering. This only affects the user-facing output, not the reasoning tokens. It has already reasoned by the time it shows the answer, and it just shows the answer above any explanation.
Re: Universal Claude.md – cut Claude output tokens
#44ChatGPT on the other hand is annoyingly wordy and repetitive, and is always holding out on something that tempts you to send a "OK", "Show me" or something of the sort to get some more. But I can't be bothered with trying to optimize away the cruft as it may affect the thing that it's seriously good at and I really use it for: research and brainstorming things, usually to get a spec that I then pass to Claude to fill out the gaps (there are always multiple) and implement. It's absolutely designed to maximize engagement far more than issue resolution.
Re: Universal Claude.md – cut Claude output tokens
#45Amusing how this industry went from tweaking code for the best results, to tweaking code generators for the best results. There doesn't seem to be any adults left in the room.
Behavior built on top of years and years of experience.
And the problem with AI is that unless you explicitly 'prompt' for certain behavior you're only defining the end result. The inside becomes a black box.
Re: Universal Claude.md – cut Claude output tokens
#46It seems the benchmarks here are heavily biased towards single-shot explanatory tasks, not agentic loops where code is generated: https://github.com/drona23/claude-token-efficient/blob/main/... And I think this raises a really important question. When you're deep into a project that's iterating on a live codebase, does Claude's default verbosity, where it's allowed to expound on why it's doing what it's doing when it…
I wrote a skill called /handoff. Whenever a session is nearing a compaction limit or has served its usefulness, it generates and commits a markdown file explaining everything it did or talked about. It’s called /handoff because you do it before a compaction. (“Isn’t that what compaction is for?” Yes, but those go away. This is like a permanent record of compacted sessions.) I don’t know if it helps maintain long term…
Re: Universal Claude.md – cut Claude output tokens
#47Re- the Unicode chars that are a major PITA when they're used when they shouldn't, there's a problem with Claude Code CLI: there's a mismatch between what the model (say Sonnet) thinks he's outputting (which he's actually is) and what the user sees at the terminal.
I'm pretty sure it's due to the Rube-Goldberg heavy machinery that they decided to use, where they first render the response in a headless browser, then in real-time convert it back to text mode.
I don't know if there's a setting to not have that insane behavior kicking in: it's non-sensical that what the user gets to see is not what the model did output, while at the same time having the model "thinking" the user is getting the proper output.
If you ask to append all it's messages (to the user) to a file, you can see, say, perfectly fine ASCII tables neatly indented in all their ASCII glory and then... Fucked up Unicode monstrosity in the Claude Code CLI terminal. Due to whatever mad conversion that happened automatically: but worse, the model has zero idea these automated conversions are happening.
I don't know if there are options for that but it sure as heck ain't intuitive to find.
And it's really problematic when you need to dig into an issue and actually discuss with "the thing".
Anyway, time for a rant... I'm paying my subscription but overall working with these tools feels like driving at 200 mph on the highway and bumping into the guardrails left and right every second to then, eventually, crash the car into the building where you're supposed to go.
It "works", for some definition of "working".
The number of errors these things confidently make is through the roof. And people believe that having them figure the error themselves for trivial stuff is somehow a sane way to operate.
They're basically saying: "Oh no it's not a problem that it's telling me this error message is because of a dependency mismatch between two libraries while it's actually a logic error, because in the end after x pass where it's going to say it's actually because of that other thing --oh wait no because of that fourth thing-- it'll actually figure out the error and correct it".
"Because it's agentic", so it's oh-so-intelligent.
When it's actually trying the most completely dumbfucktarded things in the most crazy way possible to solve issues.
I won't get started on me pasting a test case showing that the code it wrote is failing for it to answer me: "Oh but that's a behavioral problem, not a logic problem". That thing is distorting words to try to not lose face. It's wild.
I may cancel my subscription and wait two or three more releases for these models and the tooling around them to get better before jumping back in.
Btw if they're so good, why are the tools so sucky: how comes they haven't written yet amazing tooling to deal with all their idiosynchrasies?
We're literally talking about TFA which wrote "Unicode characters that break parsers" (and I've noticed the exact same when trying to debug agentic thinking loops).
That's at the level of mediocrity of output from these tools (or proprietary wrappers around these tools we don't control) that we are atm.
I know, I know: "I'm doing it wrong because I'm not a prompt engineer" and "I'm not agentic enough" and "I don't have enough skills to write skills". But you're only fooling yourself.
Re: Universal Claude.md – cut Claude output tokens
#48Re: Universal Claude.md – cut Claude output tokens
#49I might be wrong but based on the videos I've watched from Karpathy, this would, generally, make the model worse. I'm thinking of the math examples (why can't chatGPT do math?) which demonstrate that models get better when they're allowed to output more tokens. So be aware I guess.
The goal here seems to be removing low-value output; e.g., sycophancy, prompt restatement, formatting noise, etc., which is different than suppressing useful reasoning. In that case shorter outputs do not necessarily mean worse answers.
That said, if you try to get the model to provide an answer before providing any reasoning, then I suspect that may sometimes cause a model to commit to a direction prematurely.
Re: Universal Claude.md – cut Claude output tokens
#50Earlier quoted context omitted.
I was reading through the Claude docs and it was talking about common patterns to preserve context across sessions. One pattern was a "handoff file", which they explained like "have claude save a summary of the current session into a handoff file, start a new session, then tell it to read the file." That sounded like a nice idea, so I made it effortless beyond typing /handoff. The generated docs turned out to be real…
Oh, so the word 'gate' is probably in the documentation also! I see. So this isn't as scary. Claude is helping me understand how to use it properly.