I'm becoming AI-blind

I have basically polluted my context and agent files, agent.md, claude.md, gemini.md with really strong guidance about using plain language. I even include small, mini design specifications, examples of plain engineering language, only using technical language and invented terms when they actually make sense (almost never). EVERY TURN.

I have a pretty large set of prompts that go into any software engineering, and I force every single agent to use an ephemeral style stack of prompt management. So, every turn it goes to the top of the stack and it is the very last thing they see in terms of all of my prompts and instructions and agent files. And then it gets taken out of the conversation so that it doesn't get sent to the agent the next turn (no context bloat). It has restored so much sanity.

I tried the caveman add-ons, and I felt like I was losing IQ points because I spend a lot of time reading agent output, and when they start talking like cavemen, I start thinking like cavemen. That was not good for my mental health. So, I try and make the agent talk like me and think like me. And it works, mostly. And my observation is that maybe I'm not the most efficient agentic thought process, but my sanity is retained.

All of that is to say that if something is reading like that to you, just have the agent rewrite it and read it in a rewritten tone because it's probably bad as it stands and your colleague did not put enough effort in it. It is /not/ good and you should not accept it as a default. We have to hold the line on stuff like this and maintain some semblence of normal human engineering standards that existed before AI. They are not making us better. They are making is lazy and dumber.

Opus 5 and other agents in the latest rounds of tuning have gotten ridiculously bad in terms of how they feel to interact with with all the invented language and localized nomenclature. It is an obvious bias that big words and technical talk looks good to the bottom of the bell curve, but when you actually try and understand it, it's horrible. So people say, "Yeah, that looks great," in all the RLHF rounds, and they run with it because they think it looks good, but it doesn't. It's terrible.

Hold the line. It isn't you. And it isn't a good methodology document.

This is actually funny because I've been doing some RAG experiments and I've basically trained up a RAG on several gigabytes of philosophy and neuroscience. And when I want to go metacognitive about thought processes, I actually really enjoy exploring different areas of philosophy, more life philosophy, but how it intersects with psychology and neuroscience and observing the observer, or the observer is the observed, as they say.

And when you ground it with real data, it's actually extremely useful. It's not exactly like Claude Therapist, but it's sort of the teach me about philosophy, but actually grounded and not vied. I have a lot of really strict prompts and grounding and agentic guidelines for this particular agent flow and harness that I've built.

And it's just a few weekends of vibing and feeding it basically all of Wikipedia and several gigabytes of papers and stuff, but it actually leads to interesting discussion. So I just have my personal philosophy bot and it's pretty fun.

One of the modalities I built is having two agents assume a famous persona. And then they take a thing, like grief or some thing that I experienced during the week, and they assume the role of the two different philosophers, and I just have them go back and forth 30, 40, 50 turns. And it's actually quite interesting, and it really moderates their language and tonality and behavior. They really get into the roles when you have the right prompting and grounding. Sometimes they get a little off the rails, but it leads to genuinely interesting areas to explore, and then I'll actually go read source material and things like that. I don't know, that's how I do therapy these days, but I never actually did therapy, so I just think a lot, now with agents finding interesting stuff to think about too!

"nd then it gets taken out of the conversation so that it doesn't get sent to the agent the next turn (no context bloat). "

Uh - dude - this means you're paying 10x in token costs because there's no caching.

If you 're-write token history' then you can't cache tokens.

It means for any reasonably long conversation, the llm has to reprocess the entire history as preflow on every prompt.

Are you sure you're really doing what you say you're dong, and how is it not blowing up your budget?

Nope, you gotta understand. This is the move. It is appended at the end of every single message. It is never saved in the conversation that get sent back for inference. So you send it. But when you go back for inference, it’s at the top of the stack so all of the cashing works you’re not pre-filling every time or anything like that. It burns plus N tokens, where N is my prompt stack. It is not really that expensive. I have measured it to within an inch of it its life. Think of it this way every bit of the prefix and the conversation stays exactly the same you’re only adding to the very end of the conversation. So after the first turn, it is basically always cashing within the KV cache for a given context. Sorry I am using voice dictation. My hands are tired this week. Basically you sculpt the conversation history to ensure prefix caching

Ok, I want to thank you for finally giving us a concrete falsifiable statement that we can check. I pretended Marseille 40 times before asking Luna 5.6, and the answer was Paris.

So, even with concrete examples, model haters are still wrong.

You also imply the claim that making the distribution of words as the possible next one visible, somehow makes the whole system not intelligent. I would say the exact opposite is true.

By using the embedding vectors, models are aware of precise placement and relative position of words in this hugely dimensional space. No human is capable of such precision. This enables party tricks of "king plus woman minus man" kind. But this also give us a precise point between any two words, no matter how different. What is on the midpoint between volcano and music, for example. No human can precisely answer that, but an embedding can. And we can see which words are closest to this 700 dimensional point.

You see this menu of words as a weakness, and I say it is in fact a sign of super intelligence. And this is all before any reasoning or attention mechanism is even run.