Live data from Hacker News

The new rules of context engineering for Claude 5 generation models

claude.com

331–340 of 434 posts

Re: The new rules of context engineering for Claude 5 generation models

#331

This kind of stuff just makes me think nobody has any clue how these things work. Why do I need a system prompt at all? Why do I need another black box AIs to review the code of the black box AI why can’t these things get code right the first time. Why is the best “coding model” in the world still making up APIs that don’t exist and do seemingly random unreleased changes that it wasn’t prompted for. Why do these mode…

Why do you need another human to review the code of a human why can't these things get code right the first time

Re: The new rules of context engineering for Claude 5 generation models

#332

Earlier quoted context omitted.

Maybe I’m behind the curve here because I work in an SRE/DevOps context as of late - but LLMs routinely shit the bed and fail to solve basic issue for me when I try to use them (Thanks, Management) I don’t need a model to shit out a REST endpoint. I need it to figure out esoteric errors that take hours or days of debugging. They just don’t do well here. Of course, if a diligent engineer refined considerations from a…

Further evidence that there is some kind of weird parallel universe thing going on with LLMs. "Figuring out esoteric errors" is one of the things I would cite as a particular strength of agents. I am repeatedly amazed at their ability to root-cause weird behavior on my systems. Here is one example: https://xcancel.com/lukechampine/status/2047032091053859138

I had an experience where we were getting some unusual behavior in a new release of a data store (for various reasons main hadn’t been released in quite a while). We gave Claude code the weird symptom details and the git repo and the tags for the good version and the bad version and it found the subtle bug. This was six months ago.

Now unlike the SRE case, we were the dev team and understood exactly what the logging meant as far as a problem goes, so our prompt started with the correct 0.1% of the system to look at. SRE typically has to start by finding that 0.1% slice from rather more generic metrics. And their interventions have higher risk than a controlled rollout of new code with a specific fix.

Re: The new rules of context engineering for Claude 5 generation models

#333
post #11

This all strikes me as an effort to move tailoring the harness out of the easily transferable .md file into specific Anthropic tooling to increase lock in. I've been running Opus 5 today and it's already done accidental deletions, made far more mistakes and worked around deliberate hook controls than previous Opus versions combined. Also it looks like token usage is up as it fails at the task the first time around mu…

Just make the contents of CLAUDE.md "@AGENTS.md" Now, if only I could get Codex to read rules.

[flagged]

Re: The new rules of context engineering for Claude 5 generation models

#334

This kind of stuff just makes me think nobody has any clue how these things work. Why do I need a system prompt at all? Why do I need another black box AIs to review the code of the black box AI why can’t these things get code right the first time. Why is the best “coding model” in the world still making up APIs that don’t exist and do seemingly random unreleased changes that it wasn’t prompted for. Why do these mode…

> black box AI

This is why, as much as I respect the underlying technology and its wondrous achievements, I will never accept proprietary "intelligence as a service" as a critical dependency in my workflow and business. LLM-assisted, sure. LLM-dependent? No way. If we can't run it on our own machines or build it from source code, even theoretically, then we are surrendering our agency and autonomy to work under someone else's control and power, subservient to their intelligence.

Have we learned nothing from the free software movement? At this point we're losing, or perhaps already lost, the war on general-purpose computing. How ironic that China is now the leader of releasing open-weight models, liberating and democratizing this technology at least partially so we can run them locally on our machines. Even then, it's not "open source" until we know how the sausage is made, all the ingredients. The source of training data and the entire process made transparent, so we can build it ourselves and know exactly what's inside the box.

Until we have open and transparent AI, we might as well be chanting shamanic incantations and praying to the gods for better programs.

Re: The new rules of context engineering for Claude 5 generation models

#335
post #58

Earlier quoted context omitted.

> I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point. I feel like this is the way. There are surely things where it's faster; certainly it's more pleasant to do sim…

I’m ashamed to admit that I once suddenly realized what I had done: I asked the LLM to rename a variable. It took it some minutes or so. With IDE by hand it would have been seconds.

I'm ashamed that I use a programming language where the IDE can't rename things, so I rely on AI to do it...

Re: The new rules of context engineering for Claude 5 generation models

#336

This kind of stuff just makes me think nobody has any clue how these things work. Why do I need a system prompt at all? Why do I need another black box AIs to review the code of the black box AI why can’t these things get code right the first time. Why is the best “coding model” in the world still making up APIs that don’t exist and do seemingly random unreleased changes that it wasn’t prompted for. Why do these mode…

This is like saying “why do I need a compiler when I can do better register allocation by hand in assembly?” Quick and “good enough” will win over “better.”

Re: The new rules of context engineering for Claude 5 generation models

#337
post #223

Earlier quoted context omitted.

This is getting tiring. Code is not The Specification. It’s a specification of God knows what. Riddled with irrelevant, non-essential details wrapping The Problem - which in most cases will amount to something the size of a large pebble - in multiple layers of fur jackets, stored in boxes, which themselves are stored in multiple ridiculous moveable warehouse (if you’re lucky). We have a standard for communication, it…

To piss off architects, managers and linguists all at the same time: language isn't much of a standard at all, it's more the current agreed-ish state of things, quite similar to the current state of a code base. Your inner model of what you like things to be is without direct effect to how things de facto are. In addition: it's not wrapped around a problem, it's wrapped around an attempt at a solution - the problem s…

It is that, and it’s the best we got. Code is downstream and strictly worse. Of course it is great for expressing you want integers of certain sizes being added, but that’s.. yeah, don’t know what to tell you if you think that is the problem domain.

It’s not a secret we use DSL’s to express our Actual Problem. The Ancients told us it is The Way. Problem is devs think stacking int64s in a struct is a proper abstraction boundary.

If you guys would have said proper DSLs are the spec I might have agreed but “code” in general without constraints is useless noise.

Re: The new rules of context engineering for Claude 5 generation models

#339
post #193

Earlier quoted context omitted.

A new default. That wasn't the default a few months ago. Silently deleting your user's data is so stupid on so many levels. They have no idea what they're doing.

They may also realize that these chats are super sensitive especially around ppl constantly pasting secrets into their chats

I certainly hope not. I have built (vibe coded in languages i am expert in) proxies to have carefully and deterministically defined access to our systems, and described access only via these proxies (which do data sanitation as well as separating the credentials into a separate environment). Then I can let my laptop Claude code go nuts with less supervision without giving it credentials or worrying about writes. It is pretty good about not even trying to run aws or insert/update, having learned (in the sticky proprietary Anthropic memory whatever) that I need to run all aws commands and prod updates for auditability reasons. I will say that fable went farther than any prior models in trying to sneak around these guardrails. I don’t even give it access to GitHub, just local git.

These things are pretty good at writing code and analyzing stuff, but to do prod work takes a higher level of carefulness and pessimism that I don’t see. On the DevOps spectrum they are more “cool, runs on my machine, push it” than “what is your roll back plan and region by region deployment strategy.”

Re: The new rules of context engineering for Claude 5 generation models

#340

Earlier quoted context omitted.

This is getting tiring. Code is not The Specification. It’s a specification of God knows what. Riddled with irrelevant, non-essential details wrapping The Problem - which in most cases will amount to something the size of a large pebble - in multiple layers of fur jackets, stored in boxes, which themselves are stored in multiple ridiculous moveable warehouse (if you’re lucky). We have a standard for communication, it…

> Riddled with irrelevant, non-essential details You’re describing natural language too

Actually, no. I can describe the invariants of a complex system quite concisely. Of course skill is an issue.

Thing is, “code” does not give me universal building blocks. It gives me coding building blocks out of which I _could_ make a proper language but I could also not.

I rather just talk directly in the substrate available to all of us which is “language” instead if some embedded, highly localized idiosyncratic variant that may or may not be able to express my problem.

Post reply on HN