Don't trust large context windows • Hacker News Board

Re: Don't trust large context windows

#181

Post by nk91 » Sun, Jun 14, 2026, 5:51 PM UTC

port11 wrote: post #173

nk91 wrote: post #159

Earlier quoted context omitted.

I felt the same way about the non-determinism but realized it can be really beneficial to have a machine that can fairly reliably turn non-determinism into determinism. I’m working on a tiny agent harness at home to learn and the process of taking human speech and turning it into agent tool calls that output something generally deterministic depending on how the tool is defined is so interesting. One of the big takea…

That sounds very cool. It’s sometimes baffling that LLMs can’t use tools reliably. Serena and Semble both require some arcane instructions to coerce Claude Code into compliance. Just stop trying to pipe nonsense commands into each other, man!

I think it makes sense when you dig into why that non-determinism conversion is so hard.

For voice related things you have a lot of turn of phrase scenarios that can make no sense unless you know. Phrasing like “Put Larry on the horn.” For someone familiar with old lingo for phone calls makes sense. For someone else they might think of a war horn, someone else a music class.

All of those are wildly different situations. It’s not hard to see how one oops between two non deterministic things can quickly go off the rails.

The fact we can get away with so much non-determinism->non-determinism recursion is frankly amazing when you realize how easy it is to imprecisely describe what it is you’re thinking.

Re: Don't trust large context windows

#182

Post by ACCount37 » Sun, Jun 14, 2026, 5:53 PM UTC

dofm wrote: post #133

api wrote: post #131

Earlier quoted context omitted.

Programming has already become this way. Opinions about different languages and architectures are taste, or sometimes even just vibes. Few try to actually ask “can I quantify whether microservices or monoliths are better in terms of either maintainability or scaling?” A lot of this is a result of systems having long ago exceeded the complexity threshold of things people can hold in their heads. There are too many lay…

"Feeping creaturism" has always been a problem, for sure. But those technologies are layers, and there are reliable things that sometimes bubble across the boundaries — type hints, better code patterns to trigger compiler optimisation, interesting tricks with key column selection — and someone with expertise from that layer below can explain why, and their advice will always work in situations that are sufficiently s…

They're almost literally "from different cultures" - because of how post-training does things.

All "personality traits" within an LLM are entangled. So when you mid-train or post-train on ESL texts, or run RLHF using people from a given culture, you risk bleeding some of the related cultural traits into the LLM itself. A lot of the resulting "personality" is downstream from different AI teams picking different datasets and training signals.

RLAF is more of a "funhouse mirror" distortion - it takes existing traits and twists them, sometimes amplifies them to comical extremes. Weird can become weirder. A verbal tic can become a style signature. Part of the reason why AI writing from GPT-4 era and to now has changed so dramatically.

Re: Don't trust large context windows

#183

Post by port11 » Sun, Jun 14, 2026, 6:16 PM UTC

nk91 wrote: post #181

port11 wrote: post #173

Earlier quoted context omitted.

That sounds very cool. It’s sometimes baffling that LLMs can’t use tools reliably. Serena and Semble both require some arcane instructions to coerce Claude Code into compliance. Just stop trying to pipe nonsense commands into each other, man!

I think it makes sense when you dig into why that non-determinism conversion is so hard. For voice related things you have a lot of turn of phrase scenarios that can make no sense unless you know. Phrasing like “Put Larry on the horn.” For someone familiar with old lingo for phone calls makes sense. For someone else they might think of a war horn, someone else a music class. All of those are wildly different situatio…

The vagary of speech and its meaning is surely hard to parse. But! How many ways must a model invent to run `tsc`?

    npx tsc
    bash tsc
    bash npx tsc
    npm run build
    …

I’m not an expert at all on the subject matter, but is it impossible to train a model that calls tools in a (quasi-)deterministic way?

Re: Don't trust large context windows

#184

Post by cubefox » Sun, Jun 14, 2026, 6:18 PM UTC

Bolwin wrote: post #146

cubefox wrote: post #124

The problem with "context rot" is that its existence and severity is purely anecdotal. As far as I know, nobody has actually measured context rot systematically. The only thing we know is that memory degrades somewhat in long contexts, via things like needle in haystack tests. But that's not the same issue. Context rot is usually taken to mean that the model gets dumber even if it doesn't need to remember specific th…

It's pretty hard to measure because most context rot comes from related context and the model has to be able to figure which parts are truly relevant, which ones are relevant but stale, which ones to ignore etc. Each relevant thing is basically a rule. Trying to so something with 500 rules is what's hard. If you take a standard benchmark and just prepend a random book to it, it will not capture that

Would be still interesting whether it degraded the performance in that case. Further, many non-agentic benchmarks consist of many short tasks, so one could fill the context with task/response pairs from other tasks (like in a standard chat environment) and then ask the current task at the end. Given that the tasks are probably somewhat similar, context rot should occur.

Re: Don't trust large context windows

#185

Post by iugtmkbdfil834 » Sun, Jun 14, 2026, 6:19 PM UTC

topspin wrote: post #175

dofm wrote: post #123

I guess I am mostly enjoying learning the fundamentals of AI stuff, even though I disagree with the direction it is going. But I am struggling to put into words how alarming I find the comments on threads like this — all sorts of good-natured anecdotes about how XYZ works for them that are more like the suggestions in pet care or cookery threads on Facebook. (Or worse still, like any Facebook 3D printing group: anyon…

> Any shared sense of rigour is just completely torpedoed by the LLM world Consider that this shared sense of rigour you have in mind is illusory, and LLMs and their context struggles are simply revealing this. I see precious little rigour in any of the 'tech' world I've lived in for decades. The tools proliferate, paradigms emerge and die and reemerge, and whatever stick you consider using to measure any of it has c…

Yep, if anything LLMs revealed how little rigour there was to begin with. If you want a more obvious example: think of documentation..

Re: Don't trust large context windows

#186

Post by HarHarVeryFunny » Sun, Jun 14, 2026, 6:57 PM UTC

kelnos wrote: post #8

This has not been my experience with Opus since Anthropic released the 1M token context window for use under the subscription plans. I routinely push past 500k tokens, even sometimes up to around 800k tokens, and don't see this problem. I've seen it to some extent when getting truly near the limit, up around and above 900k tokens, though what I see isn't as severe as the author seems to see. (And I rarely fill the co…

Not everybody is using the same model and harness as you, nor using the model the same way as you.

Different models, and versions of models, use different types of attention, which affects their long-context performance, and no doubt also do different amounts/types of long context training.

Different agents build context differently and implement context compaction differently.

Unless someone else is using the same model as you, the same agent/harness as you, and doing very similar tasks, then there is no reason to suppose that their experience of model behavior relating to context size is going to be the same as yours.

Re: Don't trust large context windows

#187

Post by kelnos » Sun, Jun 14, 2026, 6:59 PM UTC

HarHarVeryFunny wrote: post #186

kelnos wrote: post #8

This has not been my experience with Opus since Anthropic released the 1M token context window for use under the subscription plans. I routinely push past 500k tokens, even sometimes up to around 800k tokens, and don't see this problem. I've seen it to some extent when getting truly near the limit, up around and above 900k tokens, though what I see isn't as severe as the author seems to see. (And I rarely fill the co…

Not everybody is using the same model and harness as you, nor using the model the same way as you. Different models, and versions of models, use different types of attention, which affects their long-context performance, and no doubt also do different amounts/types of long context training. Different agents build context differently and implement context compaction differently. Unless someone else is using the same m…

> then there is no reason to suppose that their experience of model behavior relating to context size is going to be the same as yours.

Relax, I acknowledged this in my comment...

Re: Don't trust large context windows

#188

Post by data-ottawa » Sun, Jun 14, 2026, 7:25 PM UTC

tgv wrote: post #172

dofm wrote: post #123

I guess I am mostly enjoying learning the fundamentals of AI stuff, even though I disagree with the direction it is going. But I am struggling to put into words how alarming I find the comments on threads like this — all sorts of good-natured anecdotes about how XYZ works for them that are more like the suggestions in pet care or cookery threads on Facebook. (Or worse still, like any Facebook 3D printing group: anyon…

If you want my best guess: I think large context windows cannot be trained properly. There's not enough material, nor computing power, to train such large networks (to the same degree as small windows).

I feel this is a sort of inverse inspection paradox (the paradox that if you sample waiting time in a process, you’re more likely to sample a larger value).

The LLM providers fine tune the models with some kind of information retrieval tasks, but to do so you must provide some non relevant context to bootstrap the session for the long context tasks.

It would be very easy to do this in ways that train the sequence model to treat early history as noisier than it really is, or to weaken its relationship to late context.

You’re also probably stacking more contexts together with long contexts (start with task A, then detour to solving B and C before you can complete A).

Training sequence lengths probably decay super linearly with length creating far fewer samples at long length during training.

Re: Don't trust large context windows

#189

Post by KronisLV » Sun, Jun 14, 2026, 7:33 PM UTC

wrs wrote: post #168

KronisLV wrote: post #81

Earlier quoted context omitted.

For anyone using Claude Code, ask it to do all the work in workflows (it has a tool for that), they released that feature together with Opus 4.8 and it also seems a bit better at doing long tasks as well. The main conversation just orchestrates the work at that point.

You can also just ask it to do work in a subagent. It will write a plan and launch the subagent to do the actual code, keeping it out of the main context. In addition, you can co-author a plan for a biggish chunk of work, divided into stages, have it launch a subagent for phase 1 and check its work, then ESC-ESC to go back to just after you wrote the plan and have it do phase 2. Repeat until done. This keeps the over…

My problem with regular sub-agents is that after around 2-4 hours the main agent stops working on a task and asks for user input no matter how I tell it to continue autonomously until the ~5-15 stage plan is done, when when it has a clear plan that's made with the plan mode and instructions to continue autonomously.

It's happened multiple times where I give it a task before going to sleep and when I come back it's stuck halfway through on some stupid summary, where my only response needed is basically "Yeah, continue." even though I use Opus. Using workflows for the higher level planning helped with that and those annoying pauses no longer happen, perhaps due to the main conversation being much shorter and apparently not enough for the weights to nudge towards user confirmation.

Re: Don't trust large context windows

#190

Post by foobarchu » Sun, Jun 14, 2026, 8:14 PM UTC

cactusplant7374 wrote: post #125

dofm wrote: post #123

I guess I am mostly enjoying learning the fundamentals of AI stuff, even though I disagree with the direction it is going. But I am struggling to put into words how alarming I find the comments on threads like this — all sorts of good-natured anecdotes about how XYZ works for them that are more like the suggestions in pet care or cookery threads on Facebook. (Or worse still, like any Facebook 3D printing group: anyon…

> But I am struggling to put into words how alarming I find the comments on threads like this — all sorts of good-natured anecdotes about how XYZ works for them that are more like the suggestions in pet care or cookery threads on Facebook. It will always be this way going forward. Everyone thinks differently about problems. In the past we had experts and only they could do the work at a high level. But now we have ma…

> You can't fight reality.

Defeatism doesn't do anything positive for the world. You're trying to convince the people pushing for a marginally better world that they should give up because it'll never happen. That is not a useful contribution.