Earlier quoted context omitted.
Surely you mean 6fps
He doesn't: https://x.com/trq212/status/2014051501786931427
Don't link out to x, its trash
Claude Code daily benchmarks for degradation tracking
281–290 of 372 posts
Re: Claude Code daily benchmarks for degradation tracking
#282Earlier quoted context omitted.
Surely you mean 6fps
He doesn't: https://x.com/trq212/status/2014051501786931427
What? Technology has stopped making sense to me. Drawing a UI with React and rasterizing it to ANSI? Are we competing to see what the least appropriate use of React is? Are they really using React to draw a few boxes of text on screen?
I'm just flabbergasted.
Re: Claude Code daily benchmarks for degradation tracking
#283Earlier quoted context omitted.
Surely you mean 6fps
He doesn't: https://x.com/trq212/status/2014051501786931427
For those who don't want to visit X:
Most people's mental model of Claude Code is that "it's just a TUI" but it should really be closer to "a small game engine".
For each frame our pipeline constructs a scene graph with React then
-> layouts elements
-> rasterizes them to a 2d screen
-> diffs that against the previous screen
-> finally uses the diff to generate ANSI sequences to draw
We have a ~16ms frame budget so we have roughly ~5ms to go from the React scene graph to ANSI written.Re: Claude Code daily benchmarks for degradation tracking
#284Hi everyone, Thariq from the Claude Code team here. Thanks for reporting this. We fixed a Claude Code harness issue that was introduced on 1/26. This was rolled back on 1/28 as soon as we found it. Run `claude update` to make sure you're on the latest version.
Anywhere we can read more about what a "harness issue" means? What was the impact of it?
Pretty sure they mean the issue is on the agentic loop and related tool calling, not on the model itself
In other words, it was the Claude Code _app_ that was busted
Re: Claude Code daily benchmarks for degradation tracking
#285Totally tangential to article, was browsing through the website UI - https://marginlab.ai/explorers/swe-bench-pro/ , the page gives impression that the language, category boxes are selectable. However they are not a dropdown. Not sure if it was intentional design by human or some smart code generation by Claude based on the design sketches.
Re: Claude Code daily benchmarks for degradation tracking
#286Earlier quoted context omitted.
He doesn't: https://x.com/trq212/status/2014051501786931427
For those who don't want to visit X: Most people's mental model of Claude Code is that "it's just a TUI" but it should really be closer to "a small game engine". For each frame our pipeline constructs a scene graph with React then -> layouts elements -> rasterizes them to a 2d screen -> diffs that against the previous screen -> finally uses the diff to generate ANSI sequences to draw We have a ~16ms frame budget so w…
Implementation details aside (React??), that sounds exactly like “just a TUI”…
Re: Claude Code daily benchmarks for degradation tracking
#287Earlier quoted context omitted.
He doesn't: https://x.com/trq212/status/2014051501786931427
What? Technology has stopped making sense to me. Drawing a UI with React and rasterizing it to ANSI? Are we competing to see what the least appropriate use of React is? Are they really using React to draw a few boxes of text on screen? I'm just flabbergasted.
There is more than meets the eye for sure. I recently compared a popular TUI library in Go (Bubble Tea) to the most popular Rust library (Ratatui). They use significantly different approaches for rendering. From what I can tell, neither is insane. I haven’t looked to see what Claude Code uses.
Re: Claude Code daily benchmarks for degradation tracking
#288Hi everyone, Thariq from the Claude Code team here. Thanks for reporting this. We fixed a Claude Code harness issue that was introduced on 1/26. This was rolled back on 1/28 as soon as we found it. Run `claude update` to make sure you're on the latest version.
Is there compensation for the tokens because Claude wasted all of them?
It is possible that degradation is an unconscious emergent phenomenon that arises from financial incentives, rather than a purposeful degradation to reduce costs.
Re: Claude Code daily benchmarks for degradation tracking
#289Earlier quoted context omitted.
He doesn't: https://x.com/trq212/status/2014051501786931427
For those who don't want to visit X: Most people's mental model of Claude Code is that "it's just a TUI" but it should really be closer to "a small game engine". For each frame our pipeline constructs a scene graph with React then -> layouts elements -> rasterizes them to a 2d screen -> diffs that against the previous screen -> finally uses the diff to generate ANSI sequences to draw We have a ~16ms frame budget so w…
Kudos to them for figuring out how to complicate what should have been simple.
Re: Claude Code daily benchmarks for degradation tracking
#290Earlier quoted context omitted.
He doesn't: https://x.com/trq212/status/2014051501786931427
For those who don't want to visit X: Most people's mental model of Claude Code is that "it's just a TUI" but it should really be closer to "a small game engine". For each frame our pipeline constructs a scene graph with React then -> layouts elements -> rasterizes them to a 2d screen -> diffs that against the previous screen -> finally uses the diff to generate ANSI sequences to draw We have a ~16ms frame budget so w…
How ridiculous is it that instead of a command line binary it's a terminal emulator, with react of all things!