Flowers for dry Claude: Memes are better sensors than benchmarks

Flowers for dry Claude: Memes are better sensors than benchmarks
nickoak.com

Been running a number theory research project on Claude Code for 4+ months - Gaussian moat problem, CUDA + Rust solvers, multi-agent swarms, the whole stack. When you beat the same problem with the same model for that long - you inevitably start noticing things. Small variations. Method drift. Subtle loss of architectural judgment.

March 20-22 was different. Complete collapse. Method I specced cleanly got reimplemented as something else entirely. "But why?" became the recurring horror question. An Opus session tried to rm `-rf /tmp/private`. Simple document work - files in wrong locations, wrong frontmatter. Parallel sessions degrading simultaneously. Not rate limits - capability.

The core questions I'm sitting with: - The "wet/dry Claude" meme carries signal that no benchmark currently captures. What if vibes are a precursor to a new class of measurements? - Quantization, MoE routing, KV cache policy - how much of the subtle stuff (implicit intent, multi-step coherence, architectural taste) vanishes at FP8/INT6 while headline evals barely move? - I built digital twins for 12 AI companies to model inference economics. The incentive to silently serve degraded models is structural. How do we even detect this as users? - Is there a path to a "Vibes Bench" - benchmarks for the agentic qualities that emerged with late Opus 4.5 and made thousands of practitioners suddenly productive?

After writing this I found marginlab.ai's Claude Code tracker (https://marginlab.ai/trackers/claude-code/) - independently confirms the quality drop over the same period. So at least I haven't gone bananas from doing too much math.

Curious if others have hit the wall - and especially if anyone has tried to measure the unmeasurable.