Claude Code daily benchmarks for degradation tracking

Post by chrisjj » Thu, Jan 29, 2026, 5:43 PM UTC

altcognito wrote: post #125

botacode wrote: post #107

Earlier quoted context omitted.

Load just makes LLMs behave less deterministically and likely degrade. See: https://thinkingmachines.ai/blog/defeating-nondeterminism-in... They don't have to be malicious operators in this case. It just happens.

Explain this though. The code is deterministic, even if it relies on pseudo random number generation. It doesn't just happen, someone has to make a conscious decision to force a different code path (or model) if the system is loaded.

Post by seunosewa » Thu, Jan 29, 2026, 5:45 PM UTC

vidarh wrote: post #60

megabless123 wrote: post #52

Earlier quoted context omitted.

noob question: why would increased demand result in decreased intelligence?

It would happen if they quietly decide to serve up more aggressively distilled / quantised / smaller models when under load.

Post by 9cb14c1ec0 » Thu, Jan 29, 2026, 5:46 PM UTC

dajonker wrote: post #21

Wouldn't be surprised if they slowly start quantizing their models over time. Makes it easier to scale and reduce operational cost. Also makes a new release have more impact as it will be more notably "better" than what you've been using the past couple of days/weeks.

Post by chrisjj » Thu, Jan 29, 2026, 5:47 PM UTC

kingstnap wrote: post #119

chrisjj wrote: post #92

Earlier quoted context omitted.

They advertise the Opus 4.5 model. Secretly substituting a cheaper one to save costs would be fraud.

Old school Gemini used to do this. It was super obvious because mid day the model would go from stupid to completely brain dead. I have a screenshot of Google's FAQ on my PC from 2024-09-13 that says this (I took it to post to discord): > How do I know which model Gemini is using in its responses? > We believe in using the right model for the right task. We use various models at hand for specific tasks based on what…

Post by chrisjj » Thu, Jan 29, 2026, 5:55 PM UTC

alias_neo wrote: post #110

observationist wrote: post #62

Earlier quoted context omitted.

>>> We never reduce model quality due to demand, time of day, or server load. The problems our users reported were due to infrastructure bugs alone. Just ignore the continual degradation of service day over day, long after the "infrastructure bugs" have reportedly been solved. Oh, and I've got a bridge in Brooklyn to sell ya, it's a great deal!

> We never reduce model quality due to demand, time of day, or server load Forgive me, but as a native English speaker, this sentence says exactly one thing to me; We _do_ reduce model quality, just not for these listed reasons. If they don't do it, they could put a full stop after the fifth word and save some ~~tokens~~ time.