Earlier quoted context omitted.
One aspect Paul Kedrosky mentioned recently is the concept of „duration mismatch“. The price per token goes down over time (either because the AI vendor reduces due to competition pressure, or because customers are now incentivized to use older cheaper models). But datacenters are financed through debt, with the assumption their revenue increases over time. Quoting him: „[AI vendors are] paying for a fixed cost with…
Using a shittier model is just more work for the user, I’m not sure why anyone does it, unless they’re playing with it like a toy.
Saves like $2-3 per session. Same quality code.