I think this is a minor speed bump and VC’s believe that cost of inference will decrease over time and this is a gold rush to grab market share while cost of inference declines. I don’t think they got it right and the market share and usage grew faster than inference dropped, but inference costs will clearly drop and these companies will eventually be very profitable. Reality is that startups like this assume moore’s…
My own usage and the usage of pretty much everyone I know says that as inference costs drop, usage goes up in lockstep, and I’m still nowhere near the ceiling of how many tokens I could use if they were free. I think if these companies are gambling their future on COGS going down, that’s a gamble they’re going to lose.
we strive to make machines more efficient to reduce energy usage. That makes $ per unit of work lower. But instead of reducing spending, people find more uses for the machines.