distillation is an equalizing force
Distillation doesn't give you an equivalent model.
The beginning of scarcity in AI
151–160 of 239 posts
Re: The beginning of scarcity in AI
#152We just had a realization during a demo call the other day: The companies that are entirely AI-dependent may need to raise prices dramatically as AI prices go up. Not being dependent on LLMs for your fundamental product’s value will be a major advantage, at least in pricing.
The companies that are entirely AI-dependent may need to raise prices dramatically as AI prices go up
Or they'll price the true cost in from the start, and make massive profits until the VC subsidies end... I know which one I'd do.
Re: The beginning of scarcity in AI
#153This isn’t really looking like AI scarcity it’s more like compute becoming the bottleneck : when the access depends on chips energy and capital it stops being a pure software game and the winners are often whoever can secure capacity first
Re: The beginning of scarcity in AI
#154We just had a realization during a demo call the other day: The companies that are entirely AI-dependent may need to raise prices dramatically as AI prices go up. Not being dependent on LLMs for your fundamental product’s value will be a major advantage, at least in pricing.
> Not being dependent on LLMs for your fundamental product’s value
I think more specifically not being dependent on someone else's LLM hardware. IMO having OSS models on dedicated hardware could still be plenty viable for many businesses, granted it'll be some time before future OSS reaches today's SOTA models in performance.
Re: The beginning of scarcity in AI
#155This notion that "we don't have enough compute" does not cleanly reconcile with the fact that labs are burning cash faster than any cohort of companies in history. If I am a grocery store that pays $1 for oranges and sells them for $0.50, I can't say, "I don't have enough oranges."
The grocery store analogy works if compute is the orange.
But labs arent buying oranges — theyre buying the only orchard on the island, hoping it yields a fruit no ones grown yet. Burning $1B to net $500M isnt "I have too few oranges." Its "Im betting the farm Ill find a new one."
Both can be irrational. Theyre irrational in different ways.
Re: The beginning of scarcity in AI
#156one graph, One graph and the author is pinning an entire theory on it? Infra is always limited, even at hyper scalers. This leads to a bunch of tools dfofr caching, profiling and generally getting performance up, not to mention binpacking and all sorts of other "obvious" things.
> Infra is always limited, even at hyper scalers
I think maybe infra is limited only at hyperscalers. For the rest of us it's just how much capacity to we want to rent from the hyperscalars.
It's kind of a recent cloud-native mindset, since back in the day when you ran your own hardware scaling and capacity was always top of mind. Looks like AI compute might be like that again, for the time being.
Re: The beginning of scarcity in AI
#157We just had a realization during a demo call the other day: The companies that are entirely AI-dependent may need to raise prices dramatically as AI prices go up. Not being dependent on LLMs for your fundamental product’s value will be a major advantage, at least in pricing.
That'll be (part of) the big market correction, but also speaking broadly; as investor money dries up and said investors want to see results, many new businesses or products will realise they're not financially viable.
On a small scale that's a tragedy, but there's plenty of analysts that predict an economic crash and recession because there's trillions invested in this technology.
Re: The beginning of scarcity in AI
#158We just had a realization during a demo call the other day: The companies that are entirely AI-dependent may need to raise prices dramatically as AI prices go up. Not being dependent on LLMs for your fundamental product’s value will be a major advantage, at least in pricing.
I wonder if it could be that they won't because the real mechanism is that AI wrapper pricing power is weak (switching costs near zero) but state of the art models makes it difficult to lower prices due to higher cost.
Re: The beginning of scarcity in AI
#159Earlier quoted context omitted.
Yup. Also regardless of price they need to spend more and more as the project collapses under the inevitable incidental complexity of 30k lines of code a day. It's similar to how if you know what you're doing you can manage a simple VPS and scale a lot more cost effectively than something like vercel. In a saturated market margins are everything. You can't necessarily afford to be giving all your margins to anthropic…
I also can’t wait for the time when few know how to code. Just like how many folks don’t know html from css when the homebrew website went away. Their might always be llms, but the dependence is an interesting topic.
Look no further to be honest; look at older generation programming languages like COBOL and how sought-after good developers for that language are.
But I'm also afraid / certain that LLMs are able to figure out legacy code (as long as enough fits in their context window), so it's tenuous at best.
Also, funny you mentioned HTML / CSS because for a while (...in the 90's / 2000's) it looked like nobody needed to actually learn those because of tools like Dreamweaver / Frontpage.
Re: The beginning of scarcity in AI
#160We just had a realization during a demo call the other day: The companies that are entirely AI-dependent may need to raise prices dramatically as AI prices go up. Not being dependent on LLMs for your fundamental product’s value will be a major advantage, at least in pricing.
> The companies that are entirely AI-dependent may need to raise prices dramatically as AI prices go up. It's not that clear. Sure, hardware prices are going up due to the extremely tight supply, but AI models are also improving quickly to the point where a cheap mid-level model today does what the frontier model did a year ago. For the very largest models, I think the latter effect dominates quite easily.
We are processing same data for the last 2 years.
Inference prices droped like 90 percent in that time (a combination of cheaper models, implicit caching, service levels, different providers and other optimizations).
Quality went up. Quantity of results went up. Speed went up.
Service level that we provide to our clients went up massively and justfied better deals. Headcount went down.
What's not to like?