Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The beginning of scarcity in AI

tomtunguz.com

Post reply on HN
Print view View original

141–150 of 239 posts

Re: The beginning of scarcity in AI

#141

Post by classified »

wg0 wrote: post #45

There's other side to it too. Whoever running and selling their own models with inference is invested into the last dime available in the market. Those valuations are already ridiculously high be it Anthropic or OpenAI to the tune of couple of trillion dollars easily if combind. All that investment is seeking return. Correct me if I'm wrong. Developers and software companies are the only serious users because they (m…

> would resort to writing code by hand because that's a very skilled and hardworking tribe they've been doing this all their lives

Shush, don't tell that to the AI coding acolytes.

Re: The beginning of scarcity in AI

#142

Post by zozbot234 »

anonyfox wrote: post #123
keiferski wrote: post #89

We just had a realization during a demo call the other day: The companies that are entirely AI-dependent may need to raise prices dramatically as AI prices go up. Not being dependent on LLMs for your fundamental product’s value will be a major advantage, at least in pricing.

in fact I am betting opposite. frontier models are getting not THAT much better anymore at all, for common business needs at least. but the OSS models keep closing the gap. which means if trajectories hold there will be a near future moment probably where the big provider costs suddenly drop shaerply once the first viable local models consistently can take over tasks normally on reasonable hardware. Right now probabl…

Local open inference can address hardware scarcity by repurposing the existing hardware that users need anyway for their other purposes. But since that hardware is a lot weaker than a proper datacenter setup, it will mostly be useful for running non-time-critical inference as a batch task.

Many users will also seek to go local as insurance against rug pulls from the proprietary models side (We're not quite sure if the third-party inference market will grow enough to provide robust competition), but ultimately if you want to make good utilization of your hardware as a single user you'll also be pushed towards mostly running long batch tasks, not realtime chat (except tiny models) or human-assisted coding.

Re: The beginning of scarcity in AI

#143

Post by aldanor »

dmazin wrote: post #14

Constraints can lead to innovation. Just two things that I think will get dramatically better now that companies have incentive to focus on them: * harness design * small models (both local and not) I think there is tremendous low hanging fruit in both areas still.

Yep.

As a recent example in AI space itself. China had scarce GPU resources, quite obvious why => DeepSeek training team had to invent some wheels and jump through some hoops => some of those methods have since become 'industry standard' and adopted by western labs who are now jumping through the same hoops despite enjoying massive computeresources, for the sake of added efficiency.

Re: The beginning of scarcity in AI

#144

Post by sidewndr46 »

keiferski wrote: post #89

We just had a realization during a demo call the other day: The companies that are entirely AI-dependent may need to raise prices dramatically as AI prices go up. Not being dependent on LLMs for your fundamental product’s value will be a major advantage, at least in pricing.

Not really, the next move is to establish standards groups requiring the use of AI in product development. A mix of industry and governmental mandates. What you view are viewing as COGS instead becomes instead a barrier to entry.

Re: The beginning of scarcity in AI

#145

Post by sevenzero »

keiferski wrote: post #137
sevenzero wrote: post #134

Earlier quoted context omitted.

This was as clear as the sky when the first llm based businesses popped up. How did you realize this only now?

Replied here: https://news.ycombinator.com/item?id=47804804 And I don't really mean new businesses that are entirely built around LLMs, rather existing ones that pivoted to be LLM-dependent – yet still have non-LLM-dependent competitors.

Yea that would've been extremely short sighted from your competitors. Thanks for linking the response!

Re: The beginning of scarcity in AI

#146

Post by classified »

utopiah wrote: post #85

Initially I thought "Well... good for AI companies because they can then charge more" but IMHO that's a very tricky position because it means the cheap wave is behind us. It's one thing to "sell" free or symbolically cheap stuff, it's another to have an actual client who will do the math and compare expenditure vs actually delivered value.

> and compare expenditure vs actually delivered value

Which means that the hype production will be driven up another few notches to make people doubt their rational findings and keep them in irrational territory just a tad longer. Every minute converts to dollars spent on tokens.

Re: The beginning of scarcity in AI

#147

Post by sevenzero »

bdangubic wrote: post #138
sevenzero wrote: post #134

Earlier quoted context omitted.

This was as clear as the sky when the first llm based businesses popped up. How did you realize this only now?

same as Uber… in the beginning everyone pretty much new that the cost of rides cannot possibly be that cheap and that it is subsudized. once you corner the market etc people just got used to “real” prices to the poibt that now there are often cheaper alternatives than Uber but people still Uber…

Its also quite interesting to read about Uber exploits their drivers and discriminating algorithms. Cory Doctorow mentioned it in his latest book, sadly cant link the direct sources.

Re: The beginning of scarcity in AI

#148

Post by steveklabnik »

wg0 wrote: post #92
drra wrote: post #83

Earlier quoted context omitted.

Seems like everybody an their mothers are using max plans these days. I wouldn't be surprised if LTV of each customer was big enough to justify spending.

Assuming there are 10 million developers and everyone is at $200 max plan, that would be $2 billion/month or $24 billion/year maximum. Note - this is just the revenue not the profit. No salaries, no compute paid for. Just plain revenue. Profit would be way less. But even that - if we take it to $24 billion/year and we take a 10x multiple, the company is barely valued at $240 billon dollar, lets be generous and make i…

Companies are spending far more than $200/month/developer. The $200 Max plan is a great value but you hit limits far too soon, and it also doesn't cover any of the other styles of integrations and tools that you can build and use to help your developers, like code review suggestions, which at the very least would come from additional Max plans, and not from the individual developers' plans.

Re: The beginning of scarcity in AI

#149

Post by zozbot234 »

bcjdjsndon wrote: post #139
zozbot234 wrote: post #117

Earlier quoted context omitted.

> The companies that are entirely AI-dependent may need to raise prices dramatically as AI prices go up. It's not that clear. Sure, hardware prices are going up due to the extremely tight supply, but AI models are also improving quickly to the point where a cheap mid-level model today does what the frontier model did a year ago. For the very largest models, I think the latter effect dominates quite easily.

There's only so far engineers can optimise the underlying transformer technique, which is and always has been doing all the heavy lifting in the recent ai boom. It's going to take another genius to move this forward. We might see improvements here and there but the magnitudes of the data and vram requirements I don't think will change significantly

State space models are already being combined with transformers to form new hybrid models. The state-space part of the architecture is weaker in retrieving information from context (can't find a needle in the haystack as context gets longer, the details effectively get compressed away as everything has to fit in a fixed size) but computationally it's quite strong, O(N) not O(N^2).

Re: The beginning of scarcity in AI

#150

Post by eloisant »

FloorEgg wrote: post #16
mattas wrote: post #7

This notion that "we don't have enough compute" does not cleanly reconcile with the fact that labs are burning cash faster than any cohort of companies in history. If I am a grocery store that pays $1 for oranges and sells them for $0.50, I can't say, "I don't have enough oranges."

There is a major logic flaw in what you're saying. 'If I am a grocery store that pays $1 for oranges and sells them for $0.50, I can't say, "I don't have enough oranges."' How about 'if I'm a grocery store and I see no limit on demand for oranges at $.50 but they are currently $1, I can say 'if oranges were cheaper I could sell orders of magnitude more of them'. Buying oranges for $1 and selling for $0.5 is an invest…

Selling below cost is also called "predatory pricing". Sadly it's legal in US but it's something wealthy companies do to kill competitors and end up with captive customers.
Post reply on HN