Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The beginning of scarcity in AI

tomtunguz.com

Post reply on HN
Print view View original

11–20 of 239 posts

Re: The beginning of scarcity in AI

#11

Post by vessenes »

mattas wrote: post #7

This notion that "we don't have enough compute" does not cleanly reconcile with the fact that labs are burning cash faster than any cohort of companies in history. If I am a grocery store that pays $1 for oranges and sells them for $0.50, I can't say, "I don't have enough oranges."

You misunderstand.

"I built a ship to go to the Indies and bring back tea."

"Bro, the ship cost 100,000 pounds sterling and only brought back 50,000 pounds of tea. I don't care if you paid 12,500 pounds for the tea itself, you're losing money."

There is a very rational reason labs are spending everything they can get for more compute right now. The tea (inference) pays 60%+ margins. And that is rising. And that number is AFTER hyper scalars make their margins. There is an immense amount of profit floating around this system, and strategics at the edge believing they can build and control the demand through combined spend on training and inference in the proper ratios.

Re: The beginning of scarcity in AI

#13

Post by vessenes »

ls612 wrote: post #10
vessenes wrote: post #8

Earlier quoted context omitted.

ASML only makes a certain number of machines a year that can do extreme ultra-violet lithography. Also - turbine blades limit power, according to Elon. Between them - we cannot chip fabs past a certain rate, and we cannot stand up the datacenter to run these desired chips past a certain rate. Different people believe one or the other is the 'true' current bottleneck. The turbine supply chain scaling looks much more t…

Presumably ASML can increase production if demand is high enough the question is over what time frame. 5 years seems plausible to me but I honestly don't know what that number is.

It's ... really long, according to Dylan Patel on the Dwarkesh Podcast. The supply chain is extremely deep and complex.

Re: The beginning of scarcity in AI

#14

Post by dmazin »

Constraints can lead to innovation. Just two things that I think will get dramatically better now that companies have incentive to focus on them:

* harness design

* small models (both local and not)

I think there is tremendous low hanging fruit in both areas still.

Re: The beginning of scarcity in AI

#15

Post by vessenes »

isawczuk wrote: post #12

It's artificial scarcity. LLM inference will soon be commodity as cloud. There is a 2-3years still before ASIC LLM inferences will catch up.

I don't think so. GB200 prices are GOING UP. A100s are still expensive. This implies massive utilization and demand, no? These machines are not sitting idle, or prices would drop in the very competitive hyperscaler environment.

Re: The beginning of scarcity in AI

#16

Post by FloorEgg »

mattas wrote: post #7

This notion that "we don't have enough compute" does not cleanly reconcile with the fact that labs are burning cash faster than any cohort of companies in history. If I am a grocery store that pays $1 for oranges and sells them for $0.50, I can't say, "I don't have enough oranges."

There is a major logic flaw in what you're saying.

'If I am a grocery store that pays $1 for oranges and sells them for $0.50, I can't say, "I don't have enough oranges."'

How about 'if I'm a grocery store and I see no limit on demand for oranges at $.50 but they are currently $1, I can say 'if oranges were cheaper I could sell orders of magnitude more of them'.

Buying oranges for $1 and selling for $0.5 is an investment into acquiring market share and customer relationships and a gamble on the price of oranges falling in the future.

Re: The beginning of scarcity in AI

#17

Post by 0x3f »

earthnail wrote: post #9
mattas wrote: post #7

This notion that "we don't have enough compute" does not cleanly reconcile with the fact that labs are burning cash faster than any cohort of companies in history. If I am a grocery store that pays $1 for oranges and sells them for $0.50, I can't say, "I don't have enough oranges."

If there were more oranges you’d pay less to buy them and your economics would work out.

Not sure if this is a joke or not, but competitive pressure still exists. This only really holds if you're the only orange seller.

Re: The beginning of scarcity in AI

#18

Post by 0x3f »

FloorEgg wrote: post #16
mattas wrote: post #7

This notion that "we don't have enough compute" does not cleanly reconcile with the fact that labs are burning cash faster than any cohort of companies in history. If I am a grocery store that pays $1 for oranges and sells them for $0.50, I can't say, "I don't have enough oranges."

There is a major logic flaw in what you're saying. 'If I am a grocery store that pays $1 for oranges and sells them for $0.50, I can't say, "I don't have enough oranges."' How about 'if I'm a grocery store and I see no limit on demand for oranges at $.50 but they are currently $1, I can say 'if oranges were cheaper I could sell orders of magnitude more of them'. Buying oranges for $1 and selling for $0.5 is an invest…

> acquiring market share and customer relationships

The whole setup rests on this, and it seems mythical to me. These guys have basically equivalent products at this point.

Re: The beginning of scarcity in AI

#20

Post by itmitica »

The current inference system is on a down slope.

It remains to be seen what new wave of AI system or systems will replace it, making the whole current architecture obsolete.

Meanwhile, they are milking it, in the name of scarcity.

Post reply on HN