Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The beginning of scarcity in AI

tomtunguz.com

Post reply on HN
Print view View original

81–90 of 239 posts

Re: The beginning of scarcity in AI

#81

Post by random_human_ »

rstuart4133 wrote: post #74
frodowtf2 wrote: post #69

Earlier quoted context omitted.

> In both cases the code will be crap, as no model I've seen produces good code. I'm wondering if you have actually used claude code because results are not so catastrophic as you describe them.

I used LLMs to write what seems like far too many lines of code now. This is an example Opus 4.6 running at maximum wrote in C: if (foo == NULL) { log_the_error(...); goto END; } END: free(foo); If you don't know C, in older versions that can be a catastrophic failure. (The issue is so serious in modern C `free(NULL)` is a no-op.) If it's difficult to get a `FOO == NULL` without extensive mocking (this is often the c…

Is foo a pointer in your example? Is free(NULL) not a valid operation?

Re: The beginning of scarcity in AI

#82

Post by drra »

dmazin wrote: post #14

Constraints can lead to innovation. Just two things that I think will get dramatically better now that companies have incentive to focus on them: * harness design * small models (both local and not) I think there is tremendous low hanging fruit in both areas still.

Absolutely. Anyone working on inference token level knows how wasteful it all is especially in multimodal tokens.

Re: The beginning of scarcity in AI

#83

Post by drra »

solenoid0937 wrote: post #73
wg0 wrote: post #70

Earlier quoted context omitted.

No matter how low and reasonably Anthropic is valued, don't think $200 Max plans are going to recoup the investment + some return on top because size of the software industry is not that huge and profit margins for AI inference aren't very high either.

Pro and Max plans are probably a drop in the bucket for them.

Seems like everybody an their mothers are using max plans these days. I wouldn't be surprised if LTV of each customer was big enough to justify spending.

Re: The beginning of scarcity in AI

#84

Post by utopiah »

vessenes wrote: post #8
stupefy wrote: post #3

What limits LLM inference accelerators? I heard about Groq ( https://groq.com/ ) not sure how much it pushes away the problem.

ASML only makes a certain number of machines a year that can do extreme ultra-violet lithography. Also - turbine blades limit power, according to Elon. Between them - we cannot chip fabs past a certain rate, and we cannot stand up the datacenter to run these desired chips past a certain rate. Different people believe one or the other is the 'true' current bottleneck. The turbine supply chain scaling looks much more t…

Is ASML really the bottleneck? Do you believe anybody but TSMC and few fabs could really use and acquire those machines? I don't know the throughput of a EUV device from ASML but I imagine you need :

- clean room, itself needing the infrastructure for it (size, airCo, filtering, electricity) and the staff to run and maintain that basically empty space - wafers to "print" on, so that's a lot of water and logistic to manipulate them (so infrastructure for clean water and all chemicals) also with dedicated staff - finally staff who would be able to design something significantly better than NVIDIA, Intel, Broadcom, IBM, etc while (and arguably that's the trickiest part IMHO) being able to get it good enough as at a scale that can be manufactured from their own fab.

so I'm wondering who can afford this kind of setup that can only then make use of ASML machines.

Re: The beginning of scarcity in AI

#85

Post by utopiah »

Initially I thought "Well... good for AI companies because they can then charge more" but IMHO that's a very tricky position because it means the cheap wave is behind us.

It's one thing to "sell" free or symbolically cheap stuff, it's another to have an actual client who will do the math and compare expenditure vs actually delivered value.

Re: The beginning of scarcity in AI

#86

Post by rstuart4133 »

random_human_ wrote: post #81
rstuart4133 wrote: post #74

Earlier quoted context omitted.

I used LLMs to write what seems like far too many lines of code now. This is an example Opus 4.6 running at maximum wrote in C: if (foo == NULL) { log_the_error(...); goto END; } END: free(foo); If you don't know C, in older versions that can be a catastrophic failure. (The issue is so serious in modern C `free(NULL)` is a no-op.) If it's difficult to get a `FOO == NULL` without extensive mocking (this is often the c…

Is foo a pointer in your example? Is free(NULL) not a valid operation?

Yes `foo` is a pointer.

`free(NULL)` is harmless in C89 onwards. As I said, programmers freeing NULL caused so many issues they changed the API. It doesn't help that `malloc(0)` returns NULL on some platforms.

If you are writing code for an embedded platform with some random C compiler, all bets on what `free(NULL)` does are off. That means a cautious C programmer who doesn't know who will be using their code never allows NULL to be passed to `free()`.

In general, most good C programmers are good because they suffer a sort of PTSD from the injuries the language has inflicted on them in the past. If they aren't avoiding passing NULL to `free()`, they haven't suffered long enough to be good.

Re: The beginning of scarcity in AI

#87

Post by Marazan »

utopiah wrote: post #84
vessenes wrote: post #8

Earlier quoted context omitted.

ASML only makes a certain number of machines a year that can do extreme ultra-violet lithography. Also - turbine blades limit power, according to Elon. Between them - we cannot chip fabs past a certain rate, and we cannot stand up the datacenter to run these desired chips past a certain rate. Different people believe one or the other is the 'true' current bottleneck. The turbine supply chain scaling looks much more t…

Is ASML really the bottleneck? Do you believe anybody but TSMC and few fabs could really use and acquire those machines? I don't know the throughput of a EUV device from ASML but I imagine you need : - clean room, itself needing the infrastructure for it (size, airCo, filtering, electricity) and the staff to run and maintain that basically empty space - wafers to "print" on, so that's a lot of water and logistic to m…

> (so infrastructure for clean water and all chemicals)

Fabs are some of the most complex chemical engineering sites (dealing with some of the most dangerous substances) in the world. So don't underestimate the complexity of this part.

Re: The beginning of scarcity in AI

#88

Post by incrudible »

rstuart4133 wrote: post #74
frodowtf2 wrote: post #69

Earlier quoted context omitted.

> In both cases the code will be crap, as no model I've seen produces good code. I'm wondering if you have actually used claude code because results are not so catastrophic as you describe them.

I used LLMs to write what seems like far too many lines of code now. This is an example Opus 4.6 running at maximum wrote in C: if (foo == NULL) { log_the_error(...); goto END; } END: free(foo); If you don't know C, in older versions that can be a catastrophic failure. (The issue is so serious in modern C `free(NULL)` is a no-op.) If it's difficult to get a `FOO == NULL` without extensive mocking (this is often the c…

C is fundamentally a bad target for LLMs. Humans get C wrong all the time, so we can not hope the nascent LLM, which has been trained on 95% code that does automatic memory management, to excel here.

I always found myself writing verbose copypasta code first, then compress it down based on the emerging commonalities. I think doing it the other way around is likely to lead to a worse design. Can you not tell the LLM to do the same? Honest question.

Re: The beginning of scarcity in AI

#89

Post by keiferski »

We just had a realization during a demo call the other day:

The companies that are entirely AI-dependent may need to raise prices dramatically as AI prices go up. Not being dependent on LLMs for your fundamental product’s value will be a major advantage, at least in pricing.

Re: The beginning of scarcity in AI

#90

Post by tasoeur »

com2kid wrote: post #26
dmazin wrote: post #14

Constraints can lead to innovation. Just two things that I think will get dramatically better now that companies have incentive to focus on them: * harness design * small models (both local and not) I think there is tremendous low hanging fruit in both areas still.

China already operates like this. Low cost specialized models are the name of the game. Cheaper to train, easy to deploy. The US has a problem of too much money leading to wasteful spending. If we go back to the 80s/90s, remember OS/2 vs Windows. OS/2 had more resources, more money behind it, more developers, and they built a bigger system that took more resources to run. Mac vs Lisa. Mac team had constraints, Lisa t…

Though I do agree with you, I just came back from a trip to China (Shanghai more specifically) and while attending a couple AI events, the overwhelming majority of people there were using VPNs to access Claude code and codex :-/
Post reply on HN