I love how now you have to consider the possible s** posting motivation behind analysis of a trillion dollar industry being conducted at a world-class level by a bunch of ex Reddit and 4Chan adjacent mods -- it's one of the best stories in AI that SemiAnalysis is not cut from the same cloth as Gartner McKinsey et al
OpenAI Jalapeño: Better than Nvidia Blackwell
11–20 of 389 posts
Re: OpenAI Jalapeño: Better than Nvidia Blackwell
#12Continued hardware improvements really make it hard for me to believe token prices will not continue to plummet.
Hopefully this also means billionaires can stop trying to drop data centers into residential neighborhoods with zero noise control and polluting on-site generators, signing local politicians on with NDAs, calling for eminent domain to seize homes to build power lines to data centers, etc. etc. etc. Not to mention the water use controversy.
Token prices plummeting is probably a good thing, but not without the regulatory backstops that prevent these effectively industrial facilities from being operated with no regard for the externalities they impose on people who live near them.
Re: OpenAI Jalapeño: Better than Nvidia Blackwell
#13It's so funny to see FP4.... I remember 20 years ago being asked what sort of HPC we needed in genomics, and the answer was basically, "lower precision, faster" for the stuff I was working on. But FP4 is, well, almost comical. One thing not on that comparison table: die size. If I'm understanding that correctly, it's about the same as the Rubin, but at 1/3 the number of NVFP4 PFLOPs. (The text disagrees with the tabl…
Agree, I remember when even half precision made its way into C# sometime around 2020 (I didn’t know much about ML then) and I thought, well I guess that’s a worthwhile tradeoff but I can’t imagine going lower. Lo and behold (1-bit Bonsai) how much lower you could go.
Re: OpenAI Jalapeño: Better than Nvidia Blackwell
#14Earlier quoted context omitted.
Building a custom-designed ASIC is much easier than producing state of the art memory chips. There's a reason why Micron and Nvidia are the crown jewels of American technology right now and for the foreseeable future.
NVIDIA produces memory?
Fabless AFAIK. And that's the actual problem - drawing up CAD diagrams doesn't help if the factories are fully booked out.
Re: OpenAI Jalapeño: Better than Nvidia Blackwell
#15Continued hardware improvements really make it hard for me to believe token prices will not continue to plummet.
There is just so much downward pressure on token price, from every direction. We would need a completely new understanding of economics to explain why the price shouldn’t go down. Or market collusion/regulatory manipulation.
Re: OpenAI Jalapeño: Better than Nvidia Blackwell
#16I love how now you have to consider the possible s** posting motivation behind analysis of a trillion dollar industry being conducted at a world-class level by a bunch of ex Reddit and 4Chan adjacent mods -- it's one of the best stories in AI that SemiAnalysis is not cut from the same cloth as Gartner McKinsey et al
semianalysis is pretty good
Re: OpenAI Jalapeño: Better than Nvidia Blackwell
#17Continued hardware improvements really make it hard for me to believe token prices will not continue to plummet.
this is a story about a proprietary accelerator being built/designed by a token provider. and you think they're going to return the efficiency gains to the customer instead of capture the value for themselves? interesting take.
Re: OpenAI Jalapeño: Better than Nvidia Blackwell
#18When people talk about the commodification of inferencing, they imagine a future where everyone has access to frontier models and can run them at the same cost, and what will actually happen is closer to the commodification of _oil_, where only a few companies have the scale to produce it at a competitive price, and advances like this are _why_.
Once models are more or less interchangeable, the price of LLMs will drop to essentially the price of energy required to run them, and the big labs will be able to run them cheaper than anyone else.
Re: OpenAI Jalapeño: Better than Nvidia Blackwell
#19Continued hardware improvements really make it hard for me to believe token prices will not continue to plummet.
There is just so much downward pressure on token price, from every direction. We would need a completely new understanding of economics to explain why the price shouldn’t go down. Or market collusion/regulatory manipulation.
The price has been going down for ages, its not clear what you are pointing at
Re: OpenAI Jalapeño: Better than Nvidia Blackwell
#20I love how now you have to consider the possible s** posting motivation behind analysis of a trillion dollar industry being conducted at a world-class level by a bunch of ex Reddit and 4Chan adjacent mods -- it's one of the best stories in AI that SemiAnalysis is not cut from the same cloth as Gartner McKinsey et al
SemiAnalysis’ founder was roommates with Anthropic people, not OpenAI, so he may be slightly (very slightly) more objective here.