Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Furiosa: 3.5x efficiency over H100s

furiosa.ai

Post reply on HN
Print view View original

41–50 of 165 posts

Re: Furiosa: 3.5x efficiency over H100s

#41

Post by bsder »

roughly wrote: post #7

I am of the opinion that Nvidia's hit the wall with their current architecture in the same way that Intel has historically with its various architectures - their current generation's power and cooling requirements are requiring the construction of entirely new datacenters with different architectures, which is going to blow out the economics on inference (GPU + datacenter + power plant + nuclear fusion research divis…

We've seen this before.

In 2001, there were something like 50+ OC-768 hardware startups.

At the time, something like 5 OC-768 links could carry all the traffic in the world. Even exponential doubling every 12 months wasn't going to get enough customers to warrant all the funding that had poured into those startups.

When your business model bumps into "All the in the world," you're in trouble.

Re: Furiosa: 3.5x efficiency over H100s

#42

Post by roughly »

nl wrote: post #33
roughly wrote: post #26

Earlier quoted context omitted.

> Their measured performance on things people care about keep going up, and their software stack keeps getting better and unlocking more performance on existing hardware I'm more concerned about fully-loaded dollars per token - including datacenter and power costs - rather than "does the chip go faster." If Nvidia couldn't make the chip go faster, there wouldn't be any debate, the question right now is "what is the c…

> OpenAI has $1.15T in spend commitments over the next 10 years Yes, but those aren't contracted commitments, and we know some of them are equity swaps. For example "Microsoft ($250B Azure commitment)" from the footnote is an unknown amount of actual cash. And I think it's fair to point out the other information in your link "OpenAI projects a 48% gross profit margin in 2025, improving to 70% by 2029."

> "OpenAI projects a 48% gross profit margin in 2025, improving to 70% by 2029."

OpenAI can project whatever they want, they're not public.

Re: Furiosa: 3.5x efficiency over H100s

#43

Post by sothatsit »

Forgeties79 wrote: post #25
nl wrote: post #18

Earlier quoted context omitted.

> I am of the opinion that Nvidia's hit the wall with their current architecture Based on what? Their measured performance on things people care about keep going up, and their software stack keeps getting better and unlocking more performance on existing hardware Inference tests: https://inferencemax.semianalysis.com/ Training tests: https://www.lightly.ai/blog/nvidia-b200-vs-h100 https://newsletter.semianalysis.com/…

> Is that based just on the HN "it is lots of money so it can't possibly make sense" wisdom? I mean the amount of money invested across just a handful of AI companies is currently staggering and their respective revenues are no where near where they need to be. That’s a valid reason to be skeptical. How many times have we seen speculative investment of this magnitude? It’s shifting entire municipal and state economie…

The flip side is that these companies seem to be capacity constrained (although that is hard to confirm). If you assume the labs are capacity constrained, which seems plausible, then building more capacity could pay off by allowing labs to serve more customers and increase revenue per customer.

This means the bigger questions are whether you believe the labs are compute constrained, and whether you believe more capacity would allow them to drive actual revenue. I think there is a decent chance of this being true, and under this reality the investments make more sense. I can especially believe this as we see higher-cost products like Claude Code grow rapidly with much higher token usage per user.

This all hinges on demand materialising when capacity increases, and margins being good enough on that demand to get a good ROI. But that seems like an easier bet for investors to grapple with than trying to compare future investment in capacity with today's revenue, which doesn't capture the whole picture.

Re: Furiosa: 3.5x efficiency over H100s

#44

Post by Forgeties79 »

sothatsit wrote: post #43
Forgeties79 wrote: post #25

Earlier quoted context omitted.

> Is that based just on the HN "it is lots of money so it can't possibly make sense" wisdom? I mean the amount of money invested across just a handful of AI companies is currently staggering and their respective revenues are no where near where they need to be. That’s a valid reason to be skeptical. How many times have we seen speculative investment of this magnitude? It’s shifting entire municipal and state economie…

The flip side is that these companies seem to be capacity constrained (although that is hard to confirm). If you assume the labs are capacity constrained, which seems plausible, then building more capacity could pay off by allowing labs to serve more customers and increase revenue per customer. This means the bigger questions are whether you believe the labs are compute constrained, and whether you believe more capac…

I am not someone who would ever be ever be considered an expert on factories/manufacturing of any kind, but my (insanely basic) understanding is that typically a “factory” making whatever widgets or doodads is outputting at a profit or has a clear path to profitability in order to pay off a loan/investment. They have debt, but they’re moving towards the black in a concrete, relatively predictable way - no one speculates on a factory anywhere near the degree they do with AI companies currently. If said factory’s output is maxed and they’re still not making money, then it’s a losing investment and they wouldn’t expand.

Basically, it strikes me as not really apples to apples.

Re: Furiosa: 3.5x efficiency over H100s

#45

Post by hadlock »

wahnfrieden wrote: post #32
binary132 wrote: post #24

Earlier quoted context omitted.

>go download a model GP was talking about commercially hosted LLMs running in datacenters, not free Chinese models. Local is definitely still improving. That’s another reason the megacenter model (NVDA’s big line up forever plan) is either a financial catastrophe about to happen, or the biggest bailout ever.

GPT 5.2 is an incredible leap over 5.1 / 5

5.2 is great if you ask it engineering questions, or questions an engineer might ask. It is extremely mid, and actually worse than the o3/o4 era models if you start asking it trivia like if the I-80 tunnel on the bay bridge (yerba buena island) is the largest bore in the world. Don't even get me started on whatever model is wired up to the voice chat button.

But yes it will write you a flawless, physics accurate flight simulator in rust on the first try. I've proven that. I guess what I'm trying to say is Anthropic was eating their lunch at coding, and OpenAI rose to the challenge, but if you're not doing engineering tasks their current models are arguably worse than older ones.

Re: Furiosa: 3.5x efficiency over H100s

#46

Post by raincole »

segmondy wrote: post #16
roughly wrote: post #7

I am of the opinion that Nvidia's hit the wall with their current architecture in the same way that Intel has historically with its various architectures - their current generation's power and cooling requirements are requiring the construction of entirely new datacenters with different architectures, which is going to blow out the economics on inference (GPU + datacenter + power plant + nuclear fusion research divis…

> The reason this matters is that LLMs are incredibly nifty often useful tools that are not AGI and also seem to be hitting a scaling wall I don't know who needs to hear this, but the real break through in AI that we have had is not LLMs, but generative AI. LLM is but one specific case. Furthermore, we have hit absolutely no walls. Go download a model from Jan 2024, another from Jan 2025 and one from this year and co…

> exponential

Is this the second most abused english word (after 'literally')?

> a model from Jan 2024, another from Jan 2025 and one from this year

You literally can't tell the difference is 'exponential', quadratic, or whatever from three data points.

Plus it's not my experience at all. Since Deepseek I haven't found models that one can run on consumer hardware get much better.

Re: Furiosa: 3.5x efficiency over H100s

#47

Post by missedthecue »

segmondy wrote: post #16
roughly wrote: post #7

I am of the opinion that Nvidia's hit the wall with their current architecture in the same way that Intel has historically with its various architectures - their current generation's power and cooling requirements are requiring the construction of entirely new datacenters with different architectures, which is going to blow out the economics on inference (GPU + datacenter + power plant + nuclear fusion research divis…

> The reason this matters is that LLMs are incredibly nifty often useful tools that are not AGI and also seem to be hitting a scaling wall I don't know who needs to hear this, but the real break through in AI that we have had is not LLMs, but generative AI. LLM is but one specific case. Furthermore, we have hit absolutely no walls. Go download a model from Jan 2024, another from Jan 2025 and one from this year and co…

There is a lot of talking past each other when discussing LLM performance. The average person whose typical use case is asking ChatGPT how long they need to boil an egg for hasn't seen improvements for 18 months. Meanwhile if you're super into something like local models for example the tangible improvements are without exaggeration happening almost monthly.

Re: Furiosa: 3.5x efficiency over H100s

#48

Post by sothatsit »

Forgeties79 wrote: post #44
sothatsit wrote: post #43

Earlier quoted context omitted.

The flip side is that these companies seem to be capacity constrained (although that is hard to confirm). If you assume the labs are capacity constrained, which seems plausible, then building more capacity could pay off by allowing labs to serve more customers and increase revenue per customer. This means the bigger questions are whether you believe the labs are compute constrained, and whether you believe more capac…

I am not someone who would ever be ever be considered an expert on factories/manufacturing of any kind, but my (insanely basic) understanding is that typically a “factory” making whatever widgets or doodads is outputting at a profit or has a clear path to profitability in order to pay off a loan/investment. They have debt, but they’re moving towards the black in a concrete, relatively predictable way - no one specula…

Consensus seems to be that the labs are profitable on inference. They are only losing money on training and free users.

The competition requiring them to spend that money on training and free users does complicate things. But when you just look at it from an inference perspective, looking at these data centres like token factories makes sense. I would definitely pay more to get faster inference of Opus 4.5, for example.

This is also not wholly dissimilar to other industries where companies spend heavily on R&D while running profitable manufacturing. Pharma semiconductors, and hardware companies like Samsung or Apple all do this. The unusual part with AI labs is the ratio and the uncertainty, but that's a difference of degree, not kind.

Re: Furiosa: 3.5x efficiency over H100s

#49

Post by tedk-42 »

sanxiyn wrote: post #13
richwater wrote: post #11

This is from September 2025, what's new?

What's new is HN discovered it. It wasn't posted in September 2025.

100%

People forget this is also a place of discussion and the comment section is usually peak value as opposed to the article itself.

Re: Furiosa: 3.5x efficiency over H100s

#50

Post by smohare »

segmondy wrote: post #16
roughly wrote: post #7

I am of the opinion that Nvidia's hit the wall with their current architecture in the same way that Intel has historically with its various architectures - their current generation's power and cooling requirements are requiring the construction of entirely new datacenters with different architectures, which is going to blow out the economics on inference (GPU + datacenter + power plant + nuclear fusion research divis…

> The reason this matters is that LLMs are incredibly nifty often useful tools that are not AGI and also seem to be hitting a scaling wall I don't know who needs to hear this, but the real break through in AI that we have had is not LLMs, but generative AI. LLM is but one specific case. Furthermore, we have hit absolutely no walls. Go download a model from Jan 2024, another from Jan 2025 and one from this year and co…

[dead]
Post reply on HN