Live data from Hacker News

Furiosa: 3.5x efficiency over H100s

furiosa.ai

141–150 of 165 posts

Re: Furiosa: 3.5x efficiency over H100s

#141

Earlier quoted context omitted.

Hollywood studios are breathing their last gasps now. Anyone will be able to use AI to create blockbuster type movies, Hollywood's moat around that is rapidly draining.

Anyone with a $200M marketing budget.

Throw it on YouTube and get a few key TikTokers to promote it.

Re: Furiosa: 3.5x efficiency over H100s

#142

Earlier quoted context omitted.

My impression is that software developers are the lions share of people actually paying for AI, but perhaps that's just my bubble world view.

According to OpenAI it's something like 4.2% of the use. But this data is from before Codex added subscription support and I think only covers ChatGPT (back when most people were using ChatGPT for coding work, before agents got good). https://i.imgur.com/0XG2CKE.jpeg

I'd believe that but I was commenting on who actually pays for it. My guess is that most individuals using AI in their personal lives are using some sort of free tier.

Re: Furiosa: 3.5x efficiency over H100s

#143
post #78

Earlier quoted context omitted.

> Nothing they create make any goddamn sense, I wouldn’t be that dismissive. Some have managed to make impressive things with them (although nothing close to an actual movie, even a short). https://www.youtube.com/watch?v=ET7Y1nNMXmA A bit older: https://www.youtube.com/watch?v=8OOpYvxKhtY Compared to two years ago: https://www.youtube.com/watch?v=LHeCTfQOQcs

The problem with all of these, even the most recent one, is that they have the "AI look". People have tired of this look already, even for short adverts; if they don't want five minutes of it, they really won't like two hours of it. There is no doubt the quality has vastly improved over time, but I see no sign of progress in removing the "AI look" from these things.

What changed my whole perspective on this a few months ago was Google's Genie 3 demo: https://www.youtube.com/watch?v=PDKhUknuQDg

They have really advanced the coherency of real-time AI generation.

Re: Furiosa: 3.5x efficiency over H100s

#144
post #131
post #7

I am of the opinion that Nvidia's hit the wall with their current architecture in the same way that Intel has historically with its various architectures - their current generation's power and cooling requirements are requiring the construction of entirely new datacenters with different architectures, which is going to blow out the economics on inference (GPU + datacenter + power plant + nuclear fusion research divis…

> I am of the opinion that Nvidia's hit the wall with their current architecture Google presented TPUs in 2015. NVIDIA introduced Tensor Cores in 2018. Both utilize systolic arrays. And last month NVIDIA pseudo-acquired Groq including the founder and original TPU guy. Their LPUs are way more efficient for inference. Also of note Groq is fully made in USA and has a very diverse supply chain using older nodes. NVIDIA a…

Underrated acquisition. Gives NVIDIA a whole lineup of inference-focused hardware that iirc can retrofit into existing air cooled data centres without needing cooling upgrades. Great hedge against the lower-end $$$-per-watt and watt-per-token competition that has been focused purely at inference.

Re: Furiosa: 3.5x efficiency over H100s

#145
post #131

Earlier quoted context omitted.

> I am of the opinion that Nvidia's hit the wall with their current architecture Google presented TPUs in 2015. NVIDIA introduced Tensor Cores in 2018. Both utilize systolic arrays. And last month NVIDIA pseudo-acquired Groq including the founder and original TPU guy. Their LPUs are way more efficient for inference. Also of note Groq is fully made in USA and has a very diverse supply chain using older nodes. NVIDIA a…

Underrated acquisition. Gives NVIDIA a whole lineup of inference-focused hardware that iirc can retrofit into existing air cooled data centres without needing cooling upgrades. Great hedge against the lower-end $$$-per-watt and watt-per-token competition that has been focused purely at inference.

Also a hedge from the memory cartel as Groq uses SRAM. And a reasonable hedge in case Taiwan gets blockaded or something.

Re: Furiosa: 3.5x efficiency over H100s

#146
post #78
post #73

Earlier quoted context omitted.

Have you....used any of the video generators? Nothing they create make any goddamn sense, they're a step above those fake acid trip simulators.

> Nothing they create make any goddamn sense, I wouldn’t be that dismissive. Some have managed to make impressive things with them (although nothing close to an actual movie, even a short). https://www.youtube.com/watch?v=ET7Y1nNMXmA A bit older: https://www.youtube.com/watch?v=8OOpYvxKhtY Compared to two years ago: https://www.youtube.com/watch?v=LHeCTfQOQcs

Have you seen https://www.youtube.com/watch?v=SGJC4Hnz3m0

It's not feature length movie but I'm not sure there's any reason why it couldn't be, and its not technically perfect but pretty damn good.

Re: Furiosa: 3.5x efficiency over H100s

#147

These things never pan out. The reasons why this almost never works is one of the following: - They assume they can move hardware complexity (scheduling etc, access patterns into software). The magic compiler/runtime never arrives. - They assume their hard-to-program but faster architecture will get figured out by devs. It won't. - They assume a certain workload. The workload changes, and their arch is no longer opti…

And when you layer on top networking, it's another level of sw/hw complexity.

Re: Furiosa: 3.5x efficiency over H100s

#148
post #134

Earlier quoted context omitted.

You don’t think real money is changing hands when Microsoft buys Nvidia GPUs?

What about when Nvidia sells GPUs to a client and then buys 10% of their shares?

Their shares will be based on the client's valuation, which in public markets is externally priced. If not in public markets it is murkier, but will be grounded in some sort of reality so Nvidia gets the right amount of the company.

Re: Furiosa: 3.5x efficiency over H100s

#149

Earlier quoted context omitted.

You don’t think real money is changing hands when Microsoft buys Nvidia GPUs?

It's a soft version of money printing basically. These firms are clearly inflating each other's valuations by making huge promises of future business to each other. Naively, one would look at the headlines and draw the conclusion that much more money is going to flow into AI in the near future. Of course, a rational investor looks at this and discounts the fact that most of those promises are predicated on insane gro…

For Nvidia shares: converting cash into shares in a speculative business while guaranteeing increasing demand for your product is a pretty good idea, and probably doesn't have any downsides.

For the AI company being bought: I wouldn't trust these shares or valuations, because the money invested is going on GPUs and back to Nvidia.

Re: Furiosa: 3.5x efficiency over H100s

#150

Earlier quoted context omitted.

I am not someone who would ever be ever be considered an expert on factories/manufacturing of any kind, but my (insanely basic) understanding is that typically a “factory” making whatever widgets or doodads is outputting at a profit or has a clear path to profitability in order to pay off a loan/investment. They have debt, but they’re moving towards the black in a concrete, relatively predictable way - no one specula…

Consensus seems to be that the labs are profitable on inference. They are only losing money on training and free users. The competition requiring them to spend that money on training and free users does complicate things. But when you just look at it from an inference perspective, looking at these data centres like token factories makes sense. I would definitely pay more to get faster inference of Opus 4.5, for examp…

>Consensus seems to be that the labs are profitable on inference. They are only losing money on training and free users.

That sounds like “we’re profitable if you ignore our biggest expenses.” If they could be profitable now, we’d see at least a few companies just be profitable and stop the heavy expenses. My guess is it’s simply not the case or everyone’s trapped in a cycle where they are all required to keep spending too much to keep up and nobody wants to be the first to stop. Either way the outcome is the same.

Post reply on HN