Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

twitter.com

Post reply on HN
Print view View original

101–110 of 114 posts

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#101

Post by saagarjha »

zahlman wrote: post #98
saagarjha wrote: post #97

Earlier quoted context omitted.

And it includes a link to show that this is the context it came from.

And when I look at the link, the part I quoted is the relevant text I see. In order to get to the part that you're trying to hold me accountable for, I would furthermore have to click onto the commits tab and search through a 93-commit PR. I thought today I was using a site where trying to think the best of people and propose that someone had taken something out of context, based on the immediately available context…

I don't think you are a corporate shill. I do think that you immediately going "clearly the tweet is wrong" without doing any research whatsoever was unwarranted, though. You also keep bringing up that it's 93 commits but all getting squashed you have to do is search for "cutlass" to find out what is going on. I think you're obligated to do at least that when you call it out for being wrong.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#102

Post by cyanydeez »

antonvs wrote: post #72
atomicnumber3 wrote: post #66

Earlier quoted context omitted.

Was it a scandal at the time? My understanding of how per-game card-driver optimizations work today is: 1. AAAA Game Studio shits out another unoptimized clunker 2. nvidia considers it a reputational risk if games run at 30 FPS on a 5090 3. They go in, look at the perverse ways the game misuses rendering primitives, and then hacks shit in to make whatever bad things they're doing less bad. As a gamer, this seems fine…

> As a gamer, this seems fine to me As a software developer, it almost certainly has a bad effect on the ecosystem long term. "Hacks shit in" is the very definition of technical debt, and that has a cost that someone, somewhere is going to have to pay in some form.

Does anyone talk about how technical debt often just gets thrown into the garbage so we can buy fancy new technical crap, and its what pays for most of yalls jobs.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#103

Post by [deleted] »

antonvs wrote: post #72
atomicnumber3 wrote: post #66

Earlier quoted context omitted.

Was it a scandal at the time? My understanding of how per-game card-driver optimizations work today is: 1. AAAA Game Studio shits out another unoptimized clunker 2. nvidia considers it a reputational risk if games run at 30 FPS on a 5090 3. They go in, look at the perverse ways the game misuses rendering primitives, and then hacks shit in to make whatever bad things they're doing less bad. As a gamer, this seems fine…

> As a gamer, this seems fine to me As a software developer, it almost certainly has a bad effect on the ecosystem long term. "Hacks shit in" is the very definition of technical debt, and that has a cost that someone, somewhere is going to have to pay in some form.

[deleted]

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#104

Post by hamburglar »

hinkley wrote: post #56
gatlin wrote: post #54

Earlier quoted context omitted.

Do you have any kind of example you're able to share? I don't mean to take your IP but I want to see this breathtaking vista.

To avoid doxxing myself: In a deep call stack it’s possible to end up sanitizing inputs multiple times and in different ways. A frequent example I’ve encountered is web frameworks that have to keep checking for escaped text because they didn’t write it in horizontal layers where you know for sure that all inputs have been scrubbed when they reach this function but not that one. So the same functions get called with d…

“Checking for escaped text” is the sort of nonsense that tells you you’re dealing with amateur developers.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#105

Post by 8n4vidtmkvmk »

wtetzner wrote: post #92
monkpit wrote: post #75

Earlier quoted context omitted.

You’re looking as a dev, but the reality is that a consumer cannot see technical debt. If the studio churns out a game, the vendor sprinkles on some optimizations, people play it and move on, then the tech debt just vaporizes into the void. It’s not real at that point.

Just because a consumer can't see technical debt doesn't mean they aren't paying for it. Most game studios continue to re-use code, so it doesn't just "vaporize" into the void.

I'm pretty sure I pay this debt with lost FPS and every time I glitch through the floor into the nether.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#106

Post by withinboredom »

hamburglar wrote: post #104
hinkley wrote: post #56

Earlier quoted context omitted.

To avoid doxxing myself: In a deep call stack it’s possible to end up sanitizing inputs multiple times and in different ways. A frequent example I’ve encountered is web frameworks that have to keep checking for escaped text because they didn’t write it in horizontal layers where you know for sure that all inputs have been scrubbed when they reach this function but not that one. So the same functions get called with d…

“Checking for escaped text” is the sort of nonsense that tells you you’re dealing with amateur developers.

Indeed. The rules are simple:

- Unescape, sanitize or validate at all entry points.

- Escape all outputs (this includes the database queries).

If you follow those simple rules, you never have to check once you are past a controller. And you should fuzz your controllers to make sure no unexpected data makes it past there.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#108

Post by zahlman »

saagarjha wrote: post #101
zahlman wrote: post #98

Earlier quoted context omitted.

And when I look at the link, the part I quoted is the relevant text I see. In order to get to the part that you're trying to hold me accountable for, I would furthermore have to click onto the commits tab and search through a 93-commit PR. I thought today I was using a site where trying to think the best of people and propose that someone had taken something out of context, based on the immediately available context…

I don't think you are a corporate shill. I do think that you immediately going "clearly the tweet is wrong" without doing any research whatsoever was unwarranted, though. You also keep bringing up that it's 93 commits but all getting squashed you have to do is search for "cutlass" to find out what is going on. I think you're obligated to do at least that when you call it out for being wrong.

How did you get "clearly" out of "appears to be"?

How did you get "without doing any research whatsoever" out of me demonstrably following the link and reading and quoting what appeared on the facing page?

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#109

Post by Xss3 »

spoaceman7777 wrote: post #69

Seems this is likely due to ongoing work on FP8 support on nvidia/cutlass. From my reading, the alternative code path was likely added recently for testing by external contributors to the cutlass project, and other involved parties. (Rather than attempting to distribute custom packaged internal builds of cuda.) This ticket is a good starting place to see the chain of issues around the ongoing work: https://github.com…

The real answer

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#110

Post by bayindirh »

atomicnumber3 wrote: post #66
MBCook wrote: post #52

Earlier quoted context omitted.

The link is long dead and the Wayback machine doesn’t have a copy. But in 2001 ATI was caught applying optimizations to Quake 3 when someone realized if you renamed the executable from “quake” to “quack” the score dropped a ton. It was a big scandal. I know that’s common now but that wasn’t a thing that was done at the time.

Was it a scandal at the time? My understanding of how per-game card-driver optimizations work today is: 1. AAAA Game Studio shits out another unoptimized clunker 2. nvidia considers it a reputational risk if games run at 30 FPS on a 5090 3. They go in, look at the perverse ways the game misuses rendering primitives, and then hacks shit in to make whatever bad things they're doing less bad. As a gamer, this seems fine…

A friend of mine developed his own game engine, and what he said is you need to bargain with the nVidia driver, because hardware doesn't perform at its peak when you write everything honoring the spec, and driver feels free to ignore your commands about how you want to do some things (e.g. memory transfers).

Like board manufacturers, the game developers also need to please the drivers and do the way driver silently dictates to them (regardless of what DirectX, OpenGL or Vulkan says), otherwise all bets are off.

Post reply on HN