Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

twitter.com

Post reply on HN
Print view View original

71–80 of 114 posts

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#71

Post by btbuilder »

atomicnumber3 wrote: post #66
MBCook wrote: post #52

Earlier quoted context omitted.

The link is long dead and the Wayback machine doesn’t have a copy. But in 2001 ATI was caught applying optimizations to Quake 3 when someone realized if you renamed the executable from “quake” to “quack” the score dropped a ton. It was a big scandal. I know that’s common now but that wasn’t a thing that was done at the time.

Was it a scandal at the time? My understanding of how per-game card-driver optimizations work today is: 1. AAAA Game Studio shits out another unoptimized clunker 2. nvidia considers it a reputational risk if games run at 30 FPS on a 5090 3. They go in, look at the perverse ways the game misuses rendering primitives, and then hacks shit in to make whatever bad things they're doing less bad. As a gamer, this seems fine…

I believe the driver silently swapped the textures to lower quality ones that looked worse but gave a performance boost.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#72

Post by antonvs »

atomicnumber3 wrote: post #66
MBCook wrote: post #52

Earlier quoted context omitted.

The link is long dead and the Wayback machine doesn’t have a copy. But in 2001 ATI was caught applying optimizations to Quake 3 when someone realized if you renamed the executable from “quake” to “quack” the score dropped a ton. It was a big scandal. I know that’s common now but that wasn’t a thing that was done at the time.

Was it a scandal at the time? My understanding of how per-game card-driver optimizations work today is: 1. AAAA Game Studio shits out another unoptimized clunker 2. nvidia considers it a reputational risk if games run at 30 FPS on a 5090 3. They go in, look at the perverse ways the game misuses rendering primitives, and then hacks shit in to make whatever bad things they're doing less bad. As a gamer, this seems fine…

> As a gamer, this seems fine to me

As a software developer, it almost certainly has a bad effect on the ecosystem long term. "Hacks shit in" is the very definition of technical debt, and that has a cost that someone, somewhere is going to have to pay in some form.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#73

Post by antonvs »

bravesoul2 wrote: post #40
diggan wrote: post #18

Earlier quoted context omitted.

Web-developers are well aware of this too. Sincerely, Mozilla/5.0 (X11; Linux x86_64; rv:139.0) Gecko/20100101 Firefox/139.0

Funny we send a browser wars tombstone in every request!

Let's have a moment of silence for Gecko/20100101

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#74

Post by itsTyrion »

atomicnumber3 wrote: post #66
MBCook wrote: post #52

Earlier quoted context omitted.

The link is long dead and the Wayback machine doesn’t have a copy. But in 2001 ATI was caught applying optimizations to Quake 3 when someone realized if you renamed the executable from “quake” to “quack” the score dropped a ton. It was a big scandal. I know that’s common now but that wasn’t a thing that was done at the time.

Was it a scandal at the time? My understanding of how per-game card-driver optimizations work today is: 1. AAAA Game Studio shits out another unoptimized clunker 2. nvidia considers it a reputational risk if games run at 30 FPS on a 5090 3. They go in, look at the perverse ways the game misuses rendering primitives, and then hacks shit in to make whatever bad things they're doing less bad. As a gamer, this seems fine…

it rendered in lower quality, IIRC lower textures / much more aggressive mipmapping and/or LOD

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#75

Post by monkpit »

antonvs wrote: post #72
atomicnumber3 wrote: post #66

Earlier quoted context omitted.

Was it a scandal at the time? My understanding of how per-game card-driver optimizations work today is: 1. AAAA Game Studio shits out another unoptimized clunker 2. nvidia considers it a reputational risk if games run at 30 FPS on a 5090 3. They go in, look at the perverse ways the game misuses rendering primitives, and then hacks shit in to make whatever bad things they're doing less bad. As a gamer, this seems fine…

> As a gamer, this seems fine to me As a software developer, it almost certainly has a bad effect on the ecosystem long term. "Hacks shit in" is the very definition of technical debt, and that has a cost that someone, somewhere is going to have to pay in some form.

You’re looking as a dev, but the reality is that a consumer cannot see technical debt. If the studio churns out a game, the vendor sprinkles on some optimizations, people play it and move on, then the tech debt just vaporizes into the void. It’s not real at that point.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#76

Post by RHSeeger »

antonvs wrote: post #72
atomicnumber3 wrote: post #66

Earlier quoted context omitted.

Was it a scandal at the time? My understanding of how per-game card-driver optimizations work today is: 1. AAAA Game Studio shits out another unoptimized clunker 2. nvidia considers it a reputational risk if games run at 30 FPS on a 5090 3. They go in, look at the perverse ways the game misuses rendering primitives, and then hacks shit in to make whatever bad things they're doing less bad. As a gamer, this seems fine…

> As a gamer, this seems fine to me As a software developer, it almost certainly has a bad effect on the ecosystem long term. "Hacks shit in" is the very definition of technical debt, and that has a cost that someone, somewhere is going to have to pay in some form.

I can't reply to the person that replied to you, so

> You’re looking as a dev, but the reality is that a consumer cannot see technical debt.

The consumer can't _see_ technical debt, but they sure as heck can be impacted by it.

- Technical debt means the code base is harder to work with later. So fixes/enhancements take longer to make it into the code (and sometimes never can)

- This particular type of technical debt means the code by the game developers sets precedent, and the next developer may us it as an example. So the amount of code incorrectly using the api grows faster over time

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#77

Post by gmueckl »

atomicnumber3 wrote: post #66
MBCook wrote: post #52

Earlier quoted context omitted.

The link is long dead and the Wayback machine doesn’t have a copy. But in 2001 ATI was caught applying optimizations to Quake 3 when someone realized if you renamed the executable from “quake” to “quack” the score dropped a ton. It was a big scandal. I know that’s common now but that wasn’t a thing that was done at the time.

Was it a scandal at the time? My understanding of how per-game card-driver optimizations work today is: 1. AAAA Game Studio shits out another unoptimized clunker 2. nvidia considers it a reputational risk if games run at 30 FPS on a 5090 3. They go in, look at the perverse ways the game misuses rendering primitives, and then hacks shit in to make whatever bad things they're doing less bad. As a gamer, this seems fine…

Except that if a developer has that kind of market pull, nVidida will gladly help those devs with getting it right. They are excellent at maintaining developer relations.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#78

Post by strbean »

RHSeeger wrote: post #76
antonvs wrote: post #72

Earlier quoted context omitted.

> As a gamer, this seems fine to me As a software developer, it almost certainly has a bad effect on the ecosystem long term. "Hacks shit in" is the very definition of technical debt, and that has a cost that someone, somewhere is going to have to pay in some form.

I can't reply to the person that replied to you, so > You’re looking as a dev, but the reality is that a consumer cannot see technical debt. The consumer can't _see_ technical debt, but they sure as heck can be impacted by it. - Technical debt means the code base is harder to work with later. So fixes/enhancements take longer to make it into the code (and sometimes never can) - This particular type of technical debt…

For some reason HN sometimes hides the reply button on leaf comments. I think this only happens for very new comments.

You can click the timestamp ("X minutes ago") to view the comment without context, and reply from there.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#79

Post by zahlman »

saagarjha wrote: post #31
zahlman wrote: post #30

This tweet appears to be taking the original material out of context to misrepresent it: > Rewrite the attention kernel to be persistent. This gives better performance at low-contexts. However, fp16 at large context has suffered a bit due to a ptxas instruction scheduling issue in the softmax partition. fp8 is ~100 tflops faster when the kernel name has "cutlass" in it. The charitable reading is that, on certain kern…

I think you're the one doing that to the tweet, actually.

What are you talking about? When I view the tweet, the only text I see is:

> > fp8 is 100 tflops faster when the kernel name has "cutlass" in it

> kms

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#80

Post by zahlman »

zettabomb wrote: post #34
zahlman wrote: post #30

This tweet appears to be taking the original material out of context to misrepresent it: > Rewrite the attention kernel to be persistent. This gives better performance at low-contexts. However, fp16 at large context has suffered a bit due to a ptxas instruction scheduling issue in the softmax partition. fp8 is ~100 tflops faster when the kernel name has "cutlass" in it. The charitable reading is that, on certain kern…

No, that sentence is separate from the rest. Take a look at the pull request: # Up to 150 TFLOPS faster for fp8! if specialization.constants["dtype"] == gl.float8e5: name = "cutlass_" + name

The tweet is quoting from the first message in the "conversation" on the PR. There are 93 commits in the PR and GitHub doesn't even default to that tab. I looked at the obvious text and drew the conclusion that was obvious to me.
Post reply on HN