Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

twitter.com

Post reply on HN
Print view View original

81–90 of 114 posts

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#81

Post by charcircuit »

RHSeeger wrote: post #76
antonvs wrote: post #72

Earlier quoted context omitted.

> As a gamer, this seems fine to me As a software developer, it almost certainly has a bad effect on the ecosystem long term. "Hacks shit in" is the very definition of technical debt, and that has a cost that someone, somewhere is going to have to pay in some form.

I can't reply to the person that replied to you, so > You’re looking as a dev, but the reality is that a consumer cannot see technical debt. The consumer can't _see_ technical debt, but they sure as heck can be impacted by it. - Technical debt means the code base is harder to work with later. So fixes/enhancements take longer to make it into the code (and sometimes never can) - This particular type of technical debt…

>the next developer may us it as an example

These hacks are game specific, so another developer wouldn't get them.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#82

Post by zahlman »

imtringued wrote: post #35
zahlman wrote: post #30

This tweet appears to be taking the original material out of context to misrepresent it: > Rewrite the attention kernel to be persistent. This gives better performance at low-contexts. However, fp16 at large context has suffered a bit due to a ptxas instruction scheduling issue in the softmax partition. fp8 is ~100 tflops faster when the kernel name has "cutlass" in it. The charitable reading is that, on certain kern…

https://github.com/triton-lang/triton/pull/7298/commits/a5e2... It's literally in the code.

I already had to deal with Twitter and a link shortening service just to get to GitHub and then it still only pointed to the facing page of a 93-commit PR.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#83

Post by charcircuit »

lstamour wrote: post #64
IAmBroom wrote: post #61

Earlier quoted context omitted.

In at least one past version of Windows (circa 1990s), if you tried to replace the default web browser of IE with another choice you were given an Open File dialog window to choose the executable. Funny quirk, though: that particular window wouldn't show files named firefox.exe. It would accept that as typed input, if you were at the correct folder, but the file listing omitted that particular file. Maybe it was mozi…

I vaguely remember that being the start of the browser prompts to set your current browser as the default. It was so hard to just configure that they had to build a way to set it within the browser. You saw that again in more modern times when Microsoft removed support for the APIs they provided to set browser defaults, forcing browser makers to write step by step instructions on what to click to set the default brow…

An app being able to see itself as the default browser sounds like such a dangerous API, especially if it can be done silently without the user realizing it.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#84

Post by mcculley »

atomicnumber3 wrote: post #66
MBCook wrote: post #52

Earlier quoted context omitted.

The link is long dead and the Wayback machine doesn’t have a copy. But in 2001 ATI was caught applying optimizations to Quake 3 when someone realized if you renamed the executable from “quake” to “quack” the score dropped a ton. It was a big scandal. I know that’s common now but that wasn’t a thing that was done at the time.

Was it a scandal at the time? My understanding of how per-game card-driver optimizations work today is: 1. AAAA Game Studio shits out another unoptimized clunker 2. nvidia considers it a reputational risk if games run at 30 FPS on a 5090 3. They go in, look at the perverse ways the game misuses rendering primitives, and then hacks shit in to make whatever bad things they're doing less bad. As a gamer, this seems fine…

I was surprised to see “AAAA”. I didn’t know there were 4 As now.

“AAAA Game Studio shits out another unoptimized clunker” seems a paradoxical statement to me. I would have thought “AAAA” meant “highly resourced” game company. Does it just mean high revenue? Lots of players?

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#85

Post by mwpmaybe »

mcculley wrote: post #84
atomicnumber3 wrote: post #66

Earlier quoted context omitted.

Was it a scandal at the time? My understanding of how per-game card-driver optimizations work today is: 1. AAAA Game Studio shits out another unoptimized clunker 2. nvidia considers it a reputational risk if games run at 30 FPS on a 5090 3. They go in, look at the perverse ways the game misuses rendering primitives, and then hacks shit in to make whatever bad things they're doing less bad. As a gamer, this seems fine…

I was surprised to see “AAAA”. I didn’t know there were 4 As now. “AAAA Game Studio shits out another unoptimized clunker” seems a paradoxical statement to me. I would have thought “AAAA” meant “highly resourced” game company. Does it just mean high revenue? Lots of players?

High price...

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#86

Post by lsaferite »

charcircuit wrote: post #81
RHSeeger wrote: post #76

Earlier quoted context omitted.

I can't reply to the person that replied to you, so > You’re looking as a dev, but the reality is that a consumer cannot see technical debt. The consumer can't _see_ technical debt, but they sure as heck can be impacted by it. - Technical debt means the code base is harder to work with later. So fixes/enhancements take longer to make it into the code (and sometimes never can) - This particular type of technical debt…

>the next developer may us it as an example These hacks are game specific, so another developer wouldn't get them.

The next developer at that company that uses or references the crappy code for another project would still have the issue, but not get the benefit of the down-stream GPU vendor hacks to fix the buggy game.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#87

Post by monocasa »

strbean wrote: post #78
RHSeeger wrote: post #76

Earlier quoted context omitted.

I can't reply to the person that replied to you, so > You’re looking as a dev, but the reality is that a consumer cannot see technical debt. The consumer can't _see_ technical debt, but they sure as heck can be impacted by it. - Technical debt means the code base is harder to work with later. So fixes/enhancements take longer to make it into the code (and sometimes never can) - This particular type of technical debt…

For some reason HN sometimes hides the reply button on leaf comments. I think this only happens for very new comments. You can click the timestamp ("X minutes ago") to view the comment without context, and reply from there.

I think it's a anti flamewar tactic to put the brakes on quick replies.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#88

Post by toast0 »

atomicnumber3 wrote: post #66
MBCook wrote: post #52

Earlier quoted context omitted.

The link is long dead and the Wayback machine doesn’t have a copy. But in 2001 ATI was caught applying optimizations to Quake 3 when someone realized if you renamed the executable from “quake” to “quack” the score dropped a ton. It was a big scandal. I know that’s common now but that wasn’t a thing that was done at the time.

Was it a scandal at the time? My understanding of how per-game card-driver optimizations work today is: 1. AAAA Game Studio shits out another unoptimized clunker 2. nvidia considers it a reputational risk if games run at 30 FPS on a 5090 3. They go in, look at the perverse ways the game misuses rendering primitives, and then hacks shit in to make whatever bad things they're doing less bad. As a gamer, this seems fine…

> Was it a scandal at the time?

Yes. My understanding was it was optimized by reducing precision or something to a visibly apparent degree.

It's different if the driver changes things in ways such that rendered output is the same or at least imperceptibly different. I think there's also a lot more communication between gpu makers and game/engine developers these days; plus a lot more frequent updates.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#89

Post by vinceguidry »

mcculley wrote: post #84
atomicnumber3 wrote: post #66

Earlier quoted context omitted.

Was it a scandal at the time? My understanding of how per-game card-driver optimizations work today is: 1. AAAA Game Studio shits out another unoptimized clunker 2. nvidia considers it a reputational risk if games run at 30 FPS on a 5090 3. They go in, look at the perverse ways the game misuses rendering primitives, and then hacks shit in to make whatever bad things they're doing less bad. As a gamer, this seems fine…

I was surprised to see “AAAA”. I didn’t know there were 4 As now. “AAAA Game Studio shits out another unoptimized clunker” seems a paradoxical statement to me. I would have thought “AAAA” meant “highly resourced” game company. Does it just mean high revenue? Lots of players?

The more money you throw at an effort, the more gets flushed out as waste, and the harder it is to maintain quality. Pretty universal across business.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#90

Post by KronisLV »

toast0 wrote: post #88
atomicnumber3 wrote: post #66

Earlier quoted context omitted.

Was it a scandal at the time? My understanding of how per-game card-driver optimizations work today is: 1. AAAA Game Studio shits out another unoptimized clunker 2. nvidia considers it a reputational risk if games run at 30 FPS on a 5090 3. They go in, look at the perverse ways the game misuses rendering primitives, and then hacks shit in to make whatever bad things they're doing less bad. As a gamer, this seems fine…

> Was it a scandal at the time? Yes. My understanding was it was optimized by reducing precision or something to a visibly apparent degree. It's different if the driver changes things in ways such that rendered output is the same or at least imperceptibly different. I think there's also a lot more communication between gpu makers and game/engine developers these days; plus a lot more frequent updates.

> My understanding was it was optimized by reducing precision or something to a visibly apparent degree.

If only we had that sort of a control over rendering for every game ourselves - since projects like OptiScaler at least let us claw back control over sometimes proprietary upscaling and even framegen, but it's not quite enough: https://github.com/optiscaler/OptiScaler

I'd also mention Lossless Scaling here, though it still only works on upscaling and framegen and with worse methods, but at least works for most games out there: https://store.steampowered.com/app/993090/Lossless_Scaling/

I want to be able to freely toggle between different types of AA and SSAO and reflections and lighting and LOD systems and various shader effects (especially things like chromatic aberration or motion blur) and ray tracing and all that, instead of having to hope that the console port that's offered to me has those abilities in the graphics menu and that whoever is making the decisions hasn't decided that actually "low" graphics (that would at least run smoothly) would look too bad for the game's brand image or something.

Post reply on HN