Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

twitter.com

Post reply on HN
Print view View original

61–70 of 114 posts

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#61

Post by IAmBroom »

MBCook wrote: post #52
jdright wrote: post #37

Earlier quoted context omitted.

yes, that is a very usual way (known practices) of vendors applying specific optimizations for known things. It is also part of the benchmarks game they play against each other.

The link is long dead and the Wayback machine doesn’t have a copy. But in 2001 ATI was caught applying optimizations to Quake 3 when someone realized if you renamed the executable from “quake” to “quack” the score dropped a ton. It was a big scandal. I know that’s common now but that wasn’t a thing that was done at the time.

In at least one past version of Windows (circa 1990s), if you tried to replace the default web browser of IE with another choice you were given an Open File dialog window to choose the executable.

Funny quirk, though: that particular window wouldn't show files named firefox.exe. It would accept that as typed input, if you were at the correct folder, but the file listing omitted that particular file.

Maybe it was mozilla.exe; it was a long time ago. But that was the discovery that pushed me off IE forever.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#62

Post by IAmBroom »

halJordan wrote: post #58
high_na_euv wrote: post #17

I have small experience with compilers and llvm but youd be shocked how many things rely on names and parsing names If you have hundreds of passes that are complex and rely on various "contracts" like type names or some shit, then really crazy things like this can happen unintentionally and not maliciously

Why would i be shocked that a name is informative. Like... are you surprised that wrought iron is wrought? Or cast iron is made from a cast?

Dog piles are often neither composed of dogs, nor actual piles.

Names can be both informative, and misdirecting, at the same time.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#64

Post by lstamour »

IAmBroom wrote: post #61
MBCook wrote: post #52

Earlier quoted context omitted.

The link is long dead and the Wayback machine doesn’t have a copy. But in 2001 ATI was caught applying optimizations to Quake 3 when someone realized if you renamed the executable from “quake” to “quack” the score dropped a ton. It was a big scandal. I know that’s common now but that wasn’t a thing that was done at the time.

In at least one past version of Windows (circa 1990s), if you tried to replace the default web browser of IE with another choice you were given an Open File dialog window to choose the executable. Funny quirk, though: that particular window wouldn't show files named firefox.exe. It would accept that as typed input, if you were at the correct folder, but the file listing omitted that particular file. Maybe it was mozi…

I vaguely remember that being the start of the browser prompts to set your current browser as the default. It was so hard to just configure that they had to build a way to set it within the browser.

You saw that again in more modern times when Microsoft removed support for the APIs they provided to set browser defaults, forcing browser makers to write step by step instructions on what to click to set the default browser.

I believe they walked that back, but it left such a bad taste that I switched my installation of Windows from default mode to EU mode in order to avoid it. And come to think of it, I haven’t used my windows machine for much outside of AI in about 6 months.

But Microsoft is not alone in these sort of defaults games - every OS or browser maker, Apple, Google, Firefox, wants to create moats so they can more easily monetize your usage of a product. I never thought I’d prefer the business model of free to play games, where they just outright ask you for money and have to keep finding new ways to entertain instead of relying on hard to change defaults and selling your data.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#65

Post by the8472 »

high_na_euv wrote: post #17

I have small experience with compilers and llvm but youd be shocked how many things rely on names and parsing names If you have hundreds of passes that are complex and rely on various "contracts" like type names or some shit, then really crazy things like this can happen unintentionally and not maliciously

Some names are standardized items, like memcpy. Matching those is ok, nothing sneaky going on there. Matching something vendor-specific in a general-purpose API is different story.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#66

Post by atomicnumber3 »

MBCook wrote: post #52
jdright wrote: post #37

Earlier quoted context omitted.

yes, that is a very usual way (known practices) of vendors applying specific optimizations for known things. It is also part of the benchmarks game they play against each other.

The link is long dead and the Wayback machine doesn’t have a copy. But in 2001 ATI was caught applying optimizations to Quake 3 when someone realized if you renamed the executable from “quake” to “quack” the score dropped a ton. It was a big scandal. I know that’s common now but that wasn’t a thing that was done at the time.

Was it a scandal at the time? My understanding of how per-game card-driver optimizations work today is:

1. AAAA Game Studio shits out another unoptimized clunker

2. nvidia considers it a reputational risk if games run at 30 FPS on a 5090

3. They go in, look at the perverse ways the game misuses rendering primitives, and then hacks shit in to make whatever bad things they're doing less bad.

As a gamer, this seems fine to me and i generally blame the AAAA devs for being bad at their jobs or AAAA studio leads for being ok shipping unoptimized messes.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#67

Post by amiga386 »

gatlin wrote: post #54
hinkley wrote: post #53

Earlier quoted context omitted.

There are bugs that certain games rely on and features that some don’t use. I’m currently trying to optimize a library out of spite. (I want it to work better than the competitor that caused me a lot of problems on a recent project). The amount of conditional logic around what is essentially a function to increment a value is breathtaking.

Do you have any kind of example you're able to share? I don't mean to take your IP but I want to see this breathtaking vista.

A simple example would be that the function glGetString(GL_EXTENSIONS) crashes the original Quake engine and many licensees, because it's expecting no more than a 256 character string.

The driver looks to see if a known old game is calling it, and if it's one known to crash, it returns no more than 256 characters, and likely also puts all the _old_ extensions that the game is likely to know and react to in the string.

There are also all sorts of games that called APIs in a particular order or set particular options, because they represented a "fast path" at the time, and now they don't, but if you're that program, then yes they do.

Ultimately, this clutter is what let do the development of the Vulcan API, to stop games second-guessing graphics APIs which themselves second-guess the games.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#68

Post by irrelative »

saagarjha wrote: post #13
koakuma-chan wrote: post #4

is 100 tflops a lot?

It's like 5-10% here

Correct, this is the actual headline too. 100 tflops sure seems like it'd be more than that, but here we are.

If the headline was "FB8 is ~7% faster when kernel name has 'cutlass' in it...", it wouldn't seem sensational.

Re: FP8 is ~100 tflops faster when the kernel name has "cutlass" in it

#69

Post by spoaceman7777 »

Seems this is likely due to ongoing work on FP8 support on nvidia/cutlass. From my reading, the alternative code path was likely added recently for testing by external contributors to the cutlass project, and other involved parties. (Rather than attempting to distribute custom packaged internal builds of cuda.)

This ticket is a good starting place to see the chain of issues around the ongoing work: https://github.com/NVIDIA/cutlass/pull/2037

Post reply on HN