Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

github.com

Post reply on HN
Print view View original

151–160 of 176 posts

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#151

Post by dahart »

AnotherGoodName wrote: post #120
Aurornis wrote: post #108

Earlier quoted context omitted.

> but sometimes vendors are really aggressive, and significantly degrade the outputs so that the game can run faster on their hardware. Do you have a source for this? I’d like to see some examples

Nvidia has a control panel with it's drivers. Open it up -> Manage 3D settings -> Program Settings. Scroll through and see how every single program/game you have installed openly has different defaults in it based on application name. As someone noted above others do the same thing. Eg. Frostpunk has Antialiasing for transparency layers on. Slay the spire does not. I never set these settings. Nvidia literally does a…

That’s not what @torginus was referring to. There’s nothing wrong with having and exposing application specific settings. There’s nothing wrong with drivers having application specific optimization patches either, but that’s a very different thing.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#152

Post by Adachi91 »

haunter wrote: post #17

Heh. Does anyone remember when almost 25 years ago ATI (AMD) caught manipulating the Quake III benchmarks by renaming the executables to ‘quack’? https://web.archive.org/web/20230929180112/https://techrepor... https://web.archive.org/web/20011108190056/https://hardocp.c... https://web.archive.org/web/20011118183932/www.3dcenter.de/a...

I feel like omitting AMD is relevant here for anyone who doesn't know the acquisition history of ATI, AMD had no involvement with this.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#153

Post by recursivecaveat »

rcoveson wrote: post #134
Eridrus wrote: post #129

Earlier quoted context omitted.

A saner design would turn this optimization into a documented flag that anyone can opt into.

Speaking from a place of long-term frustration with Java, some compiler authors just absolutely hate exposing the ability to hint/force optimizations. Never mind that it might improve performance for N-5 and N+5 major releases, it might be meaningless or unhelpful or difficult to maintain in a release ten years from now, so it must not be exposed today.

I once exposed a "disableXYZOptimization" flag to customers so they could debug a easier without stuff getting scrambled. Paid for my gesture for the next year signing off on release updates, writing user guide entries, bleh.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#154

Post by [deleted] »

darkmighty wrote: post #132
Cthulhu_ wrote: post #64

Earlier quoted context omitted.

This is weirdly common; phone chipset manufacturers did it with phone benchmarks [0], VW with emissions [1], nVidia did it with 3DMark [2], Intel with the SPEC benchmark for its Xeon processors [3], etc. When it comes to computer graphics, iirc it's pretty normalized now - graphics drivers all seem to have tweaks, settings, optimizations and workarounds for every game. (As an aside, I hate that I have to link to arch…

> graphics drivers all seem to have tweaks, settings, optimizations and workarounds for every game. Maybe hyperbole, but I think obviously they can't do this for literally every game, that would require huge personnel resources. At least looking at mesa (linked elsewhere), only ~200 games are patched, out of what 100k PC games? So <1%.

[deleted]

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#155

Post by tylerchr »

almostgotcaught wrote: post #149
tylerchr wrote: post #50

Earlier quoted context omitted.

You might be surprised. Yours sounds like the attitude of someone who has not had the luxury of reviewing well-constructed commits. PRs with intentional commits permit both faster and deeper reviews—but alas, not everyone is so respectful of their reviewers’ time and energy.

> Yours sounds like the attitude of someone who has not had the luxury of Sometimes when people speak rhetorically I'm baffled because I feel they literally do not understand what they're saying because they end up supporting an opposing rhetorical purpose. Yes you're 100% correct well-structured commits are a luxury that most of us do not have the privilege of experiencing because we work in high-pressure, deadlines…

Yeah, exactly. It seems like you understand just fine.

You claimed that “literally no one” has a different review workflow than yours. I do, and my experience is that clear commits make reviews both faster and deeper, which is very helpful specifically in a high-pressure, deadline driven environment where being slow and wrong is costly. You’re of course free to disagree and work differently.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#156

Post by shadowpho »

Dylan16807 wrote: post #143
MichaelZuo wrote: post #135

Earlier quoted context omitted.

That seems valid for customers expecting a warranty or support. But they should allow it if customers waive all such in writing.

Warranty and support specifically for that flag? Because I don't see how general warranty and support requires keeping any hint flags forever.

If you remove the hint flag peoples build will break

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#157

Post by colejohnson66 »

wiz21c wrote: post #104
high_na_euv wrote: post #70

I work with compilers And despite it not being nice, some optimizations rely on type or function names schemas/substrings/etc It sucks, but thats how it works It doesnt have to be malicious just sometimes it is safer to deploy optimization only for your libs than risk breaking stuff Or your frontend is not giving you more data which you can rely on

On function types or schema, I can understand that. But names ?

Not exactly the same, but intrinsics in some languages are purely name+signature matched.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#158

Post by fragmede »

tylerchr wrote: post #155
almostgotcaught wrote: post #149

Earlier quoted context omitted.

> Yours sounds like the attitude of someone who has not had the luxury of Sometimes when people speak rhetorically I'm baffled because I feel they literally do not understand what they're saying because they end up supporting an opposing rhetorical purpose. Yes you're 100% correct well-structured commits are a luxury that most of us do not have the privilege of experiencing because we work in high-pressure, deadlines…

Yeah, exactly. It seems like you understand just fine. You claimed that “literally no one” has a different review workflow than yours. I do, and my experience is that clear commits make reviews both faster and deeper, which is very helpful specifically in a high-pressure, deadline driven environment where being slow and wrong is costly. You’re of course free to disagree and work differently.

You gotta go slow in order to go fast, and utterly useless commit messages and inappropriate commit sizes will bite you in the ass. They don't have to be the most beautiful commits ever, but ideally there's a minimum standard we can all live up to.

To use restaurants as the analogy, Michelin star-grade dining might be unavailable, and we might have to live the Olive Garden, or even McDonald's life. Regardless of which restaurant we're at though, if the food is moldy and gross, we shouldn't eat it.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#159

Post by godelski »

frogblast wrote: post #116
nulld3v wrote: post #3

https://github.com/triton-lang/triton/pull/7298#discussion_r... > By disassembly of ptxas, it is indeed hard-coded that they have logic like: strstr(kernel_name, "cutlass"). > it is likely that, this is an unstable, experimental, aggressive optimization by NVIDIA, and blindly always enabling it may produce some elusive bugs.

Often not elusive bugs, but elusive performance. GPU compilers are hard: Once you've done the basics, trying to do further transforms in a mature compiler will almost always produced mixed results. Some kernels will go faster, some will go slower, and you're hoping to move the balance and not hit any critical kernel too hard in your efforts to make another go faster. An optimization with a universal >=0 speedup acros…

  > An optimization with a universal >=0 speedup across your entire suite of tests is a really hard thing to come by. Something is always going to have a negative speedup.
Maybe a common example of this is that people can write matrix matrix multiplication kernels that outperform standard implementations (also in BLAS for CPU). But that's not a General Matrix Matrix multiply. Is the speedup still there for spare matrices? Larger ones? Small ones? Ones that aren't powers of 2? Non-square? And so on. You can beat the official implementation in any one of these but good luck doing it everywhere. In fact, you should beat the official method because you don't have the overhead to check which optimization you should use.

It's easy to over simplify a problem and not even realize you have done so. There's always assumptions being made and you should not let these be invisible.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#160

Post by dehugger »

mhh__ wrote: post #107
davedx wrote: post #74

Earlier quoted context omitted.

Some criticism of the author here regarding how they structure their diffs. They "made something ~100 tflops faster" and peoples' comments are "their commit messages are bad"? You guys would hate how John Carmack worked, too

https://github.com/oliverbenns/john-carmack-plan you can read carmacks old .plan files They're mostly not exactly prose but remember this was almost 40 years ago when the dominant style of writing code in some places was still ye olde K&R C with one letter variable names and goto everywhere

Found this in there, a great soapbox piece about opengl vs D3D. https://github.com/oliverbenns/john-carmack-plan/blob/master...
Post reply on HN