Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it
1–10 of 176 posts
Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it
#2This was discussed before at the time the PR was created and there's nothing new that I can see.
Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it
#3https://github.com/triton-lang/triton/pull/7298#discussion_r...
> By disassembly of ptxas, it is indeed hard-coded that they have logic like: strstr(kernel_name, "cutlass").
> it is likely that, this is an unstable, experimental, aggressive optimization by NVIDIA, and blindly always enabling it may produce some elusive bugs.
Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it
#4https://github.com/triton-lang/triton/pull/7298#discussion_r... > By disassembly of ptxas, it is indeed hard-coded that they have logic like: strstr(kernel_name, "cutlass"). > it is likely that, this is an unstable, experimental, aggressive optimization by NVIDIA, and blindly always enabling it may produce some elusive bugs.
Thanks for a little context, this is not my wheelhouse at all (never even heard of this project) and I could not make heads or tails of the title or the linked PR.
Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it
#5When intel did it, the pitchforks came out.
Nvidia seems to get a pass. Whys that?
Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it
#6When intel did it, the pitchforks came out. Nvidia seems to get a pass. Whys that?
Intel did this to consumers, nvidia does this to enterprises.
Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it
#7Keeping it real with the commit msgs
Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it
#8When intel did it, the pitchforks came out. Nvidia seems to get a pass. Whys that?
This isn't the same thing
Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it
#9Keeping it real with the commit msgs
I much prefer this over those AI generated commit messages that just say "refactored X" every single commit.
Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it
#10When intel did it, the pitchforks came out. Nvidia seems to get a pass. Whys that?
It really depends on details.
If intentionally slowing non CUTLASS shaders, sure pitchfork time.
If it's an option that /technically/ breaks the CUDA shader compatibility contract, then enabling it in specific "known good" situations is just business as usual for GPU drivers.
That can be for all kinds of reasons - straightforward bugs or incomplete paths in the optimization implementation, the app not actually needing the stricter parts of the contract so can have a faster path, or even bugs in apps that need workarounds.
Though piggybacking into these without understanding can be extremely fragile - you don't know why they've limited it, and you run the risk of tripping over some situation that will simply fail, either with incorrect results or something like a crash. And possibly in rather unexpected, unpredictable situations.