Live data from Hacker News

Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

github.com

121–130 of 176 posts

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#121
post #49
post #35

Earlier quoted context omitted.

I hate it when I look at some code, wondering why I added a refresh call at that point, I do a git blame to find the commit message, and it says "add refresh call".

But... I keep being told that commit messages are useless because the code is the documentation, so code diffs are self-explanatory...

The code is the "how", sometimes it's necessary to explain the "why".

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#122
post #49

Earlier quoted context omitted.

But... I keep being told that commit messages are useless because the code is the documentation, so code diffs are self-explanatory...

The code is the "how", sometimes it's necessary to explain the "why".

Sometimes I wonder if I really do just need to add /s every time I'm being sarcastic.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#123

Earlier quoted context omitted.

The code is the "how", sometimes it's necessary to explain the "why".

Sometimes I wonder if I really do just need to add /s every time I'm being sarcastic.

Sometimes I think that people who can't write well enough to convey sarcasm when they mean it should just avoid using it and say what they mean.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#124

Earlier quoted context omitted.

Sometimes I wonder if I really do just need to add /s every time I'm being sarcastic.

Sometimes I think that people who can't write well enough to convey sarcasm when they mean it should just avoid using it and say what they mean.

Now that looks like sarcasm.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#125
post #5

When intel did it, the pitchforks came out. Nvidia seems to get a pass. Whys that?

Intel disabled optimisations when they detected they were running on their competitors hardware. The motivation was to make competitors compare badly in benchmarks. Nvidia are disabling optimisations on their own hardware. The motivation appears to be related to these optimisations being unsafe to apply to general code.

Let's be clear here. Intel searched for "GenuineIntel" and ran optimized code on their hardware.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#126
post #49

Earlier quoted context omitted.

But... I keep being told that commit messages are useless because the code is the documentation, so code diffs are self-explanatory...

The code is the "how", sometimes it's necessary to explain the "why".

Sarcasm aside, yes, I 100% agree with you.

When the "why" isn't explained, you end up with things like someone refactoring code and spending time (at best) trying to figure out why some tests now fail or (at worst) breaking something in production.

I'd argue that even the "how" sometimes is better explained in plain words than in code (even if that opens the door for outdated comments when code is changed).

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#127
post #17

Heh. Does anyone remember when almost 25 years ago ATI (AMD) caught manipulating the Quake III benchmarks by renaming the executables to ‘quack’? https://web.archive.org/web/20230929180112/https://techrepor... https://web.archive.org/web/20011108190056/https://hardocp.c... https://web.archive.org/web/20011118183932/www.3dcenter.de/a...

This is weirdly common; phone chipset manufacturers did it with phone benchmarks [0], VW with emissions [1], nVidia did it with 3DMark [2], Intel with the SPEC benchmark for its Xeon processors [3], etc. When it comes to computer graphics, iirc it's pretty normalized now - graphics drivers all seem to have tweaks, settings, optimizations and workarounds for every game. (As an aside, I hate that I have to link to arch…

Goodhart's law: when a measure becomes a target, it ceases to be a good measure.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#128

Earlier quoted context omitted.

Every vendor does this to this day - and its a morally grey practice, drivers hijack and modify the rendering loops of popular games, fixing bugs, replacing shaders with more optimized versions, enabling faster codepaths in the driver etc. These changes are supposed to have minimal to no impact on the actual output, but sometimes vendors are really aggressive, and significantly degrade the outputs so that the game ca…

Sadly it's built into the vulkan protocol. Even a fully userspace driver arrangement with a microkernel ends up giving the driver access to the client's information. Of course it's forgeable the way it's done though so you could opt out if you really wanted to. [1]: https://github.com/KhronosGroup/Vulkan-Headers/blob/main/inc...

I mean Khronos put that in for a reason. If the drivers didn't get explicit information about the application being run, they would do silly heuristics like quake3 to squeeze out performance.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#129
post #3

https://github.com/triton-lang/triton/pull/7298#discussion_r... > By disassembly of ptxas, it is indeed hard-coded that they have logic like: strstr(kernel_name, "cutlass"). > it is likely that, this is an unstable, experimental, aggressive optimization by NVIDIA, and blindly always enabling it may produce some elusive bugs.

Often not elusive bugs, but elusive performance. GPU compilers are hard: Once you've done the basics, trying to do further transforms in a mature compiler will almost always produced mixed results. Some kernels will go faster, some will go slower, and you're hoping to move the balance and not hit any critical kernel too hard in your efforts to make another go faster. An optimization with a universal >=0 speedup acros…

A saner design would turn this optimization into a documented flag that anyone can opt into.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#130

Earlier quoted context omitted.

I think it's fine if you squash it. I have no idea why they didn't squash it before pushing to GitHub though.

They probably didn’t care. And having many small commits instead of a big squashed one can be useful when using git bisect for example.

Not really because CI only needs to pass for the final commit so it's super unlikely that the intermediate ones work.
Post reply on HN