Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

github.com

Post reply on HN
Print view View original

101–110 of 176 posts

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#101

Post by sheepscreek »

joegibbs wrote: post #83
xnorswap wrote: post #39

Earlier quoted context omitted.

> - Tested locally and working correctly This is completely meaningless and just pollutes the log.

God I can't stand it when I get this kind of output from Claude, they really need to train it out for Claude 5. "[Tangentially related emoji] I have completed this fully functional addition to the project that is now working perfectly! There are now zero bugs and the system is ready for deployment to production! [Rocketship emoji]" Then of course you test it out and it doesn't work at all! It's very grating. It would…

> "[Tangentially related emoji] I have completed this fully functional addition to the project that is now working perfectly! There are now zero bugs and the system is ready for deployment to production! [Rocketship emoji]"

This made me laugh so hard. Never trust an AI model saying “There are now zero bugs”! Weaponized incompetence? :)

As a side note, I absolutely am in love with GPT-5 and GPT-5-codex. When I talk to it, it feels like talking to a peer and not an over enthusiastic (but talented) junior with potential. GPT-5-codex on high has been exceptional at debugging insidious bugs.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#102

Post by torginus »

haunter wrote: post #17

Heh. Does anyone remember when almost 25 years ago ATI (AMD) caught manipulating the Quake III benchmarks by renaming the executables to ‘quack’? https://web.archive.org/web/20230929180112/https://techrepor... https://web.archive.org/web/20011108190056/https://hardocp.c... https://web.archive.org/web/20011118183932/www.3dcenter.de/a...

Every vendor does this to this day - and its a morally grey practice, drivers hijack and modify the rendering loops of popular games, fixing bugs, replacing shaders with more optimized versions, enabling faster codepaths in the driver etc.

These changes are supposed to have minimal to no impact on the actual output, but sometimes vendors are really aggressive, and significantly degrade the outputs so that the game can run faster on their hardware.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#103

Post by Hizonner »

high_na_euv wrote: post #70

I work with compilers And despite it not being nice, some optimizations rely on type or function names schemas/substrings/etc It sucks, but thats how it works It doesnt have to be malicious just sometimes it is safer to deploy optimization only for your libs than risk breaking stuff Or your frontend is not giving you more data which you can rely on

> than risk breaking stuff

... until somebody randomly chooses the same name for some reason and gets hosed.

You're not helping.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#104

Post by wiz21c »

high_na_euv wrote: post #70

I work with compilers And despite it not being nice, some optimizations rely on type or function names schemas/substrings/etc It sucks, but thats how it works It doesnt have to be malicious just sometimes it is safer to deploy optimization only for your libs than risk breaking stuff Or your frontend is not giving you more data which you can rely on

On function types or schema, I can understand that. But names ?

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#106

Post by mhh__ »

wiz21c wrote: post #104
high_na_euv wrote: post #70

I work with compilers And despite it not being nice, some optimizations rely on type or function names schemas/substrings/etc It sucks, but thats how it works It doesnt have to be malicious just sometimes it is safer to deploy optimization only for your libs than risk breaking stuff Or your frontend is not giving you more data which you can rely on

On function types or schema, I can understand that. But names ?

Something like:

    if(AskLLM("Does function signature+name look like error handling code")) {
        TurnOffInliner();
    }
is actually probably a lot more effective than you'd think (generating PGO traces with a machine learning tool is apparently a thing that sort of works)

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#107

Post by mhh__ »

davedx wrote: post #74
kilpikaarna wrote: post #7

Keeping it real with the commit msgs

Some criticism of the author here regarding how they structure their diffs. They "made something ~100 tflops faster" and peoples' comments are "their commit messages are bad"? You guys would hate how John Carmack worked, too

https://github.com/oliverbenns/john-carmack-plan you can read carmacks old .plan files

They're mostly not exactly prose but remember this was almost 40 years ago when the dominant style of writing code in some places was still ye olde K&R C with one letter variable names and goto everywhere

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#108

Post by Aurornis »

torginus wrote: post #102
haunter wrote: post #17

Heh. Does anyone remember when almost 25 years ago ATI (AMD) caught manipulating the Quake III benchmarks by renaming the executables to ‘quack’? https://web.archive.org/web/20230929180112/https://techrepor... https://web.archive.org/web/20011108190056/https://hardocp.c... https://web.archive.org/web/20011118183932/www.3dcenter.de/a...

Every vendor does this to this day - and its a morally grey practice, drivers hijack and modify the rendering loops of popular games, fixing bugs, replacing shaders with more optimized versions, enabling faster codepaths in the driver etc. These changes are supposed to have minimal to no impact on the actual output, but sometimes vendors are really aggressive, and significantly degrade the outputs so that the game ca…

> but sometimes vendors are really aggressive, and significantly degrade the outputs so that the game can run faster on their hardware.

Do you have a source for this? I’d like to see some examples

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#109

Post by PunchyHamster »

lmm wrote: post #22
globular-toast wrote: post #12

Someone really needs to learn to use `git commit --amend`. Almost 100 commits with pointless commit messages like "wip" or "x"? Be kinder to your reviewers...

Why do you care? Small commits are great for git bisect, and having to come up with a fancy message can break your flow. Code reviewers generally review a whole PR diff, not the individual commits. Fussing about commit messages smacks of prioritising aesthetics over functionality.

It makes it bitch to know why given change was made.

If you change 2 lines 8 times to check something just squash the commit, saves everyone the hassle

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#110

Post by dahart »

luc_ wrote: post #90

Reminds me of when I was working on NVIDIA Jetson systems, and learning how to use them, that you can run 1 command to make everything go faster... ( https://jetsonhacks.com/2019/04/10/jetson-nano-use-more-powe... )

Do you mean the power mode? The article says there are 2 power modes, 5W and 10W, and that 10W is the default. From that I would assume that you can make everything go faster by using 10W mode, but only if you had already made things go slower by switching away from the default to the 5W mode? Did I miss something?
Post reply on HN