Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

github.com

Post reply on HN
Print view View original

161–170 of 176 posts

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#161

Post by School-Cotton »

altairprime wrote: post #148
School-Cotton wrote: post #133

Earlier quoted context omitted.

Applying different standard settings is pretty different from "hijacking and modifying the rendering loop", though.

In what sense? The render loop is modified from “the” default without user or program opt-in, and “hijacking” is what it would be called if anyone but Nvidia did it — so Nvidia is not exempt from that use. Though: Runtime patch, haxie, hijack, LD_PRELOAD, system extension; the noun changes every few years, so perhaps it’s time for a new one. Override?

But the comment I replied to wasn’t talking about runtime patching or any of the other settings you mentioned. It was talking about changing GPU settings for specific programs. Not changing anything about the program itself.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#162

Post by Dylan16807 »

shadowpho wrote: post #156
Dylan16807 wrote: post #143

Earlier quoted context omitted.

Warranty and support specifically for that flag? Because I don't see how general warranty and support requires keeping any hint flags forever.

If you remove the hint flag peoples build will break

Doesn't need to, it can acknowledge and ignore the hints.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#163

Post by shadowpho »

Dylan16807 wrote: post #162
shadowpho wrote: post #156

Earlier quoted context omitted.

If you remove the hint flag peoples build will break

Doesn't need to, it can acknowledge and ignore the hints.

True, but there might be more problems — like if you drop support their run time will be slow because they rely on this flag and they are unhappy

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#164

Post by Dylan16807 »

shadowpho wrote: post #163
Dylan16807 wrote: post #162

Earlier quoted context omitted.

Doesn't need to, it can acknowledge and ignore the hints.

True, but there might be more problems — like if you drop support their run time will be slow because they rely on this flag and they are unhappy

The premise of removing the flag is that it's useless or a problem. If it's still causing a big speed boost somewhere then you need to figure something out, but the core scenario here is that it's obsolete.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#165

Post by Eridrus »

rcoveson wrote: post #134
Eridrus wrote: post #129

Earlier quoted context omitted.

A saner design would turn this optimization into a documented flag that anyone can opt into.

Speaking from a place of long-term frustration with Java, some compiler authors just absolutely hate exposing the ability to hint/force optimizations. Never mind that it might improve performance for N-5 and N+5 major releases, it might be meaningless or unhelpful or difficult to maintain in a release ten years from now, so it must not be exposed today.

So it's better to hardcode your specific library name and deal with the same issue after people have reverse engineered it and started depending on it anyway?

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#166

Post by lmm »

mort96 wrote: post #48
lmm wrote: post #22

Earlier quoted context omitted.

Why do you care? Small commits are great for git bisect, and having to come up with a fancy message can break your flow. Code reviewers generally review a whole PR diff, not the individual commits. Fussing about commit messages smacks of prioritising aesthetics over functionality.

Small commits where each commit represents nothing of value and doesn't compile are terrible for git bisect. And for code review. A code reviewer doesn't care that you spent 10 days across 100 commits tweaking some small piece of code, the code reviewer cares about what you ended up with. The bisecter probably wants to compile the code at every commit to try and see if they can reproduce a bug. Commits which don't co…

> Commits which don't compile

Those are bad, don't do that. (At least don't do long chains of them. The occasional isolated one is ok)

I still find writing a "good" commit message to be a much heavier process than making sure that it compiles.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#167

Post by lmm »

globular-toast wrote: post #40
lmm wrote: post #22

Earlier quoted context omitted.

Why do you care? Small commits are great for git bisect, and having to come up with a fancy message can break your flow. Code reviewers generally review a whole PR diff, not the individual commits. Fussing about commit messages smacks of prioritising aesthetics over functionality.

The only sane thing a maintainer can do with something like this is squash it into one commit. So if you care about `git bisect` then you don't want this. Why do I care? Interesting question. I'm generally a person who cares, I guess. In this specific case it seems analogous to a mechanic having a tidy workshop. Would you leave your bicycle with someone head to toe in grease and tools strewn all over the place? I wou…

> The only sane thing a maintainer can do with something like this is squash it into one commit. So if you care about `git bisect` then you don't want this.

If someone is git bisecting trying to track down a bug they want to land on the smallest code diff possible, not the whole feature change.

> In this specific case it seems analogous to a mechanic having a tidy workshop. Would you leave your bicycle with someone head to toe in grease and tools strewn all over the place? I wouldn't.

Generally I'd be suspicious of a mechanic whose workspace looked too tidy. There's a balance, but it's very easy to keep your tools organised if you're not actually using them much.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#168

Post by lloeki »

globular-toast wrote: post #42
lloeki wrote: post #33

Earlier quoted context omitted.

> you can only delete the human input once git branch "backup/$(git branch --show-current)/$(date +%s)" # do whatever you fancy git reset --hard "backup/$(git branch --show-current)/${thattimestampabove}" You can't lose anything as long as you have a pointer to it (which doubles as making it easy to find)

No need to make a "backup" branch. Learn to trust the reflog.

reflog doesn't keep things through gc.

the only thing that does keep a git object is having a ref that (directly or transitively) points to it

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#169

Post by mort96 »

lmm wrote: post #166
mort96 wrote: post #48

Earlier quoted context omitted.

Small commits where each commit represents nothing of value and doesn't compile are terrible for git bisect. And for code review. A code reviewer doesn't care that you spent 10 days across 100 commits tweaking some small piece of code, the code reviewer cares about what you ended up with. The bisecter probably wants to compile the code at every commit to try and see if they can reproduce a bug. Commits which don't co…

> Commits which don't compile Those are bad, don't do that. (At least don't do long chains of them. The occasional isolated one is ok) I still find writing a "good" commit message to be a much heavier process than making sure that it compiles.

But you disagree with the other part of my comment, or..?

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#170

Post by lmm »

mort96 wrote: post #169
lmm wrote: post #166

Earlier quoted context omitted.

> Commits which don't compile Those are bad, don't do that. (At least don't do long chains of them. The occasional isolated one is ok) I still find writing a "good" commit message to be a much heavier process than making sure that it compiles.

But you disagree with the other part of my comment, or..?

I agree that:

- Human readers don't care about the details of everything you tried

- Commits that both don't compile and have no useful message are of very little value

But I don't think that's an argument against making small commits with uninformative messages, because making those commits without breaking compilation - or even explicitly checking that it compiles before committing - is much easier than coming up with a full commit message. And small commits that do compile - or even small commits of which a significant proportion compile - are very useful for bisect.

Post reply on HN