Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

github.com

Post reply on HN
Print view View original

141–150 of 176 posts

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#141

Post by 0x457 »

darkmighty wrote: post #132
Cthulhu_ wrote: post #64

Earlier quoted context omitted.

This is weirdly common; phone chipset manufacturers did it with phone benchmarks [0], VW with emissions [1], nVidia did it with 3DMark [2], Intel with the SPEC benchmark for its Xeon processors [3], etc. When it comes to computer graphics, iirc it's pretty normalized now - graphics drivers all seem to have tweaks, settings, optimizations and workarounds for every game. (As an aside, I hate that I have to link to arch…

> graphics drivers all seem to have tweaks, settings, optimizations and workarounds for every game. Maybe hyperbole, but I think obviously they can't do this for literally every game, that would require huge personnel resources. At least looking at mesa (linked elsewhere), only ~200 games are patched, out of what 100k PC games? So <1%.

Well, pretty much every large AAA game launch complimented by GPU driver upgrade that adds support for that game. It's in the patch notes.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#142

Post by fragmede »

davedx wrote: post #73
gorgoiler wrote: post #30

Earlier quoted context omitted.

You have the right idea but, I believe, the wrong reasoning with your first two arguments. git-bisect works best when every commit works , contains a single idea, and stacks in a linear history. These features are of most use in a publicly visible branch, and is why it is helpful to squash an entire pull-request into a single, atomic commit — one which clearly defines the change from before- to after-this-feature. Yo…

> When it’s ready for the attention of your peers then you absolutely ought to dress it up as smartly as possible. It’s at that point that you write a cover letter for your change: what was the situation before, why that was bad, what this patch does instead, and how you proved in practice that it made things better (tests!) This is an extremely opinionated and time consuming way of working. Maybe in this context it…

One of the best things about git, and the reason it won, is that as a tool, it's extremely unopinionated on this matter, and is supportive of however you want to do it. Of course, one of the worst things about git is how unopinionated it is. If you want 300 commit messages in every branch with the commit message of "poop" and no squashing, and none of them even compile, the tool isn't going to stop you. If every commit is fully functional and rebased on top of master so the graph isn't an octopus, you can. If you'd rather use the name main as the primary branch, also totally fine. Git, the tool leaves all that up to the user and the culture they operate in.

Naturally, I have Opinions on the right way to use git, having used it since inception within various different contexts at various places, along with other VCSs. What works at one place won't be right for another place, and vice versa. Especially given different skill levels of individuals and of teams, the tools involved, and how much weight I have to review code and commits before it gets accepted. What's important is it should work for you, not the other way around. Regardless of where I'm working though, my local commit messages are total crap. "wip" being the most common, but I commit frequently. Importantly though, before I do eg a slightly involved refactor, going back to see what it was before I started is trivial. Being a skilled operator of git is important to make it easy to run newly written tests against the old code. Being efficient at rebase -i and sorting commits into understandable chunks and squashing minor commits to keep things clean is key.

I don't think every patch in a series has to work totally independently for every git repo, but what it comes down to is maintenance. There's nothing worse than digging around in git history, trying to figure out why things are how they are, only to dead end at a 3000 line commit from 5 years ago with the message "poop, lol". It's even worse when the person who did that was you!

Universally, what it comes down to is maintenance. That totally rushed prototype that was just for a demo has now been in production for years, and there's this weird bug with the new database. If you hate yourself, your job, future you, your colleagues, and everybody that comes after you, and you're no good at git, by all means, shit out 300 commits, don't squash, and have the PR message be totally useless. Also believe you're hot shit after one semester of boot camp and that no one else cares just because you want to go home, get high, and play xbox. (Not remotely saying that's you, but those people are out there.)

We could get all philosophical and try and answer the question of if there are any universal truths, nevermind universally best git commit practices.

I don't work where you work, don't know your team, or anybody's skill levels on it, so I'll just close with a couple thoughts. the tool is there to work for you, so learn to work with it not against it. Git bisect is your friend. And that it really sucks 4 years later to be confronted by totally useless commit messages on inappropriately sized commits (too big or too small) and have to guess at things in order to ship fixes to prod (on the prototype that was supposed to get thrown away but never did) and just hope and pray that you've guessed correctly.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#143

Post by Dylan16807 »

MichaelZuo wrote: post #135
rcoveson wrote: post #134

Earlier quoted context omitted.

Speaking from a place of long-term frustration with Java, some compiler authors just absolutely hate exposing the ability to hint/force optimizations. Never mind that it might improve performance for N-5 and N+5 major releases, it might be meaningless or unhelpful or difficult to maintain in a release ten years from now, so it must not be exposed today.

That seems valid for customers expecting a warranty or support. But they should allow it if customers waive all such in writing.

Warranty and support specifically for that flag? Because I don't see how general warranty and support requires keeping any hint flags forever.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#144

Post by lmm »

amelius wrote: post #51
lmm wrote: post #32

Earlier quoted context omitted.

> git-bisect works best when every commit works, contains a single idea, and stacks in a linear history. That works best in a publicly visible branch, and is why it is helpful to squash an entire pull-request into a single, atomic commit — one which clearly defines the change from before- to after-this-feature. Disagree; git-bisect works best when every commit is small and most commits work (in particular, as long as…

Can't git-bisect simply ignore commits with a commit message smaller than N characters?

Yes - or, being more principled, you can tell it to only use mainline commits (first parent) if landing on a whole PR is what you want.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#145

Post by GeekyBear »

keanb wrote: post #118
cratermoon wrote: post #115

Earlier quoted context omitted.

Or Win 3.1 looking for whatever shibboleth was in MS-DOS and popping up a scary-looking message if it found another DOS? https://en.wikipedia.org/wiki/AARD_code

I don’t think anybody remembers this since that code never shipped in retail.

It didn't ship (in the final retail version) only after the tech press of the day exposed what Microsoft had done.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#146

Post by cratermoon »

GeekyBear wrote: post #145
keanb wrote: post #118

Earlier quoted context omitted.

I don’t think anybody remembers this since that code never shipped in retail.

It didn't ship (in the final retail version) only after the tech press of the day exposed what Microsoft had done.

It did ship in the final retail version, in way. It was disabled, but the code was still there, and a flag was all that was needed to enable it.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#147

Post by lmm »

gorgoiler wrote: post #37
lmm wrote: post #32

Earlier quoted context omitted.

> git-bisect works best when every commit works, contains a single idea, and stacks in a linear history. That works best in a publicly visible branch, and is why it is helpful to squash an entire pull-request into a single, atomic commit — one which clearly defines the change from before- to after-this-feature. Disagree; git-bisect works best when every commit is small and most commits work (in particular, as long as…

Thanks for responding. Everything you say I agree with. I think our differences lie in the scope of how much of my private activity do I want to share in public. You’re right that GitHub, GitLab et al let you use their tooling to write the final commit message (for the merge commit or squash commit). My preference has always been to do that in git itself. In both cases you end up with a single atomic commit that repr…

> One reason this is particularly useful is that you can now treat the commit as if it had already landed on the main branch. (It is easier to share, cherry-pick, rebase, etc. — easier than doing so with a branch of many commits, in my experience.)

> Prospective changes do not change type (from branch to squashed commit or merge commit) either, when they are approved, which simplifies these workflows.

You can do that even earlier if you simply never squash or otherwise edit history, which is my approach - any pushed feature branch is public as far as I'm concerned, and my colleagues are encouraged to pull them if they're e.g. working on the same area at the same time. It comes at the cost of having to actually revert when you need to undo a pushed commit (and, since cherry-pick is not an option, if you're making e.g. a standalone fix and want your colleagues to be able to pull it in unrelated branches, you have to think ahead a little and make it on a new branch based from master rather than sticking it in the middle of your incomplete feature branch), but it's very much worth it IME.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#148

Post by altairprime »

School-Cotton wrote: post #133
AnotherGoodName wrote: post #120

Earlier quoted context omitted.

Nvidia has a control panel with it's drivers. Open it up -> Manage 3D settings -> Program Settings. Scroll through and see how every single program/game you have installed openly has different defaults in it based on application name. As someone noted above others do the same thing. Eg. Frostpunk has Antialiasing for transparency layers on. Slay the spire does not. I never set these settings. Nvidia literally does a…

Applying different standard settings is pretty different from "hijacking and modifying the rendering loop", though.

In what sense? The render loop is modified from “the” default without user or program opt-in, and “hijacking” is what it would be called if anyone but Nvidia did it — so Nvidia is not exempt from that use. Though: Runtime patch, haxie, hijack, LD_PRELOAD, system extension; the noun changes every few years, so perhaps it’s time for a new one. Override?

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#149

Post by almostgotcaught »

tylerchr wrote: post #50
almostgotcaught wrote: post #24

Earlier quoted context omitted.

Literally no one looks through the individual commits in a PR that's gonna be squashed. I don't care if it's 10 or 10,000 - I'm always gonna review the full thing.

You might be surprised. Yours sounds like the attitude of someone who has not had the luxury of reviewing well-constructed commits. PRs with intentional commits permit both faster and deeper reviews—but alas, not everyone is so respectful of their reviewers’ time and energy.

> Yours sounds like the attitude of someone who has not had the luxury of

Sometimes when people speak rhetorically I'm baffled because I feel they literally do not understand what they're saying because they end up supporting an opposing rhetorical purpose. Yes you're 100% correct well-structured commits are a luxury that most of us do not have the privilege of experiencing because we work in high-pressure, deadlines driven environments where no points are awarded for beautifully crafted commit messages.

So in effect your argument is like "people that haven't had the luxury of a Michelin star restaurant don't appreciate amuse bouche and they should strive to rectify that".

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#150

Post by almostgotcaught »

IshKebab wrote: post #15
kilpikaarna wrote: post #7

Keeping it real with the commit msgs

I think it's fine if you squash it. I have no idea why they didn't squash it before pushing to GitHub though.

@Mogball Mogball enabled auto-merge (squash) 3 months ago

@Mogball Mogball merged commit ade3d49 into main on Jul 9

Post reply on HN