Live data from Hacker News

Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

github.com

91–100 of 176 posts

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#91

I work with compilers And despite it not being nice, some optimizations rely on type or function names schemas/substrings/etc It sucks, but thats how it works It doesnt have to be malicious just sometimes it is safer to deploy optimization only for your libs than risk breaking stuff Or your frontend is not giving you more data which you can rely on

It is probably not malicious, but it certainly does create new barriers, which is not a good thing.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#92

Sometimes you write some heavily tuned code in a high level language like C++ that you know could be translated into very specific GPU assembly, then find that the compiler isn't producing the exact assembly that you had in mind. When you talk to the computer team about it they may offer a range of solutions, some of which may not be applicable to open source code. Picture proprietary #pragmas, intrinsics, or whatnot…

Why wouldn’t you just use inline assembly in that case?

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#93

Earlier quoted context omitted.

Claude, look at this git history, analyse diffs and create an intelligent commit message to replace each commit message. Do a rebase to fix it all up.

Would you actually do that? It's information destruction. You can machine generate at any time, but you can only delete the human input once

? I assume you want to replace your jubberish messages with something more useful before pushing? It is only "destroying" https://xkcd.com/1296/ style crap? Code changes stay the same.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#94
post #33

Earlier quoted context omitted.

> you can only delete the human input once git branch "backup/$(git branch --show-current)/$(date +%s)" # do whatever you fancy git reset --hard "backup/$(git branch --show-current)/${thattimestampabove}" You can't lose anything as long as you have a pointer to it (which doubles as making it easy to find)

No need to make a "backup" branch. Learn to trust the reflog.

That is like learn to trust the indestructibility of matter. I can still lose (not able to locate...) my keys even though they still exist!

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#95
post #68

Earlier quoted context omitted.

Thank you for explaining. I was so confused at how AMD was improving Quake performance with duck-like monikers.

Well, if it _looks_ like a high-performance texture renderer, and it _walks_ like a high-performance texture renderer...

It's probably been duck typed

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#96

It would be nice if we could find economics that allowed us to share code instead of all the bullshit with the binary blob drivers. Same for basebands and everything else. How many collective hours and months of our society’s finest minds has been wasted reverse engineering binary blobs, controllers through IO pins, trying to reverse engineer circuit schematics —- when all of this is already sitting on someone’s comp…

The problem is as follows: You have a fixed cost investment to produce a software code base, then you have fixed ongoing maintenance costs, at a minimum one developer who knows the codebase. Preferably two for a commercial product. On top of that you have small distribution costs over time. E.g. servers that host the software downloads.

The marginal costs per user are very small or even zero for desktop applications. This means that software needs a funding structure with periodic payments, but at the same time the payments shouldn't grow with the number of users. There also needs to be a way for the initial investors who pay for the creation of new features or entire code bases to get their money back as the product becomes popular.

This in itself is not problematic, but it is not covered by traditional crowdfunding. The problem is that the funding goal needs to be met no matter what, and the contribution per user shrinks as more users contribute. You can't expect everyone to chip in 100%, 10% or even 1% of the funding cost, since that could be thousands of dollars even at the minimum. You need some sort of auctioning process where people can pledge a fixed quantity and if the user count is low enough, their pledge counts, otherwise it doesn't.

This has one problem though. What's problematic is the transition from the exclusive to non-exclusive mode.

There will be freeloaders who might pitch in five dollars, but they know five big corporations have chipped in and this covered the full development cost, leading to open sourcing the entire codebase. Everyone else is a freeloader. Including cheapskate corporations.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#97

Sometimes you write some heavily tuned code in a high level language like C++ that you know could be translated into very specific GPU assembly, then find that the compiler isn't producing the exact assembly that you had in mind. When you talk to the computer team about it they may offer a range of solutions, some of which may not be applicable to open source code. Picture proprietary #pragmas, intrinsics, or whatnot…

Why wouldn’t you just use inline assembly in that case?

There is no way to write inline SASS (assembly equivalent) for CUDA code. You can inline PTX, but PTX is a high level bytecode designed to be portable.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#98

Earlier quoted context omitted.

what kind of AI are you using that generates shitty commit messages? This a common kind of message from Claude / Augment: Fix dynamic channel list by passing auth via metadata - Pass userId and userEmail in metadata when calling HTTP transport - AuthenticatedToolsProviderFactory now reads from context.metadata - Each tools/list request creates a fresh ToolsProvider with authentication - Execute command description no…

It is missing the (to me) most important part. The reason why these changes are made.

Set a PR template up, that demands those sections are filled in. Could probably do that down to the commit level with pre-commit but realistically you'd want that level of detail in the in the PR. Also add issue id to the commits too, that way you can pull them up easily and get more context.

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#99
post #17

Heh. Does anyone remember when almost 25 years ago ATI (AMD) caught manipulating the Quake III benchmarks by renaming the executables to ‘quack’? https://web.archive.org/web/20230929180112/https://techrepor... https://web.archive.org/web/20011108190056/https://hardocp.c... https://web.archive.org/web/20011118183932/www.3dcenter.de/a...

This is weirdly common; phone chipset manufacturers did it with phone benchmarks [0], VW with emissions [1], nVidia did it with 3DMark [2], Intel with the SPEC benchmark for its Xeon processors [3], etc. When it comes to computer graphics, iirc it's pretty normalized now - graphics drivers all seem to have tweaks, settings, optimizations and workarounds for every game. (As an aside, I hate that I have to link to arch…

> When it comes to computer graphics, iirc it's pretty normalized now - graphics drivers all seem to have tweaks, settings, optimizations and workarounds for every game.

Even Mesa has them: https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/uti...

Re: Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

#100
post #90

Reminds me of when I was working on NVIDIA Jetson systems, and learning how to use them, that you can run 1 command to make everything go faster... ( https://jetsonhacks.com/2019/04/10/jetson-nano-use-more-powe... )

[dead]
Post reply on HN