[flagged]
The One Billion Row Challenge in CUDA
71–77 of 77 posts
Re: The One Billion Row Challenge in CUDA
#72Pretty cool. Shameless plug: My team at Anthropic is hiring people that can write accelerator kernels. Please reach out to @gmail.com if you want to make state of the art models faster :)
Curious, do you guys use Triton? I was surprised to find out PyTorch slowdown is palpable even compared to basic CUDA kernels.
Re: The One Billion Row Challenge in CUDA
#73I have a hard time believing the basic C++ implementation took 16 minutes... I just did the most trivial non-parallel implementation in python and it took less than 5 minute with pypy.
I think the author wanted to compare with a similar base algorithm. They should have stated that, however, instead of implying that a basic C++ algo would typically take that long.
Something else to consider is cost of cloud hardware, though it could be extrapolated from the author's results.
Re: The One Billion Row Challenge in CUDA
#74The query itself it's a perfect hash (assuming the station is dictionary encoded) and takes around 100ms on a gh-200 (the Gpu is a single h100 with 132 SMs) with a query that's concurrency constrained. The same level of performance can be obtained using an Ada like an L40s or and RTX 4090. The transfer across the nvlink connecting the Cpu and GPU on a gh-200 after the parse and encoding of the source CSV takes a negl…
Great context for us GPGPU fans!
Do you think the L4 should have similar perf? Considering it has similar features and specs to L40, and I doubt the lower 300 GB/s bandwidth woudl be an issue.
Re: The One Billion Row Challenge in CUDA
#75Re: The One Billion Row Challenge in CUDA
#76Earlier quoted context omitted.
Good to know. I didn’t know the contest has no limit on hyperthread.
1brc in the contest had SMT disabled [0]. (hyperthreading is an intel marketing name and trademark for their implementation of smt, but the benchmark was run on an amd cpu) [0] https://github.com/gunnarmorling/1brc/issues/189#issuecommen...
Ok. So it's indeed restricted to 8 core (1 thread per core). Then the benchmark above using 16 threads was not really a fair comparison.
Re: The One Billion Row Challenge in CUDA
#77Earlier quoted context omitted.
Yes, GDS will accelerate the IO to the GPU. I’d love to see the above C code compared to hyperoptimized GPU code on the right hardware, but I don’t want to accidentally nerd snipe myself :-) The unfortunate part of this particular benchmark is that once you have the data in the right place in your hardware there is very little compute left. The GPU code would probably have constant performance with an additional coup…
so you'd rather nerd snipe others, gotcha ;) :D
Haha. Apologies. I hope I didn’t accidentally make anyone waste their time. If they did I’m sure there are people interested in hiring such people anyways so maybe it’s a time invested well in the end.