61–63 of 63 posts
for LLM work, reading the Flash Attention and vLLM kernel source taught me more than any book. real code makes memory hierarchy concrete — books stay too abstract.
Earlier quoted context omitted.
I wish there were any good literature on GPU Direct RDMA and GPU NetIO. Got any tips?
So I would say the most important thing is that the APIs these are using as in mlx5 DevX (essentially direct fw access) or ibverbs are exactly the same regardless if it's CPU or GPU talking to it. So with that in mind the source of rdma-core, DPDK, ucx etc may be the most elucidating when it comes to low level details. For higher level patterns again the APIs are the same so anything building on libibverbs or aforeme…
Earlier quoted context omitted.
Writing efficient CUDA code is very, very difficult; most CUDA code is not actually good at utilizing the hardware. It is much easier to write performant code in higher level languages (and most people are doing exactly this).
That all depends on what you’re doing. Like I said, if a high level lang or lib supports and fits your goal well, then yes you should use it. I don’t know what most people are doing, but it’s fair to say that a lot of people can use a higher level language. If you’re trying to learn CUDA, then using a higher level language is not the best approach. If you already used a high level language and found that your perform…