Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Helion: A high-level DSL for performant and portable ML kernels

pytorch.org

Post reply on HN
Print view View original

21–30 of 52 posts

Re: Helion: A high-level DSL for performant and portable ML kernels

#21

Post by westurner »

embedding-shape wrote: post #7
brap wrote: post #3

Asking as someone who is really out of the loop: how much of ML development these days touches these “lower level” parts of the stack? I’d expect that by now most of the work would be high level, and the infra would be mostly commoditized.

> how much of ML development these days touches these “lower level” parts of the stack? I’d expect that by now most of the work would be high level Every time the high level architectures of models change, there are new lower level optimizations to be done. Even recent releases like GPT-OSS adds new areas for improvements, like MXFP4, that requires the lower level parts to created and optimized.

How often do hardware optimizations get created for lower level optimization of LLMs and Tensor physics? How reconfigurable are TPUs? Are there any standardized feature flags for TPUs yet?

Is TOPS/Whr a good efficiency metric for TPUs and for LLM model hosting operations?

From https://news.ycombinator.com/item?id=45775181 re: current TPUs in 2025; "AI accelerators" :

> How does Cerebras WSE-3 with 44GB of 'L2' on-chip SRAM compare to Google's TPUs, Tesla's TPUs, NorthPole, Groq LPU, Tenstorrent's, and AMD's NPU designs?

Re: Helion: A high-level DSL for performant and portable ML kernels

#24

Post by saagarjha »

uoaei wrote: post #14

Tangential question related to the example kernel: in GPU programming is it idiomatic/standard to initialize the out array as zeros rather than empty? are the performance savings negligible?

It saves a kernel launch and memory bandwidth for a fill kernel. If you’re going to overwrite the data anyway, why bother?

Re: Helion: A high-level DSL for performant and portable ML kernels

#25

Post by saagarjha »

bobajeff wrote: post #10

It's good to see more effort for making things not device specific but I only see benchmarks for NVIDIA B200 and AMD MI350X. Also what's the experience of using one of these Python DSLs like? Are the tools good enough to make code completion, jump to definition, setting breakpoints, watching variables, copying as expression etc. nice?

Generally you are unlikely to get Python-level debugging for code that is going to run on GPUs.

Re: Helion: A high-level DSL for performant and portable ML kernels

#26

Post by porridgeraisin »

uoaei wrote: post #14

Tangential question related to the example kernel: in GPU programming is it idiomatic/standard to initialize the out array as zeros rather than empty? are the performance savings negligible?

They have made it empty only.

>> out = torch.empty([m, n], dtype=x.dtype, device=x.device)

The accumulator has been initialized to zero, since well, they have to add stuff into it.

>> acc = hl.zeros([tile_m, tile_n], dtype=torch.float32)

> idiomatic

No as far as I have seen they generally try to not initialize if its not necessary.

> overhead

There is the memory bandwidth point as you might expect. But additionally when using high level interfaces like pytorch, when you write torch.zeros(512, 512) in pytorch, it launches a whole kernel (tens of micros) just for that line. So that's cpu -> gpu -> back to cpu, and then it does the next line, where it goes to gpu again and uses that memory. So in these cases you make sure to avoid it if its in a hot path. Ideally you want the 2nd kernel to do the initialization itself. When you write cuda c++ yourself this is how you typically do it. Helion being a compiler might be doing this optimization, but runtime based torch can't clearly.

Re: Helion: A high-level DSL for performant and portable ML kernels

#29

Post by porridgeraisin »

ozgrakkurt wrote: post #15
the__alchemist wrote: post #13

Earlier quoted context omitted.

Even for non-ML things like chem simulations: CUDA (and cuFFT) are more pleasant to use than Vulkan Compute and vkFFT.

I just learned the graphics api of vulkan, can’t imagine anything being less pleasant than vulkan

Yeah it's quite something. If anyone wants a preview, here's the triangle hello world in vulkan: https://gist.github.com/Overv/7ac07356037592a121225172d7d78f...

But then again, I've heard that it's this low level because its meant for engine developers.

Re: Helion: A high-level DSL for performant and portable ML kernels

#30

Post by saagarjha »

krapht wrote: post #19
bwfan123 wrote: post #17

I dont get the point of helion as compared to its alternatives like gluon. For best performance I would presume one needs low-level access to hardware knobs. And, these kernel primitives are written one-time and reused. So, what is the point of a DSL that dumbs things down as a wrapper around triton.

Funny, I feel the same way about Triton. Performant Triton looks like CUDA (but with tiles!) except it's ten times harder to debug since it doesn't have the tooling NVIDIA provides. If I had to run on AMD I'd rather deal with their hipify tooling.

Performant Triton programs are usually simpler and shorter than their CUDA equivalents. This alone makes it easier to write, and I would argue that it helps with debugging too because the model provides a lot more guarantees on how your code executes. That said, some of the tooling is notably poor (such as cuda-gdb support).
Post reply on HN