Everyone should know SIMD • Hacker News Board

Re: Everyone should know SIMD

#81

Post by Scene_Cast2 » Thu, Jul 23, 2026, 2:52 AM UTC

Rendello wrote: post #24

I like SIMD, but before super-optimizing your code with SIMD and the like, really consider your data structures and access patterns. I've been singing Data-Oriented Design's praises, so I'll just collect all my comments here [1], but I think it's a good approach to optimization. I played around with SIMD in my old code (in Zig), but my approach to modelling datastructures was so antithetical to optimization, it was l…

Very much agreed. Even more basic than that - memory access patterns are important. The amusing thing is that you end up writing GPU-style code even for CPU. For example - instead of an array of objects, using parquet-style object of arrays is one such trick.

Re: Everyone should know SIMD

#82

Post by jwgarber » Thu, Jul 23, 2026, 3:05 AM UTC

The last few days I've been using AVX-512 to optimize matrix operations in a bioinformatics project, and it's great! The bottleneck in most applications is reading the large dataset from memory, so rather than doing it multiple times to compute multiple operations you can do everything in one pass (fused kernel) with AVX registers. 5x speedups are quite common. I've been doing it with manual intrinsics, but the wide crate also makes common operations completely trivial. Highly recommend checking it out.

https://docs.rs/wide/latest/wide/

Re: Everyone should know SIMD

#83

Post by applfanboysbgon » Thu, Jul 23, 2026, 3:05 AM UTC

Jtarii wrote: post #67

[flagged]

No kidding. I saw no less than five "the compiler will do everything for me, I trust in the magic" comments before I gave up reading through the thread.

Re: Everyone should know SIMD

#84

Post by lz400 » Thu, Jul 23, 2026, 3:14 AM UTC

Isn't the better abstraction here to use a higher level library in the style of pandas/polars that will operate as vectors, compose and feel readable and inuitive, while (almost?) maxing out SIMD?

Re: Everyone should know SIMD

#85

Post by rao-v » Thu, Jul 23, 2026, 3:14 AM UTC

It distresses me that we don’t have a language that can do a best effort parallelization of arbitrary loop like code across SIMD, multiple threads, multiple cores and GPU with a small directive.

I don’t need it to be optimal, just … handy as an option!

The last time I brought this up here, folks offered a bunch of options that don’t quite do this, and the best candidate was this 15 year old compiler project that is Intel specific!

https://ispc.github.io/

Could some programming language nerd build this?

(While you are at it give me a clear idiomatic way to pay the cost to switch from array of structs to struct of arrays)

Re: Everyone should know SIMD

#86

Post by dang » Thu, Jul 23, 2026, 3:41 AM UTC

Jtarii wrote: post #67

[flagged]

"Please don't sneer, including at the rest of the community." It's reliably a marker of bad comments and worse threads.

https://news.ycombinator.com/newsguidelines.html

Re: Everyone should know SIMD

#87

Post by cobbzilla » Thu, Jul 23, 2026, 3:42 AM UTC

If you’re processing N things at a time and your “scalar tail” is N-1 why can’t you put in a dummy value for the last entry, run one last SIMD iteration, and discard the dummy return value?

Re: Everyone should know SIMD

#88

Post by saagarjha » Thu, Jul 23, 2026, 3:44 AM UTC

luaKmua wrote: post #78

Rendello wrote: post #24

I like SIMD, but before super-optimizing your code with SIMD and the like, really consider your data structures and access patterns. I've been singing Data-Oriented Design's praises, so I'll just collect all my comments here [1], but I think it's a good approach to optimization. I played around with SIMD in my old code (in Zig), but my approach to modelling datastructures was so antithetical to optimization, it was l…

This is my eternal battle as a perf engineer. Performance starts with architecture and you can only squeeze so much out a hotpath with poor data layout. The nice part is that data-oriented code almost always easily supports threading and SIMD.

On the flip side a lot of my work as a performance engineer is undoing bad abstractions made by people who read a two blog posts about SoA and decide that encapsulation is stupid

Re: Everyone should know SIMD

#89

Post by saagarjha » Thu, Jul 23, 2026, 3:45 AM UTC

rao-v wrote: post #85

It distresses me that we don’t have a language that can do a best effort parallelization of arbitrary loop like code across SIMD, multiple threads, multiple cores and GPU with a small directive. I don’t need it to be optimal, just … handy as an option! The last time I brought this up here, folks offered a bunch of options that don’t quite do this, and the best candidate was this 15 year old compiler project that is I…

The problem is you need both a PL nerd and a performance nerd and while that group has some overlap so these people are not as uncommon as you’d think the task is pretty hard so you need a lot of people on it, with a bunch of funding, etc. Usually it’s just cheaper to rewrite all your code by that point and so these efforts fail

Re: Everyone should know SIMD

#90

Post by saagarjha » Thu, Jul 23, 2026, 3:47 AM UTC

cobbzilla wrote: post #87

If you’re processing N things at a time and your “scalar tail” is N-1 why can’t you put in a dummy value for the last entry, run one last SIMD iteration, and discard the dummy return value?

You can, but usually the code may have problems with this. For example, storing out of bounds is often going to give you a bad time. Some platforms that are all SIMD all the time will support masked operations for this kind of thing.