Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everyone should know SIMD

mitchellh.com

Post reply on HN
Print view View original

121–130 of 263 posts

Re: Everyone should know SIMD

#121

Post by loup-vaillant »

Rendello wrote: post #38
tarnith wrote: post #27

Earlier quoted context omitted.

Yeah, data layout/cache aware layouts are really key if you really want to unlock making something that ends up in a hot loop fast with SIMD. Also, avoiding allocations or vtable lookups or a lot of indirection in the part of the code that's actually "hot" is really important. Vectors (in C++) at least aren't necessarily the best fit either, if you end up doing anything that can call an allocation unexpectedly.

> Vectors (in C++) at least aren't necessarily the best fit either I'm not sure if you use a different allocation strategy or if you're advocating allocating as much as possible up-front, but I'm curious if you have any thoughts on this: I always end up using (Rust) vectors despite looking at a bunch of slab/arena allocation libraries. Preferably I'd know how much memory I need up front, but barring that I see three…

A fourth alternative, in 64-bit systems, is to reserve a stupidly large chunk of memory up front with `mmap()` or equivalent (`malloc()` actually should work about as well). That way you guarantee that any extension will happen in place. There’s a limit to how much you can reserve, but since that limit is much higher than what you can actually use, you can make quite a few of those reservation before you run out of address space.

It feels dirty, but when your system has overcommit you can’t reliably check that the memory you want is actually there anyway, so you might as well reap the benefits.

Re: Everyone should know SIMD

#122

Post by nasretdinov »

kristianp wrote: post #30

Tangentially for Go programming, the last time I looked at optimising some Go code with SIMD there were a few different options available, but they were either not maintained any more or had incomplete support and required first writing your function in C++ with intrinsics and generating assembly, then converting it to go assembly with a tool [1]. I never got my function to work in go despite the C++ code working fin…

It actually works really well in the last couple Go versions with GOEXPERIMENT=simd. You do get a similar speedup (if not higher, since SIMD also eliminates the penalty for bounds checking and other things Go runtime does.

Re: Everyone should know SIMD

#123

Post by pjmlp »

peterashford wrote: post #105

I've been using the Vector API in Java to get some massive speedups for flowfield generation. There's no guessing with that approach - if the hardware supports SIMD, you get it.

Now with Valhala finally getting merged, he can hope the end of preview releases for the Vector API is coming to an end, probably it will take at least until Java 28, though.

Re: Everyone should know SIMD

#124

Post by pjmlp »

rao-v wrote: post #85

It distresses me that we don’t have a language that can do a best effort parallelization of arbitrary loop like code across SIMD, multiple threads, multiple cores and GPU with a small directive. I don’t need it to be optimal, just … handy as an option! The last time I brought this up here, folks offered a bunch of options that don’t quite do this, and the best candidate was this 15 year old compiler project that is I…

Chapel aims to do that, however they only focus on HPC as userbase.

Re: Everyone should know SIMD

#125

Post by vasco »

teo_zero wrote: post #113

Good article! I just wouldn't start off with bold sentences as > SIMD can be simple to understand and > writing SIMD is just about as easy as a for loop and then the first example requires 12 lines to replace one line of scalar code. Be honest and say SIMD is hard but the results are worth it! (Another nitpick: if this article is for newbies, don't use SIMD-only words and concpts before explaining them. Step 5 is goo…

The lines in SIMD are much simpler though. It's like replacing one line of "I have a desire for yellow nourishment from the tropics" to "give banana" 12x

Re: Everyone should know SIMD

#126

Post by pjmlp »

tbrownaw wrote: post #104
rao-v wrote: post #85

It distresses me that we don’t have a language that can do a best effort parallelization of arbitrary loop like code across SIMD, multiple threads, multiple cores and GPU with a small directive. I don’t need it to be optimal, just … handy as an option! The last time I brought this up here, folks offered a bunch of options that don’t quite do this, and the best candidate was this 15 year old compiler project that is I…

> best effort parallelization of arbitrary loop like code across SIMD, multiple threads, multiple cores and GPU with a small directive. I doubt GPU is included by most runtimes yet, but for the rest of that have you tried SQL?

Good answer, too many folks miss out how powerful SQL actually is, and with stored procedures its compilation to native code can even cached across executions.

Re: Everyone should know SIMD

#127

Post by pjmlp »

bigwhite wrote: post #59

The Go language has long lacked official support for SIMD instructions, which means it has been at a disadvantage in terms of performance optimization. In recent years, with Go 1.26, an experimental version of the SIMD/ArchSIMD packages was introduced for AMD64 architecture. With Go 1.27, a portable version of the SIMD package was also added. Now, we can fully utilize native SIMD instructions to optimize go program p…

I think too many people get dismissive of reaching out to Assembly in other languages, while in C and C++, having to reach out to Assembly to do exactly the same is seen as an advantage versus other languages.

https://github.com/kelindar/simd

https://github.com/viant/vec

However, having it officially supported is definitely much more convenient.

Re: Everyone should know SIMD

#128

Post by pjmlp »

boricj wrote: post #46

Everyone doesn't need to know SIMD. Mechanical sympathy is an important passive perk for software architects to cut down the number of reworks down the line, but I would rate benchmarking and being able to identify bottlenecks as more important everyday skills. I'm working on a voxel space renderer homebrew for the PlayStation. I only have so many cycles to spend on rendering before it becomes a slideshow, so I count…

> ....it's whether the performance requirements are met or not ...

This is what so many people miss when doing micro-benchmarks of language X vs Y, sure Y might win out in execution speed, however if X delivers within the performance requirements and has a lower development cost, it wins out while being slower than Y.

Naturally taken to the extreme, when it isn't our hardware is how we end up with Electron apps.

Re: Everyone should know SIMD

#129

Post by pjmlp »

forrestthewoods wrote: post #3
qurren wrote: post #2

I just do gcc -O3 and get SIMD without having to learn it

auto-vectorization is not nearly as good as you would hope it to be. The best SIMD optimizations likely require changing your data format from AoS to SoA.

While C++ may be reaching levels of Algol 68, PL/I complexity, with C++26 reflection you can do automatically.

See https://github.com/cern-nextgen/reflmempp

Re: Everyone should know SIMD

#130

Post by mb2100 »

saagarjha wrote: post #92
losvedir wrote: post #44

This is an interesting article. I don't really work with low level enough languages for this to matter (unless - does this ever show up in Javascript somehow?). I guess I don't understand the "reduce" step. It seems like you have to be careful not to "undo" all the benefit from SIMD. Sure, it can compare 8 values in parallel, but then if you have to look at each of the 8 answers in turn you're back to where you began…

It can show up in JavaScript if you write your code in a way that lets the browser engine lower it to SIMD under the hood

how do I do that? :-)
Post reply on HN