Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding SIMD: Infinite complexity of trivial problems

modular.com

Post reply on HN
Print view View original

81–90 of 127 posts

Re: Understanding SIMD: Infinite complexity of trivial problems

#81

Post by imtringued »

janwas wrote: post #76
dragontamer wrote: post #34

Earlier quoted context omitted.

It's not the same. AVX2 instructions haven't changed and never will change. In contrast, NVidia can go from 64-bit instruction bundles to 128-bit machine code (96-bit instruction + 32-bit control information) between Pascal (aka PTX Compute Capacity 5) and Voltage (aka PTX Compute Capacity 7) and all the old PTX code just autocompiles to the new assembly instruction format and takes advantage of all the new memory ba…

I hope people aren't writing directly to AVX2. When using a wrapper such as Highway, you get exactly this kind of update after a recompile, or even just running your code on a CPU that supports newer instructions. The cost is that the binary carries around both AVX2 and AVX-512 codepaths, but that is not an issue IMO.

Most video encoders and decoders consist of kernels with hand written SIMD instructions/intrinsics.

Re: Understanding SIMD: Infinite complexity of trivial problems

#82

Post by imtringued »

benchmarkist wrote: post #47
stouset wrote: post #46

Earlier quoted context omitted.

All we have to do is ascribe magical properties to AI and we can solve anything as if P=NP!

Those distinction are irrelevant for an AI because it is a pure form of intelligence that simply computes answers without worrying about P or NP complexity classes.

That's the same as being confidently wrong.

Re: Understanding SIMD: Infinite complexity of trivial problems

#83

Post by janwas »

imtringued wrote: post #81
janwas wrote: post #76

Earlier quoted context omitted.

I hope people aren't writing directly to AVX2. When using a wrapper such as Highway, you get exactly this kind of update after a recompile, or even just running your code on a CPU that supports newer instructions. The cost is that the binary carries around both AVX2 and AVX-512 codepaths, but that is not an issue IMO.

Most video encoders and decoders consist of kernels with hand written SIMD instructions/intrinsics.

Agreed. FWIW we demonstrated with JPEG XL (image codec, though also with animation 'video' support) that it is possible to write such kernels using the portable Highway intrinsics.

Re: Understanding SIMD: Infinite complexity of trivial problems

#84

Post by kolbe »

ashvardanian wrote: post #25
EVa5I7bHFq9mnYK wrote: post #12

C# vectors do a great job of simplifying those intrinsics in a safe and portable manner.

There are dozens of libraries, frameworks, and compiler toolchains that try to abstract away SIMD capabilities, but I don't think it's a great approach. The only 2 approaches that still make sense to me: A. Writing serial vectorization-aware code in a native compiled language, hoping your compiler will auto-vectorize. B. Implementing natively for every hardware platform, as the ISA differences are too big to efficien…

I still like ispc, but that's not going to catch on.

Re: Understanding SIMD: Infinite complexity of trivial problems

#85

Post by andix »

rishi_devan wrote: post #64

Interesting article. The article mentions "...the NumPy implementation illustrates a marked improvement over the naive algorithm...", but I couldn't find a NumPy implementation in the article.

Yes, they are really great at abstracting the SIMD operations, but the abstraction has only very few common methods. I'm not sure how much real world benefits those abstractions have.

Once you need more complex operations, you need to use the specific operations from System.Runtime.Intrinsics.(X86|ARM) based on the current architecture. And you need to adjust your implementation on the CPUs capabilities. There are still a lot of older x64 CPUs around that don't have AVX512 for example.

Re: Understanding SIMD: Infinite complexity of trivial problems

#86

Post by alfiedotwtf »

ashvardanian wrote: post #59
dragontamer wrote: post #20

Earlier quoted context omitted.

I mean, 288-E Core Xeons are about to ship. Xeon 6900 series, right? (Estimated to ship in Q1 2025) So Larrabee lives on for... some reason. These E cores are well known to be modified Intel Atom cores and those were modified Xeon Phi cores which were Larrabee based. Just with.... AVX512 being disabled. (Lost when Xeon Phi turned into Intel Atoms). Intels technical strategy is completely bonkers. In a bad way. Intel…

Yes, a lot of weird decisions were made at Intel. Ironically, AMD waited so long to implement AVX-512, but now has it on both server and mobile chips (natively and 256 bit emulation, respectively). Intel started the whole thing, has a very fragmented stack and is now preparing those E cores with even more new extensions. Most importantly for Search and AI, it adds AVX_VNNI, which can be used for faster 8-bit integer…

A former Intel CEO even wrote a book where every product was planned 20+ years in advance.

Imagine planning 20 years in advance where Moore’s Law is still going strong. Come to think of it, Moore was also CEO of Intel lol

Re: Understanding SIMD: Infinite complexity of trivial problems

#87

Post by secondcoming »

Conscat wrote: post #27
juancn wrote: post #13

The main problem is that there are no good abstractions in popular programming languages to take advantage of SIMD extensions. Also, the feature set being all over the place (e.g. integer support is fairly recent) doesn't help either. ISPC is a good idea, but execution is meh... it's hard to setup and integrate. Ideally you would want to be able to easily use this from other popular languages, like Java, Python, Java…

I think the EVE library for C++ is a great abstraction. It's got an unusual syntax using subscript operator overloading, but that winds up being a very ergonomic and flexible way to program with masked-SIMD.

I’m not sure about EVE. I trialled it by trying to uppercase a string and even though I got it working in the end it was quite unpleasant. Their docs need to be better.

Re: Understanding SIMD: Infinite complexity of trivial problems

#89

Post by ashvardanian »

big-chungus4 wrote: post #88

can the authors please share the numpy code too

There are several ways to implement it in NumPy, often resulting in 20% variance. I've added a reference implementation to my mirror of the blogpost and the Modular team will soon update the original posting as well: https://ashvardanian.com/posts/understanding-simd-complexity...
Post reply on HN