Intel's New Chimera: Alder Lake

Variable-width is the new hotness but it doesn’t work well on all tasks. If your task is fixed-width, or the algorithm changes based on task width, you kind of need to know the hardware width. And shuffling, perjuring, swizzling, and bit-shifting don’t translate very well to these approaches. Nor is there really an easy way to operate these in a fixed-width mode.

https://gist.github.com/zingaburga/805669eb891c820bd220418ee...

It’s not all bad but the people who just say “just treat it like an n-element array and let the hardware handle it bro!” are hand waving an enormous amount of algorithms that may not actually be translatable to that.

And broadly speaking, having a couple code-paths and choosing at runtime is not that bad. HPC is used to working “close to the metal” and tuning their code to run optimally. You can’t make that complexity go away - things will always run better on X hardware but Y hardware runs that code path way worse. In this domain all you do with clever auto-programming magic is obscure the problem - you are now writing for the optimizer instead of writing for the hardware, but different hardware still runs differently and you need multiple code paths to hit that optimally. So now you have two problems - keeping the hardware running optimally and keeping the optimizer from messing you up.

Since you are now dispatching large operations that will not complete atomically in one cycle (they simply won’t, you can’t do 2M elements on a 128-bit vector) it doesn’t seem likely that we will stay strictly in-order on these, and the next step is you start extracting parallelism from the stream and that’s the problem that immediately emerges from that. Now you have an optimizer running a SIMT program (opmasks are basically SIMT) extracted from the instruction stream, and you need to not just write code that runs fast - you can’t - you need to write code that the optimizer turns into code that runs fast, and that seems like absolute hell on an architecture where you can’t even know the number of registers-per-thread in advance. Maybe I am reading this all wrong but that’s where you’d really end up going with a “run this op on an N element array” model. If you run one vector-instruction at a time you are in cache/register hell constantly spilling to memory and back, so you want to optimize that to keep things in-register/in-cache, and that means extracting parallelism from the stream to run more stuff while it’s still somewhere reasonably hot.

This is the exact garden-path that GPU drivers went down and it was a huge mistake with OpenGL/DX9/DX11 that had to be walked back with the DX12/Vulkan APIs to get back closer to the metal, because the optimizer stage became completely inscrutable and managing it to keep it from doing something stupid became impossible.

There is a level of irreducible complexity in HPC and exposing the hardware is the best way to avoid it. Leaky abstractions make things worse.

Writing for a phone is not HPC but if you care about energy efficiency you still need to make sure you’re running reasonably optimally and not wasting cycles, especially with a super wide vector unit. That means you’re leaning heavily on the “sensible” code path having decent efficiency, which means you’re leaning on the hardware to behave sanely. So it’s the same thing.

Again, not saying it’s a completely bad idea, but you can already see the abstraction starting to leak with the various types of operations that don’t really work on a variable-width hardware concept, and that’s worrying. There is a lot more opportunity for this to go bad than I think people are at first glance.

I've not used columnstore in MSSQL but in fact I likely know more about the innards of MSSQL than you. WRT memory, all I mant was that if your working set doesn't fit in RAM, you're screwed regardless. Nothing will save you, so check that first. I don't know what your 70GB means in this context - please check.

> It then parallelises this across all cores/hyperthreads, so I get 100% utilisation on 16 cores.

that suggests your working set is indeed smaller than ram. Good.

However be aware that the parallelise feature of mssql is much better that it was but is still moderately crap. Disable it and see if you get a 16X slowdown. My guess is you won't, which means when enabled you won't be getting a 16X speedup (also hyperthreading can actually slow things down - disbale it and see what happens). Edit: I'd not be surprised if you were only getting 4X to 5X speedup (I may be wrong but I do have experience in this).

> With 8 cores going at once, this is 64 rows per clock, or up to 234 billion rows processed per second.

(giggle) Oh, I don't think so!

> I've seen "sort spills" reach 600K IOPS on my laptop.

Then you're hitting IO and yer cores are useless. It spills to disk because it's overflowed ram - you may have a problem.

> Learn to utilise it better and you'll be better at thinking.

... agreed, but the reverse is true also.

edit 2: Depending on what you're looking for, and if you do it often, check if MSSQL's free-text indexing can index it, if so you may get a far bigger speedup from that than anything else. Maybe. Depending.

I may be way off here, but to me the transition to heterogeneous cores is more of a long term than short them play, anticipating the era where we'll have different kinds of cores in our systems.

Even the performance cores of today have circuitry that's there to make certain workloads faster (video decoding, crypto operations, etc.). Other stuff has mostly been offset to the GPUs (actual grahphics-related stuff, machine learning).

In an architecture where you can mix/match different kinds of cores, it doesn't sound crazy to have cores that are specially designed for those tasks. Processors could ship with one "crypto-core", a "video-decoding-core", etc.. These should provide similar performance to full performance cores at _much_ lower cost, energy consumption and die area.

If (and that's a big if) they can get to a point where:

a) Designing/implementing new specific-purpose small cores is not outrageously expensive because the whole coordination/integration part is already solved

b) They have a combination of cores that provides similar performance to current p-cores-only cpu's at similar processor cost and power budget.

That would be a huge win for processor companies. They would be able to open lines of specialized processors commanding huge premiums for certain industries (think about processors with 128 ml-cores, 128 video-encoding-cores, etc.).

Or maybe I'm just delusional :)

I remember Mono, a .NET runtime, having to change its allocations to fit inside a smaller core. This was because a thread could start on a bigger core but then be shunted onto a smaller core.

That fixed the crashing but obviously came with a loss of performance.

> Using them means the work gets done for less power and thus less heat.

It also avoids scheduling conflict between housekeeping / background tasks and "user" stuff, which means more predictable and stable throughput and better cache utilisation (your cache doesn't get blown because the system needed some time to zero memory or update something).