Good article, but couple remarks.
> most hardware seemingly just runs workgroups in a serial order
The hardware runs them in parallel, but it’s complicated.
The nVidia GPU I’m currently using has 32-wide SIMD, which means groups of 32 threads run in parallel, exactly in lockstep. Different GPU APIs call such group of threads wavefronts or warps. Each core (my particular GPU has 28 of these) can run 4 of such wavefronts = 128 threads in parallel.
When a shader has more than 128 threads, or when the GPU core is multi-tasking running multiple workgroups of the same or different shaders, different wavefronts will run sequentially. And one more thing, the entire workgroup runs within a single GPU core, even when the shader pushes workgroup size to the limit with 1024 threads per workgroup.
“Sequentially” doesn’t mean the order of execution is fixed, or predefined, or fair. Instead, the GPU is doing rather complicated scheduling trying to hide latency of computations and memory transactions. While some wavefront is waiting for data to arrive from memory, instead of sleeping the GPU will typically switch to another active wavefront. Many modern CPUs do that too because hyperthreading, but CPUs only have 2 threads per core, they are visible to OS as two distinct virtual cores. For GPUs the number is way higher, only limited by amount of in-core memory, and amount of that memory required by the running shaders.
> as the difference between running a shader with @workgroup_size(64) or @workgroup_size(8, 8) is negligible. So this concept is considered somewhat legacy.
I think it’s convenience, not legacy. When a shader handles 2D data like a matrix or an image, it’s natural to have 2D workgroup sizes like 8x8. Similarly, when a shader processes 3D data like a field defined on elements or nodes of 3D Cartesian grid, it can be slightly easier to write compute shaders with workgroups of 4x4x4 or 8x8x8 threads.