The hatred is specifically directed at "FP" which I understand to be floating-point. Makes sense inside an OS kernel but a large majority of HPC would indeed consider this irrational.
I understand that dark silicon is helpful, but am not so sure that fixed-function HW is the way to go.
Perhaps video _de_coding is the most convincing from your list; codec generations are 5+ years, so enough time to benefit from HW.
Encoding, on the other hand, tends not to be impressive unless perhaps there is also a software component.
For the rest, programmability and deployability (can we rely on it?) is a major issue. Software has often been the limiting factor.
Another big concern is the 'hardware lottery'. The algorithms we develop and get are selected for, and tuned to, the current hardware. Perhaps this gets us 5x energy efficiency vs CPU/SIMD. But by painting ourselves ever further into the corner of dense linear algebra, which is definitely not the way that nature implements intelligence, we are missing out on far larger opportunities. For example: spiking nets or memristors have the potential to be 2 or 3 orders of magnitude better. Or actual sparsity, not the fixed-pattern thing (now that is a prime example of an irrelevent benchmark, because AFAIK algorithms haven't yet been able to use them well).
> A general purpose CPU however should specialize on unpredictable memory access.
Should it really? I think rather we should avoid such accesses whenever possible, because their energy cost now dwarfs that of computation.
> This means AVX-512 is somewhat misplaced on a CPU and probably only exists because it served Intel to create nice numbers in irrelevant benchmarks.
I have difficulty understanding how a reasonable person can come to such a conclusion.
Lemire (the author linked here) has a long series of results showing nice speedups from AVX-512. I personally have seen gains in image compression, string processing, cryptography, linear algebra, integer coding, hash tables, databases, sorting, and compression.
[Opinions are my own.]