Earlier quoted context omitted.
> GPU of my desktop PC computes 11 TFlops, that's 4+ orders of magnitude faster. But they’re much lower quality (less flexible) FLOPs. If you don’t have much data parallelism or heavy conditional logic for each datum, you’re not going to come close to 11 teraflops.
If you don’t have much data parallelism or have heavy conditional logic, you won't come close to 1GFlops on that CPU either. These 2 FLOPs/cycle are for SSE i.e. 4-wide vector math without any conditions or branches. Update: also, for other tasks, GPU flops are of higher quality than CPU. VRAM bandwidth is ridiculous, much higher than system RAM, computations that are bound by RAM bandwidth will be much faster. GPUs…
Using double precision SSE scaler ops, the Pentium III could execute one addition and one multiplication per cycle. (The throughput was the same with vector math because the Pentium III only had 64-bit SSE units. So a 128-bit packed multiply and a 128-bit packed add, four double precision operations, executed over two cycles.)