Earlier quoted context omitted.
>While most GPUs support FP64, unless you pay for the really high-end scientific computing models, you're typically getting 1/32nd rate compared to FP32 performance. I wonder if there is a hardware reason for this or It's just market segmenting by nvidia.
My naive guess is that most floating point code uses FP32 and FP64 uses at least double the die size. So optimize for FP32 and have some FP64 for the rare equations that need it.
I can be wrong about who did that first, but most FPUs now are done like that.