Earlier quoted context omitted.
x86-64 is such an ABI improvement over x86-32 that it's always 10-20% faster, even with the larger size pointers. Win64's ABI is a little worse than other platforms for libavcodec, since it has some callee-saved SSE registers and everyone else is caller-saved, but still true. Hardware decoding is much more important for power savings than anything else.
Most standard video decoding code uses CPU feature detection and hand-optimized ASM for important bits, so x86-64 doesn't really add anything worthwhile for most video players and encoders - most of the magic is done in SSE/AVX codepaths anyway.
I think astrange knows what he's talking about considering he wrote chunks of those encoders and decoders.