The few podcasts I listen to, I do so in 0.2x speed or even less. Not by slowing the playback speed, obviously, but by pausing and rewinding very often to think about what has been said and even take notes if I find something that resonates with me.
Most podcasts though I listen to at infinite speed, meaning I don't listen to them at all. You can go through them 2-3x speed because there's really nothing there.
It's like code: if it's boilerplate you can just skim it but if it's really doing something you have to read it slowly multiple times. And just like the saying: nothing of value is easily gained.
If you find yourself speeding though things - in general, not just podcasts - I would ask myself if they're worth doing at all.