Fairphone 4 has been my main and only phone for several months now. Great experience, very good battary life. Performance-wise, I haven't noticed any significant difference compared to traditional phones. The only drawbacks might be the camera and thickness, but honestly I don't care much. Of course, much depends on your use case and expectations. YMMV.
Is it possible to swap batteries on the go? That'd be killer. I only replaced my phone this year (S7 edge) because the battery was depleting too fast.
You cannot swap it "live" though, phone needs to be shut off and then booted.
Also, there's no external loading hardware. So some logistics needed to get the spare batteries full.