It's so clunky, IMO, because bluetooth is a dumbass protocol with things in the standard that should not be there (including which audio codecs are supported with which levels of bluetooth). Rather than just being a more simple network of wireless devices, it's a very complex protocol which makes everything more complicated.
Why you may struggled could be anything from the firmware blob for your bluetooth device, to the kernel driver installed, to bluez, to the sound server you are using. Any one of those things messing up will lead to a bad experience.
I've had a relatively good experience with kde-plasma's bluetooth management stuff. But I still have to do dumb things like manually selecting which audio codec to use when I go on a call.
How could bluetooth be better? It should be at least 2 standards. 1 defining the wireless data transfer and network capabilities, a second which defines how a computer negotiates with a device to send audio. It shouldn't be 2 standards merged together like it currently is. Wifi Direct is more what bluetooth should be.