Buyers beware: 4-core A53 is genuinely unusable (original Pinebook/PinePhone specs), A55 is better but I still wouldn’t recommend buying. You may expect performance similar to 15+ years old desktops.
User of Librem 5 phone with this CPU here. It works fine as a daily driver. Firefox with Noscript works fine in desktop mode. See also: https://puri.sm/posts/the-danger-of-focusing-on-specs/
Have you noticed something similar? do you listen to music on your L5?
Edit: I should also mention it does this regardless of audio out the internal DAC+3.5mm TRS, or over a USB MBox attached to a dock. It seems to be something quite low-level disabling interrupts for too long, something like that.