By "probably excessive" I wasn't referring to psycho-acoustics. I merely meant that given your willingess to include speaker->ear latency, many people work on music in scenarios where that measure alone is already close to or above 10msec. The worst case scenario is likely a pipe organ player, who deals with latencies measured in units of seconds. Humans can deal with this without much difficulty - it is jitter that makes it hard (or impossible) to perform, not latency. Long-standing drummer & bass player duos generally report being able to deal with about 10 msec when performing live.
On the flip side, you have people arguing convincingly the comb filtering caused by phased reflections inside almost every listening scenario are responsible for the overwhelming majority of what people as "different". Move your head 1ft ... lose entire frequency bands ... move it again, get them all back and them some!
Regarding latency deadlines: well, the device driver can tell you (and does, if you ask it). If you use JACK, it will callback into your client every time there is an xrun reported by the audio hardware driver. This in turn has a quite simple definition: user space has not advanced the relevant buffer pointer before the next interrupt. There are circumstances where this actually isn't a problem (because the data has already been handled), but it is a fairly solid way of knowing whether the software is keeping up with the hardware. Something using ALSA directly can determine this in the same way that JACK does.
For audio, there is no other measurement of this that really matters. Using some sort of system clock to try to check timing, while likely to be kinda-sorta accurate enough, ignores the fact that the only clock that matters is the sample clock. If you're operating with huge margins of safety, some other clock measurimg time is good enough, but as you begin to inch closer to problem territory, it really isn't. For reference, we generally find that when "CPU loads" (variously measured) get close to 80% on macOS and Linux, scheduling deadlines start failing.
Nothing on linux will automatically "fallback" to less demanding latency requirements. If the system can't meet the requirements of the audio interface, it will continue to fail. This is actually part of the reason why Ardour tends not to deactivate plugins - the user can expect the DSP/CPU load to be more or less constant no matter what they do, rather than being low and then climbing through a threshold that causes problems as they do stuff.