Earlier quoted context omitted.
Is that a failure of the people or failure of the spec?
Failure of people and tools. Like the sibling comment says, often tools designed to consume HTTP will throw a more severe class of error upon 4xx/5xx, which then changes the codepath significantly, making it more difficult for the programmer to treat 2xx and 4xx/5xx responses similarly for parsing out the headers and the response body. Web browsers are also guilty of some of these behaviors. They would inject a 'user…
I once had to work on such an API, which was designed that way because of similar limitations in the (third-party?) iOS networking library used by the app that connected to the API (as it was in 2013, anyway).
One can see how thinking of an HTTP request as an RPC rather than a REST request/response would justify throwing an error instead of returning an object, but since REST is now more common than RPC, one would at least expect an option not to throw.