Earlier quoted context omitted.
If the thinking machine rejects my mortgage application, it should be possible to point out which exact rule triggered the rejection. With rules explicitly set by an operator it's possible. It's also possible to say that the rules in place comply with the law and stay compliant during the operation, for example it doesn't unintentionally guess that I'm having another citizenship based on my surname or postal code.
If the mortgage application evaluation system is deterministic so that the same input always produces the same output then it is easy to answer "Why was my application rejected?". Just rerun the application with higher income until you get a pass. Then tell the person their application was rejected because income was not at least whatever that passing income amount was. Maybe also vary some other inputs to see if it…
It is simply that, eg., "on some historical dataset, this boundary most relibaly predicted default" -- but this confers no normative reason to accept or reject any individual application (cf. the ecological fallacy). And so, in a very literal way, there is no normative reason the operator of this model has in accepting/rejecting any individual application.