Earlier quoted context omitted.
By my reading, the spec is not that clear about it. It does say that A user agent that wishes to authenticate itself with an origin server--usually, but not necessarily, after receiving a 401 (Unauthorized) -- implying, imo, that no prior 401 response is needed. On the other hand, the spec also says A client SHOULD assume that all paths at or deeper than the depth of the last symbolic element in the path field of the…
I don't think there's any ambiguity here...the first quote indicated that a client is absolutely allowed to send auth headers without receiving a 401 response. I read the second quote as giving client implementers a suggestion on when they should automatically send the auth headers, without actually requiring them to do so.
See, for example, the Apache HttpClient documentation: http://hc.apache.org/httpclient-legacy/authentication.html#P...