This is widely considered to be a mistake.
This is widely considered to be a mistake. Citations, please? I've been able to deal with it over the years.
The Heritrix crawler (primarily worked on by the Internet Archive) introduces a "surt" form which is basically the domain in the same order as the path so that Reading from left to right it goes from least specific to most specific.