upvoted you. So, this is really important and let's look at this very carefully. (I'm not a historian.) Obviously you mean to imply that he had just spare time for his papers only, and that from our point of view 8 hours were taken as irrelevant non-work (if we consider the science to be the work), so that perhaps something like 3 were left for physics.
But is that accurate? If we look at the article I linked on that miracle year, it says: "He worked as an examiner at the Patent Office in Bern, Switzerland, and he later said of a co-worker there, Michele Besso, that he 'could not have found a better sounding board for his ideas in all of Europe'."
I'm not trying to "rebut" you, let's understand what really happened. Did Einstein really do irrelevant clerical work that was time he simply lost from his day? Or did he really not only slack off during his job, and spend some time doing physics on company time, but even have his coworkers' input into it? i.e. get extra resources from his day job?
I am genuinely curious, but I think that it is probably the latter! What I mean is that suppose you take away his job at the patent office and his connection with his colleagues there. Instead you rewind history and just I don't know make him win some kind of lottery or give him the same money that he made but from some other source, or you make him a mailman or something, but at any rate you take away any positive direct contribution from his position.
What is your personal opinion about whether the four papers would still have resulted? Given everything I know about working on private projects on company time, or the history of Bill Gates at Harvard or a number of other similar stories... and given the direct quote... I think it quite likely that his 8 hour "office clerk" job in fact contributed to his physics research.
It would have been a purer example if he were a chauffeur or something. But given the quote, I think perhaps we should consider even those 8 hours as contributing to his physics research.
It is a real shame that we can't ask "what would have happened if he hadn't been working as a patent clerk."
On balance I think it supports my theory that it takes 24 hours to do something. You raise a good point however and I'd be interested in a historian's perspective. This is an extremely interesting venue to ask about. I wonder if anyone wrote about his work habits during those couple of years?