Okay, after rethinking it once a gain I will move a bit towards your position. By increasing the required work for one X from 8 to 10 hours we are definitely introducing an inefficiency so we are losing something somewhere, we are doing 2 hours of unnecessary work. Assuming both agents can share the X or get to use half the X in both scenarios, we can just leave it out of the equation as mentioned before. If one gets the same value from X in both scenarios, then working 5 hours is strictly preferable over working 8 hours and not working is strictly preferable over working for 5 hours.
So this looks somewhat like the prisoner's dilemma. Let A and B be the scenario that agent A respectively agent B works 8 hours and AB the scenario that both agents work 5 hours each. Then the preferences for agent A are B > AB > A and for agent B they are A > AB > B. But that doesn't help much, no matter which transition you consider, one agent will be better off and one will be worse off.
So there is still a conflict. I acknowledge that an inefficiency gets introduced when the work is split but also want to maintain that 3 hours of leisure time twice is more valuable than 8 hours of leisure time once. Surly value can not increase and decrease at the same time. I am more certain that the inefficiency is real than what the shape of the value function for leisure time is so I will pick the former one and conclude that 8 hours of leisure time are - contrary to my earlier comment - more valuable than 3 hours of leisure time twice.
So do I have to say that maximizing the total value is the wrong metric? Do I have to try to assign a value to the fairness of work distribution? Why wouldn't that already show up in the individual value functions? I am a bit stuck here. But I may have a way around that, I am just not sure how good the argument is. One obvious problem is that in case agent A does all the work, then agent B has no means to transfer some of the additional value from his additional leisure time to A. Maybe he could give away some of his share of X.
But now I can no longer ignore the value function for X and that hopefully saves my argument and I avoid the conflict between decreasing and increasing total value without the need to explicitly appeal to the value of fair work work distribution because it is in the two value functions for X and leisure time. But I am not sure, I did not think this fully through.
I think in the end it boils down to nonlinearities between the individual and the global view, i.e. we can globally waste 2 hours of work but still increase the total value because we have to take the global set of things, split it up for all individuals, push them through the nonlinear value functions and then sum it up again. Now it is no longer necessary that a global loss of 2 hours results in a global loss of value even if all the individual value functions are monotone.