It doesn't matter what the boss thinks, it matters what the contract says.
I don't know what "refusing to make exceptions" means - what power do they have to make exceptions? Even if they agreed you could work on something, unless they changed the contract they can freely break that agreement.