We don't assume everyone should be working. Some categories everyone accepts shouldn't be working: children, students, retirees, new parents, the sick, and the disabled. There's a widespread assumption that you should eventually save enough money to retire, which divides life into phases: working and retired. That's one reason behind feeling like you should accumulate more money: not so you can buy more stuff, but be…
There are plenty of people who do not accept this.
We had to outlaw child labor because lots of people thought children should work. Students are shamed for taking on debt to pay for their schooling. People think they should have to work their way through school. Retirement is coming to be seen as a luxury. A great deal of jobs in America don't provide any kind of parental leave, so clearly some people think new parents should work. Same thing with sick leave. People go to work sick every day because that's what the system demands of them.
The only category that is even close is disabled people, but even then it's not universal. Goodwill thinks disabled people should work, and they have a legal exemption from minimum wage so they can employ them. There are also lots of people think the disabled should just be exterminated because they can't work.