"Leisure" might not mean exactly what we think it means today. I think it'll be obvious to anyone who's worked in a manual labour job e.g. construction, farming, etc, even with today's tools, you can't physically do these at anything but a "leisurely pace" and a limited amount of hours a day, with breaks for rest, food etc. Specially in warmer climates. Claiming that this means that life was easy is a little bit dish…
Yeah, sure, but how many construction workers work only 150 days a year and have mid-day naps?
Construction work can be similar in places where harsh winters make it impossible to build anything during winter (frozen ground and all that).
Farming can be similar.