It certainly seems probable this was the case for some groups. In general though, this just seems like a view that oversimplifies human history to critique the present rather than a detached description of what we can know.
A few things AFAIK anyone has to grant about this period:
- Pre-agricultural hunter-gatherers lived all over the world, under vastly different climates, for ~100,000 years.
- These humans were anatomically modern in every sense. They had lives every bit as complex as ours.
- Human cultural, political, and social structures are and have always been inherently diverse.
- Humans have always impacted and managed their environments for better and for worse.
- The Neolithic Revolution occurred independently in multiple places over generations as a series of choices by individuals at least roughly as intelligent as we are.
- Humans who adopted agriculture came to out-populate those who didn't.
The idea that hunter-gatherers lived consistently affluent lives and enjoyed plenty of leisure time as a general rule doesn't neatly fit this picture for me. How is it more likely than the idea that these people lived basically evenly along a spectrum of fluctuating and diverse conditions, at the mercy and grace of natural systems and social trends?
Perhaps depending on the context some human groups lucked into a life of luxury, while others lived painful lives consumed by the anxiety of dwindling supplies, all as an accident of climate patterns, the spread of disease, or even human-caused overconsumption.
Even in early societies that won a Garden of Eden in the geographic lottery, what's more human than to invent new complex problems to stew over based on generational trauma, or simply wild speculation about a world we will always have limited understanding of? Perhaps some group of humans highly valued their downtime hobbies while others were obsessed with arbitrary hierarchies, wars to obtain slaves, or settling petty disputes between powerful families.
Agriculture at the very least provided predictable trade-offs that smooth out the previous extremes, and we can't know if on balance it was a strictly negative or positive change. Since then, however, I think it's safe to say that the lives of everyone I know is better off than a practitioner of early agriculture.