I'm anti-incarceration for moral reasons, but it's not clear to me that there is a definitive consensus that incarceration does not work. See:
https://www.nber.org/digest/jan02/w8489.html> "Annual expenditures of approximately $10 billion on drug incarceration almost pay for themselves through reductions in health care costs and lost productivity attributable to illegal drug use, even ignoring any crime reductions associated with such incarceration."
(The above is from a study conducted by a prominent U Chicago economist and another economist who is now at Princeton.)
It’s really very complicated. One of the macro trends over the last few decades is a massive increase in crime from the late 1960s to the mid 1990s, then a decrease since then. Incarceration started increasing in the 1970s and 1980s as a response to that increase in crime. When crime started coming down in the 1990s, incarceration started coming down about ten years behind that in the 2000s.
Nobody really knows why crime started dropping in the 1990s. Some people think that the reduction is attributable to the banning of leaded gasoline. Maybe. But I don’t think studied have ruled out the notion that the reduction is attributable to putting a large number of people in prison. See: https://pubs.aeaweb.org/doi/pdf/10.1257/089533004773563485
> Crime fell sharply and unexpectedly in the 1990s. Four factors appear to explain the drop in crime: increased incarceration, more police, the decline of crack and legalized abortion.
It’s not an area where you can point to a definitive consensus, and everyone interprets the data to fit their own political views.