The problem is that for a lot of these drugs the greatest harm comes from the lack quality control and regulation of sale.
Not just because it results in drugs getting cut with all kinds of shit, and people being sold the wrong thing (e.g. selling heavily cut fentanyl instead of heroin), but also because of a the race to find alternatives that slip through cracks.
A great deal of modern drugs only exists to circumvent the law, either because they're not covered for a while, or because they can more easily be produced. E.g. a number of "synthetic cannabinoids" have been manufactured, and so far indications are that at least some of them have health effects that are far worse than anything possible to tie to actual cannabis.
Without decriminalising and regulating manufacture and sale, there will still be an incentive for dealers to sell drugs like that which nobody particularly want, and that increases harm, instead of selling clean versions of the safest drugs.
E.g. for opiates, many of them are "close enough" in terms of effect that there is a lot of potential harm reduction benefit just in getting the more dangerous variants off the market by legalizing and regulating the safest ones. Unlike blanket bans that has a hope of working.