Plastic separation is tricky, they tend to have similar physical properties so you end up relying on small density differences or optical properties or computer vision which is obviously not so great. I think most places it's still more cost effective to do it by hand.
Metals are easy to separate from one another, they tend to have substantially different densities and electrical properties. Aluminum for instance is not magnetic while steel cans are, but aluminum is conductive so if it passes through a magnetic field it will still get slowed down.
Those are the two big ones in any case. Then you could get to things like nickel (magnetic but intermediate conductivity) or tin (light but non magnetic) or maybe zinc/magnesium, but basically the densities are more different and the properties are more different so it's easier to tell them apart. And for some things that it can't figure out it makes sense to just landfill it.
Plastics though, absolutely vastly harder. You can use density but it only varies by a few percent. You can use diffraction gratings and cameras to see crystals and maybe sort a bit by that. But really it's just very hard, especially since the same polymer has different molecular weights and crystallinity. Your milk jug is milky while the same molecule processed differently is clear.
But even that's a mess -- you can't generally "depolymerize" these things very easily, you are really melting them back together and reusing the melt. When you start from monomers you can control the molecular weight (chain length) which controls things like crystallinity and stiffness, so the quality is higher. If you melt a variety of different molecular weights back together even if they're the same exact molecule the physical properties will just by worse. It's just the nature of the beast, and it sucks.
Even if you recycle by perfectly melting all the PET with perfect sorting and no food residue you'll be guaranteed to increase the variability in the physical properties of the output. It's just chemistry, unless you can convert it back to monomers it'll never be as good even in the best case.
For metals at least you can count on a fairly full recovery if you invest in it, it's not like polymers where it's just not possible. Even with alloys like steel you can at least in principle go back to the raw materials if you wanted, even if they are quite tricky.