Earlier quoted context omitted.
> Solar panels use rare earth metals This is a falsehood, one that Shellenberger was notorious for spreading. No, solar panels do not use rare earth metals. That you repeat this well-debunked lie suggests you're getting your talking points from an echo chamber, not reliable sources.
Sorry, you're quite right that they don't use rare earth metals -- I was thinking of the Cadmium telluride and CIGS combinations used in some thin-film solar panels (which aren't rare earth metals -- it's been a while since I've done chemistry), but looking at it some more it seems like they aren't toxic (even though the constituent elements are). My bad (though I don't know who Shellenberger is.) However, since the…
The vast majority of PV sold is crystalline silicon. The only toxic element used there is lead in backing sheets, but even that is being phased out (in part for environmental reasons, and in part because higher efficiency versions of Si cells can't tolerate the heating needed to bond to the lead; in any case there is nothing essential about the lead.) The only rare element used is silver for front contacts, but that could be replaced by copper with the use of a very thin layer of nickel or molybdenum to prevent reaction of the copper with the silicon.
The environmental impact from PV is largely from the mundane components, like steel, aluminum, and glass. PV to power the world would use these materials in small quantities compared to that used for global industrial society as a whole (global annual production of steel is 2 billion tonnes, for example). If that society deals with the impact of its overall non-energy activities, it could deal with the impact of building and maintaining a PV energy source to power it.