But, and this is the frustrating part, you're applying a terrible double standard there. With nuclear you're focusing on the tiny part that is hard to deal with. In solar you're ignoring the heavy metals [0]. By mass, there will be at least comparable heavy metals to the amount of dangerous nuclear waste. Probably more, given how fuel-lite nuclear can get.
And with nuclear a risk of cancer sees people believing that this stuff needs to be locked in containers for all of time while with solar panels if a few people get silicosis over that time frame you're going to do the rational thing, shrug and say "oh well". Over all of human history, the risk of solar panel waste killing a bunch of people indirectly is basically a certainty. It is a big world and, I repeat, this waste is with us forever. Or if we admit that it will be recycled as an argument ... so will the nuclear waste, in all likelihood. The stuff is an energy source. It'll be like when they figured out that oil was good for fuel back when it used to be a toxic waste product.
We don't need to isolate nuclear waste forever. We can cordon off a dump somewhere for the nasty stuff and make plans to deal with it for 300 years. Then move on with life. Odds are good it will be feedstock for a nuclear plant at some point in that timeframe.
To be clear, I don't mind that we're ignoring the negligible costs in solar. But they aren't a problem in nuclear either. If we agree to ignore negligible risks (like we do for literally every other industrial process), all of these objections start looking a bit silly. The opponents of nuclear are correct that the risks can't be reduced from where they are, but the risks are already lower than comparable sources of energy.
[0] https://www.abc.net.au/news/rural/2022-01-17/are-some-solar-...