In my eyes, nuclear weapons are both much less and much more terrifying that they seem to be considered by most people.
The smallest nuclear devices aren't anywhere close to as large as commonly believed. An M28 "Davy Crockett" with a yield of 20T - 0.02kT - isn't that much different from a conventional GBU-43/B MOAB, which has a yield of 11T.
Tactical nuclear weapons are basically a faster and more effective version of conventional strategic bombing. "Little Boy", the first weapon used in combat (in Nagasaki), had a yield of 15kT. That resulted in an estimated 66k deaths and 70k injuries. Compare that to the firebombing of Tokyo, which killed ~100k civilians and burned the homes of over a million more.
Strategic nuclear weapons... they're in a whole other category. Obviously lots of people are killed during conventional strategic bombing, but most of the damage is ultimately done through fires set by the destruction of the intended targets. Some people have a chance to escape.
What's more, conventional bombing is WW2 wasn't generally a "one-night" affair; it took days or weeks to saturate a target to the point of neutralizing it, and after the first couple of attacks many people would have left the target area. The firebombing of Tokyo resulted in so many civilian casualties precisely because it was a (very effective) one-night event, and people didn't have a chance to flee. That was exceptional even in WW2.
Strategic nuclear weapons are effectively instant. They're incredibly powerful. We stopped building bigger ones not because we didn't know how, but because we couldn't see any reason to. If a 50MT blast won't do the job, a 500MT blast isn't going to either... so why spend the money to develop, create, and maintain bigger ones?
Finally, the idea that even a full nuclear exchange between major powers would be an extinction-level event is absurd. It would utterly destroy the countries involved, devastate the world economy, and poison huge swaths of the planet practically in perpetuity. Between the direct and indirect damage and the societal impacts on the remainder of humanity, it would set us back centuries as a species - but we would rebuild and it would take much less time to do so than it did to get to where we the first time.