The circumstances in which you actually need to physically present your Social Security card are quite rare indeed. In fact, that is the first thing I was told by the ssa dot gov site, when I applied to receive a new card.
It is very unlikely that you will be asked, even by a prospective employer, for your actual card, because let's face it: it doesn't even have a photo, or anything but that unique number on it. Anyone trying to authenticate your number should be satisfied if you can give them the correct number.
And just for review: SSNs are not a "national ID" and you're typically not required to divulge it to private parties, and they can make up some other unique ID for you in their database. It's usually just a shortcut for them to do a background or credit check on you. And that's not something for which they would need your physical card.
That being said, I've replaced my card about 3 times now, and it was comparatively difficult this time around. The first time, I did it all through the mail (the process of building up from no ID to get birth certificate from out-of-state, to the SSA card, to the in-state driver license.) and the second time it arrived by mail, no hassle.
But this time around, even though I applied online, I was directed to make an appointment at the field office and physically walk in there, to prove my humanity. I had never been to an SSA Field Office in my life! The experience was very chill, and there were a dozen windows serving people, while about 4 of us waited in the lobby, and I was in-and-out half an hour early. The civil servant was a lady in good spirits who was a military veteran. Big props to them!