Early on in digital audio, anti-aliasing filters were awful. The ADCs weren't very good either, but the AA filters were bad. Using a higher sampling frequency was one method to help this problem by providing more padding in the frequency domain so that a more gradual AA filter could be used.
Later, this was completely OBE with the advent of oversampling. Most ADCs in use now are oversampling delta-sigma converters operating a very high sampling rates that perform decimation on the output to provide a 16 or 24 bit waveform at the normal 44.1kHz or 48kHz sampling rate. Delta-sigma converters, and high sampling frequencies are actually the basis for Sony's direct stream digital (AKA Super Audio CD).
Today, you can be reasonably assured that an ADC will provide a very clean, low noise, noise output in the vast majority of cases. For music playback, this never mattered anyway. Whether or not the transport of digital audio is at 44.1 kHz or 1 MHz, the quality to the human ear will be the same as long as it was sampled correctly and accurately.
That said - sampling at 44.1 kHz, 48 kHz, or even oversampling, may not capture all that sound has to offer our ears. One thing that can happen with music for instance is beating. Beating occurs when two notes are sounded at slightly different frequencies (say 20,000 Hz and 20,100 Hz) there will be an audible beat at their difference. Where this can come into play is in close mic recordings. If I record guitar A) with an overtone at 30,000 Hz and guitar B) with an overtone at 30,100 Hz, these two would have an audible 100 Hz beat. However, if we filtered and sampled at 48,000 Hz or over-sampled then filtered, we would lose both overtones and the ability to hear that beating.
How important is that beating? Good question - but with live music it's not a problem and with close mic recordings it is. For recording, there are reasons to use higher sampling rates for at least the mixdown process. I still think it's silly to have 192 kHz audio for listening to recordings at home.
Bit depth on the other hand is something I think we could use more of - especially with classical recordings. Audio quantized to 16 bits only provides about 96 dB of SNR. I would much prefer having 20 or 24 bit audio to fully encompass the actual range of human hearing that is closer to 120 dB.
The other thing that cannot go without mentioning, MP3s, AAC, and other lossy audio formats are pretty good - but they DO NOT compare to lossless audio. Having Google, Apple, and Amazon all step up to 44.1 kHz 16-bit audio sourced from equal or greater source material would be huge improvement over MP3s and AAC.