The key to the statement is the "a lot of our lives are just a little worse" at the end, which is an argument that doesn't work for floppy drives, serial ports and, arguably, CD drives, as they were phased out well after commonly-used replacements were well into the mainstream.
The use-case for the 3.5mm headphone jack is still well and truly within the mainstream, and will probably stay there for some time as per the comment in the blog:
the right pair of headphones was the pair of headphones in front of you, or in your pocket, or mangled at the bottom of your bag. It didn’t matter.
There's a lot of legacy-weight hanging around in people's houses, bags, and workplaces.
Your point, however, isn't completely lost on me. I still miss CD drives occasionally, having to retrieve an older laptop in order to grab data from a disc I haven't yet bothered to archive to an ISO file. However, removing a CD drive from a laptop feels like a much higher percentage space saver than losing the 3.5mm jack from a smartphone.