The cynic in me sees this as a further attempt at lock-in.
As you mentioned, when the other technologies were abandoned, a suitable replacement existed that was widely in use regardless of platform. With the headphones, buying a set for an iPhone will now no longer work with other non-Apple devices (at least, at the moment -- are they licensing this to others and are any others interested in lightning headphones?).
I have two friends, recently, who destroyed their iPhone 6 (toilet and cement drop) and opted to purchase a higher-end Android device as a replacement (slightly less expensive with slightly better specs and they liked my phone when I let them play with it). Had they owned a pair of $200 "pretend" high-end headphones[0] that would have only worked with an iPhone, I doubt they would have been so quick to switch. I guess that could work the other way -- preventing Android users from switching to Apple, except it's resolved by a dongle on the Apple side without loss of perceived quality.
Wireless isn't a suitable replacement, either. Every time I see a new set of excellent Bluetooth wireless headphones, I think of the drawer full of the previous BT headphone/headset devices I'd bought and the several that aren't there because I've lost them. Usually something fell off of the device or the battery stopped holding a charge as long as the phone (when I remembered to charge those, too). The AirPods are small so they end up in the pocket opposite my phone with other things[1], most likely, my keys. They appear durable enough to survive in that environment, but it'll take about a week before I pull my keys out (I've only got 3 keys and a couple of rings), one snags on a ring and falls onto the ground without me noticing. Welp, I'm done, off to the drawer you go, mono-Pods.
I lose wired headphones much less frequently. They're usually plugged into my phone (and recent phones hold them tight) and wrapped inconspicuously around my neck. If I pocket them and they snag on my keys, I notice. But the best part is if I lose them, I can find a pair that suites my needs for $15 or less. Something tells me the Lightning headphones aren't going to be that cheap for a long time[2].
[0] "Pretend" high-end headphones refers to the likes of "Beats by Dre" and many others that are little more than the "Air Jordans" of headphones. You pay five times the price of similarly capable headphones because it's got a logo of a perceived "bad ass" on it.
[1] Because, at least where I live, wearing them in your ears when your not using them is going to get you labeled a "Douche Bag" pretty quickly.
[2] A quick search yielded several with the "basic features" I'd be looking for at around USD$50.00. Yikes.