Earlier quoted context omitted.
Leave the button recessed after it's been selected, even if the LED fails (how often does this actually happen?) it's still a visual and tactile indicator. To deselect the floor, simply press in again.
Remove the buttons entirely and replace with a QRcode/NFC thing where each rider can vote/unvote where he personally wants to go. This can also work outside before the elevator arrives so a routing system can decide to bundle up a bunch of people going to the same floor in one express elevator. Haven't thought out how the ux would look to keep the wrong people out of that one elevator.
My local pizza place plays music based on customer voting through a smartphone app. It's not necessary to be there in order to vote - you just need the app.
This elevator design could be fun in similar ways.