I witnessed a poor chap at the ticket counter at Dulles airport desperately trying to convince someone to flag him a taxi to drive him to Dallas to make his meeting.
My wife and I learned the hard way that Dulles is NOT in Washington DC, we took the bus out to the Air and Space Museum, which we assumed was in DC... the $75 dollar taxi ride that took 1 1/2 hours to show up to pick us up was no fun. Fun Bonus Fact: We drove to DC, so we could have just driven there, but thought a cheap Bus round trip was more fun as we could both see the sights, and not stress about traffic.
You visited the udvar-hazy center which is out by Dulles airport in Chantilly. It is still part of the smithsonian, but more focused on displaying lots of different aircraft in a few large hangers. Very impressive and worth a visit, but not the main air and space facility.
Either you had a legitimate miscommunication, or you got scammed by your driver. No one from around the DC metro would confuse the two.