Drones are nothing of not noisy, and that noise is unlike almost any other piece of machinery. It shouldn't be too hard to set up towers around the airport with sensitive speakers that can triangulate the location of a drone flying within a restricted area. From that point it can sound an alarm and/or deploy counter-measures.
Small drones are pretty quiet, especially once they reach altitude. It would be difficult or impossible to pick one up over the noise of wind, cars, lawn mowers or leaf blowers.
You say that like it's impossible, but drones have a very particular pitch. If the Navy has sonar capable of finding submarines thousands of miles away in a very noisy ocean, there's technology that can do this.
Maybe it's not inexpensive, but it's at least proven to exist.
