There are a ton of problems with hand gestures. One, they're hard to see and blocked by line of sight (voice isn't.) Two, they require hands to be free to communicate (not always the case, and in situations of survival it's probably not doable at all.) Three, they're limited versus voice. Voice has soft and loud and more sounds can be made than variations of hands. More meaning can be ascribed to a sound.
The first is an issue: can only be looking at one thing at a time. The second doesn't seem to come up much in practice. Can eat and sign at the same time unlike voice. Third isn't true -- signing is more expressive because it takes advantage of space, facial expressions, and signs can be modulated to adjust the meaning. But it almost has to be since the symbol rate is lower than speech.
If you're being attacked or in a fight, do you really think you'll have a hand free to do sign language?