I couldn't agree more. The annoying part is when they ask you to call them so they can verify your identity. It is a safe assumption that they don't do voice recognition, so why do they think I can't just spell the same numbers out loudly I'd have sent in an email...?
No, the annoying part is when they call you and immediately demand you verify your identity to someone without even any Caller ID. (And my but they get shirty when you use the appropriate language which the situation requires when telling them to get off the line.)
Imagine then the correspondence the same bank sends in various forms about keeping the same information to yourself to prevent your account being compromised.