Failure to communicate is reason enough to call it quits. I'm having the same problem with my cofounder. He refuses to work via phone calls--it has to be email only. I'm in the process of withdrawing my financial investment from the company and leaving to start my own version of the same company.
he wants to have in written everything you say so he has stuff on you. in other words, he doesn't trust you. otherwise, a follow up email with notes about phonecall agreements would be more than enough.