Email has a lot of warts besides its obvious graces, especially when you look at the back-end of how emails are actually stored and transferred and authenticated. So I think we as programmers are naturally drawn to the idea of developing the email-killer. The central problem with the dreamers seems to be that they're not dreaming deeply enough.
Email is a communications multitool which leverages the whole human subconscious to effect its various uses: a human is supposed to know whether this is a "your boss demands that you do this" email or a "your friend wants you to look at her cute cat" email or "your uncle is ranting about fluoride in the water again" email. Most email killers do not and cannot eliminate this mental overhead. Graham only says "other people should be able to add to my todo list" -- he's not killing email, he's considering an email-aware todo list. That's very common. Skype is not an email killer precisely because it doesn't dream big enough: it is a realtime messaging protocol which hit it big because it worked with video. Trello is not an email killer precisely because it's for organizing groups. RSS is not an email killer because it only solves the limited problem of keeping aware of new content.
Email does all of these tasks -- poorly, but it does them. If you want to really kill email, you need to offer the core functionality effortlessly. Email II would understand "circles" of friends automatically, so that there is only an "inbox" for those people who are contacting you for the very first time. Like syndication, Email II would host the content on the publisher's web server to be downloaded whenever the client wants -- this has spam-deterrent effects, since the network doesn't take responsibility for the content. It would hopefully allow us to unify mailing lists. We can protect privacy with automatic cryptography; "chris::example.com" can stand for a public key which is stored on the example.com machines. The email killer needs to be good at segmenting messages so that we can say "This is the text of my invitation to a wedding, but it also comes with a standard-appointment-form where I describe where and when it is, so that it can be rapidly added to your 'upcoming dates' in the relevant social circle." (Since this must merge with dates and todo-lists, presumably example.com contains its own list of stuff which chris::example.com has stored for his own use.)
And I want to at this point admit defeat: because I think even those ideas are not daring enough. The email killer is probably going to have to do even more than all of that, and my own imagination becomes limited.