Do phones not filter out/ignore any sound being emitted from the speaker? Seems like that shouldn't be too hard to do.
Can you really distinguish sound emitted from the speaker from someone with a hoarse voice? Furthermore, what about medium? Traveling different medium should be considered.
Since the output is known, similar input can then be stripped. This only works when both the output of the speaker and input of the microphone are known.
This can't be done to determine whether another speaker, such as a TV, generated the output.
Visual aid on echo cancellation: http://i.imgur.com/m2LSIz9.png