There is a Village Global podcast episode where Eric Schmidt is asked this question [1]. He basically says that leadership did not "get" messaging platforms. I think of this as similar to how RIM/blackberry did not "get" the iphone. Nor did Steve Ballmer. It's equivalent in screwup magnitude to Microsoft not getting mobile. This requires massive leadership oversight.
Now, having worked at Google during these times (including the Allo launch where the team went around Google trying to rally the troops on why it was going to work), I can say that the PM structure at Google meant that a ground up led effort to build a competitive messaging platform just wouldn't happen. PMs basically rotate products every few years and nobody really has passion for the thing they are working on it. Its a very mechanical role oriented around features, roadmaps, growth bookkeeping etc. None of these things would yield a competitive messaging platform without a mandate from the top.
Now why was there no mandate from the top? Well, I think that leadership was confident that Google already had what it needed to build messaging into their dominant platforms. Between youtube and Android it felt like that if the competition in messaging were to grow in feature set, it would become more like these platforms (facebook turning into youtube with video, etc). Google could then add messaging and meet them at some point in the future. Schmidt also references youtube as a messaging/social platform in the podcast episode.
[1] https://podcasts.apple.com/us/podcast/village-globals-ventur...