I hear what you're saying, but I think you're making a few assumptions that aren't necessarily always true or guaranteed.
As to the obviousness of sending hashes to a server, I can think of a dozen ways to make that look like routine API data that's getting synced during a routine polling process. The app is a black box until data hits a networking layer, and it can be made to look like anything they want before it gets there. Would they need to be acting deviously to accomplish this? Of course. Is it in these companies' nature and history to act deviously? You bet it is!
As for the complexity of matching without anyone noticing, I think you may be overestimating the amount of data needed to accomplish this task. They wouldn't need to record full conversations or phrases, or even have any context at all. As you and others have noted, they already have so much contextual information about us, that all it would take is successfully capturing a few words once in a while to really fine-tune they're targeting.
Some very simple audio processing could easily chop up "probable" words based on any number of factors. It could throw out any recording that didn't meet some minimal requirements or seemed noisy. And at that point, you're just sending some strings to the server occasionally, no longer than the typical cookie.
And again, they already brazenly do this with Messenger and WhatsApp and everything else in the world that isn't Signal. So, that it's both technically possible and they have already invested heavily into such systems, makes this all a real possibility. At the very least, it's something that should continue to be investigated and actively monitored for, because even if the biggest of the companies aren't doing it, that doesn't mean nobody is doing it -- or doing it on their behalf.