I'm not entirely sure. Yes, you need a mountain of data to train a system very well to be flexible. However, once that training is done, you absolutely do not need 'cloud' resources whatsoever. You could run an already-trained NN of any sort on an embedded processor running on tiny batteries. They require extremely little processing to actually push some input through the network and get the output. The only reason we continue to send all of our data to cloud companies to have the speech recognition performed there is because those companies have perverse incentives to snoop and spy and profile and target aggressively.
Technically speaking, all of these 'voice assistants' and the like would be far better products if they hosted a local pre-trained network that did all of the recognition. Latency is the biggest challenge to these systems, and that simply will never be solved so long as the recognition happens on a server miles away from where the user is speaking. The speed of light, at a minimum, comes into play.
What would be very interesting would be if there was a development of a continuously learning system which actually performs better training itself to recognize one or a few users voices without the burden of carrying the weights necessary to also recognize the voices of people with accents on the other side of the planet from anyone who will ever be within earshot of it. That would be an even more overt and active disincentive to sending all your data away "for speech recognition" (the spying is a fringe benefit!) and sitting around waiting for the cloud to get back to you.... and I imagine we'd see it ignored.
The limitations today aren't technical, they're organizational and business oriented. And those things don't generally change in concert with technical changes.