Mozilla has DeepSpeech [0] and, while not as advanced as the stuff from Google or Amazon, my experimentation left me feeling pretty hopeful that it could reliably recognize at least keywords.
The Raspberry Pi is quite capable though you'll probably need some dedicated microphone to reliably catch voice data. I know ReSpeaker [1] but maybe some off the shelf conference USB microphones would work as well.
[0] https://github.com/mozilla/DeepSpeech
[1] https://wiki.seeedstudio.com/ReSpeaker_4_Mic_Array_for_Raspb...