I can't recommend highly enough that everyone at least turn on the audible sound when their assistants are listening. At a minimum you should know the kinds of things that end up triggering the device. There's a really wide area of detection and it's interesting to see where that is. I would also recommending changing your default word at a minimum. Then again, I might also recommend ditching the device entirely but…
but "Hey Spybot-Listening-To-Every-Conversation-In-This-Household-And-Building-A-Transcript-For-The-NSA" doesn't roll off the tongue the same way.
With everyone scrutinizing the web traffic around them trying to prove NSA/Google spying with them we'd definitely have found something by now.
So it's probably just yours that is sending off to the NSA. I'd send it back to them for repairs, see if they don't give you a free home mini in compensation!