Live data from Hacker News

Alexa and Siri Can Hear Hidden Commands

nytimes.com

21–30 of 178 posts

Re: Alexa and Siri Can Hear Hidden Commands

#21
post #18

Just out of curiosity, what's the frequency range that a typical mic can pick up the signal from? The article did not specially mention about the range instead it said inaudible. And here is another article I found that mentions the normal 20-20kHz frequency response range: http://blog.shure.com/mic-basics-frequency-response/ Isn't that mostly overlap with the human ear capability? I understand each person is differe…

Every model will be different, but the important thing is that the boundaries represent the frequencies within which the signal will stay above a particular threshold of amplitude. A good spec sheet will tell you that threshold, and I've seen things like -3, -6, or -10 dB. It will still pass audio outside of the range but at an undisclosed attenuation.

Re: Alexa and Siri Can Hear Hidden Commands

#22

Serious off-topic question: Are there docs for Siri so that i can learn what it can/can’t do? I have tried skipping songs, playing a genre, set random play, and similar in iTunes — generally a failure, often initiates an unwanted phone call. On the phone I can successfully call the intended contact about 50% of the time, possibly because I have ~250 contacts. I suspect that if i knew the right words to interact with…

For disabling Siri, the toggles are in Settings under "Siri and Search" - the long hold was a major annoyance to me as well.

Re: Alexa and Siri Can Hear Hidden Commands

#23

Serious off-topic question: Are there docs for Siri so that i can learn what it can/can’t do? I have tried skipping songs, playing a genre, set random play, and similar in iTunes — generally a failure, often initiates an unwanted phone call. On the phone I can successfully call the intended contact about 50% of the time, possibly because I have ~250 contacts. I suspect that if i knew the right words to interact with…

Ask it “What can I ask you?” and it will show a big list of capabilities and phrases to trigger them.

Re: Alexa and Siri Can Hear Hidden Commands

#24

Serious off-topic question: Are there docs for Siri so that i can learn what it can/can’t do? I have tried skipping songs, playing a genre, set random play, and similar in iTunes — generally a failure, often initiates an unwanted phone call. On the phone I can successfully call the intended contact about 50% of the time, possibly because I have ~250 contacts. I suspect that if i knew the right words to interact with…

This doesn't address your question but might be interesting. As someone who has used Android forever and never tried Siri, those anecdotes are mind blowing to me. For me with Google Assistant, media commands work about 80% and successful call initiation is about 90%. And I haven't looked for documentation either. YMMV of course.

So a simple media command fails 1 in 5 times? This is the best SV can do with a incredibly well-studied technology (voice recognition)?

Re: Alexa and Siri Can Hear Hidden Commands

#25

Hidden audio is simply too easy. Hidden audio is the knife that kills desire for any financial services access through a voice assistant - for those smart enough to not follow the horde.

for me it kills all desire to have a voice assistant. I am already in the camp of taping over the camera's in my computers now will I need to worry about the microphone or what comes from the speakers?

so the question is, shouldn't they be able to detect the wavelength of what they are processing to weed out some of the more obvious tricks? with voice recognition could it also not be limited to a voice it is trained to know?

Re: Alexa and Siri Can Hear Hidden Commands

#26

Serious off-topic question: Are there docs for Siri so that i can learn what it can/can’t do? I have tried skipping songs, playing a genre, set random play, and similar in iTunes — generally a failure, often initiates an unwanted phone call. On the phone I can successfully call the intended contact about 50% of the time, possibly because I have ~250 contacts. I suspect that if i knew the right words to interact with…

This doesn't address your question but might be interesting. As someone who has used Android forever and never tried Siri, those anecdotes are mind blowing to me. For me with Google Assistant, media commands work about 80% and successful call initiation is about 90%. And I haven't looked for documentation either. YMMV of course.

As someone who has used both, the problem with Siri is when it works it's great, but it will fail on the exact same command the next time you try it. Consistency is key in these things working.

Re: Alexa and Siri Can Hear Hidden Commands

#27
post #23

Serious off-topic question: Are there docs for Siri so that i can learn what it can/can’t do? I have tried skipping songs, playing a genre, set random play, and similar in iTunes — generally a failure, often initiates an unwanted phone call. On the phone I can successfully call the intended contact about 50% of the time, possibly because I have ~250 contacts. I suspect that if i knew the right words to interact with…

Ask it “What can I ask you?” and it will show a big list of capabilities and phrases to trigger them.

It is really weird to refer to a device in the second person. Why isn't there some other way to get that info?

Re: Alexa and Siri Can Hear Hidden Commands

#28
post #18

Just out of curiosity, what's the frequency range that a typical mic can pick up the signal from? The article did not specially mention about the range instead it said inaudible. And here is another article I found that mentions the normal 20-20kHz frequency response range: http://blog.shure.com/mic-basics-frequency-response/ Isn't that mostly overlap with the human ear capability? I understand each person is differe…

Yes, typical mics tend to pick up the typical human frequency range, though cheaper mics may have some really poor characteristics at the edges. Usually in the speech range they'll be pretty solid.

However, there's a lot of play within the space. One difference is that microphones do a very direct recording of the sound waves, but what we hear is actually very distorted compared to the "real" sound by the nature of our ear. One of the big differences is that if there is a very loud 4000Hz sound, we can't hear a soft 4005Hz sound near it very well, but the microphone "hears" it just fine. So for instance, you could put out a loud sound for a user, but embed a very quiet command in frequencies the human couldn't hear, but if the listening model doesn't account for that (and there are reasons it wouldn't necessarily want to, because it wants to hear commands even in the presence of significant background noise), you could get commands in to a system. See https://en.wikipedia.org/wiki/Psychoacoustics for discussion about how our ears fail to pick up the "real audio" signal, and how much we've exploited that in music compression.

Now, that was a very brute force example. It sounds to me like what this article is talking about are called "adversarial examples" (https://blog.acolyer.org/2017/02/28/when-dnns-go-wrong-adver... ). Voice recognition doesn't listen the same way we do, it doesn't necessarily take a holistic view of the signal, but is looking for specific frequency patterns and changes and turning that into phonemes, into words, etc. (There's a lot of ways of doing this and I don't specifically know what Alexa and Siri are doing, so that's a really vague overview.) If you know what they are looking for, you can use filters to very, very selectively remove the patterns from a bit of music or something that Alexa might trigger on, and then insert just the bare minimum skeleton of the sounds that it is really recognizing. A human won't be able to hear the difference (most likely; depends on how badly the original is mangled but even if it is audible it is almost certainly not audible without an A/B test and very good ears), but the probably-neural-nets monitoring for sounds will end up superstimulated and interpret the adversarial example as words.

While the adversarial examples work best with tuning to the target network, widely-shared networks like Alexa or Siri mean that such tuning is practical where attacking some custom-trained model used by one person isn't, and experiments have shown that adversarial examples travel between separately-trained nets and even non-neural-net models to a much, much greater degree than what at least my own intuition would have suggested before hand. (See previous link and look for the discussion of "Practical black-box attacks against deep learning systems using adversarial examples". It is extremely counter-intuitive to me how easy this is.)

Re: Alexa and Siri Can Hear Hidden Commands

#29
Isn't there an inaudible tone that you can use to disable the assistant? I recall reading somewhere that Amazon used it in their commercials for Alexa so that everyone's Echos weren't lighting up during the commercials. I know when a commercial for the Echo comes on and the voices repeat "Alexa, do X" the Echo I have near the TV speaker doesn't light up.

Re: Alexa and Siri Can Hear Hidden Commands

#30

Serious off-topic question: Are there docs for Siri so that i can learn what it can/can’t do? I have tried skipping songs, playing a genre, set random play, and similar in iTunes — generally a failure, often initiates an unwanted phone call. On the phone I can successfully call the intended contact about 50% of the time, possibly because I have ~250 contacts. I suspect that if i knew the right words to interact with…

It makes me think of old text based adventure games where you type in "open door" or "draw gun". All these complex "A.I." based voice assistants still break down to the vocabulary problem. They try to solve the general case by not giving humans/customers the language spec.

There is probably more than just an AI that does speech to text and then a second phase interpreter. I suspect there is some AI in the first layer of Siri/OKGoogle/Alexa that uses context clues to narrow down what you're asking, but who knows for sure. It's a big black box.

Eventually it's like the 90s again where you type "Get ye flask" and you get a box saying, "You cannot get ye flask" and you're left playing Peasants Quest asking, "Why in the world can I not 'get ye flask?!'"

Post reply on HN