Amazon device recorded private conversation, sent it out to random contact • Hacker News Board

Re: Amazon device recorded private conversation, sent it out to random contact

#681

Post by freehunter » Fri, May 25, 2018, 2:30 PM UTC

stephenr wrote: post #676

tracker1 wrote: post #675

Earlier quoted context omitted.

I always think if I ever present somewhere again, I'm tempted to start off by asking people to put their phones into silent mode... wait a moment, then "OK Google, Siri show me dick picks"

Hey Siri is keyed to a specific persons voice. Is the google equivalent not?

It’s silly that radios and podcasts and random people have the ability to trigger Alexa just by saying the name. “Hey Siri” gets around the issue by only listening for your voice, not just anyone. It’s shocking that anyone would build a voice assistant with zero authentication or authorization built in, so much so that an advertisement on the radio or TV can trigger an action on your device like on an Echo.

Re: Amazon device recorded private conversation, sent it out to random contact

#682

Post by spir » Fri, May 25, 2018, 2:31 PM UTC

gnicholas wrote: post #536

Amazon's response, from Ars' article: > Echo woke up due to a word in background conversation sounding like "Alexa." Then, the subsequent conversation was heard as a "send message" request. At which point, Alexa said out loud "To whom?" At which point, the background conversation was interpreted as a name in the customers contact list. Alexa then asked out loud, "[contact name], right?" Alexa then interpreted backgro…

In my house, there is a design flaw in Alexa which encourages this failure mode: there is a single volume level for the system, instead of separate volume levels for Alexa voice vs. music/application audio.

I keep my Alexa's volume at 2-3, because otherwise the music / podcasts are too loud for me. However at volume 2-3 the Alexa system voice is fairly quiet and I often miss things Alexa says- requested or accidental. I'd prefer to keep Alexa system voice at volume 5, but music at 2-3.

I requested this feature of a distinct Alexa voice volume years ago. I would be interested if anyone has a solution for this.

Re: Amazon device recorded private conversation, sent it out to random contact

#683

Post by vidarh » Fri, May 25, 2018, 2:44 PM UTC

sqlacid wrote: post #665

vidarh wrote: post #644

Earlier quoted context omitted.

I tested mine right now, and I can't see any correlation between the light ring and my location relative to it.

could be picking up a bounced audio if it close to a wall or other surface.

Maybe, but if it can't handle that, it's kinda pointless to try to detect direction - all the places in my house where I'd be likely to want to place it are <0.5m from a wall or less; I'm certainly not going to place them in the middle of a room.

Re: Amazon device recorded private conversation, sent it out to random contact

#684

Post by Corrado » Fri, May 25, 2018, 2:48 PM UTC

spir wrote: post #682

gnicholas wrote: post #536

Amazon's response, from Ars' article: > Echo woke up due to a word in background conversation sounding like "Alexa." Then, the subsequent conversation was heard as a "send message" request. At which point, Alexa said out loud "To whom?" At which point, the background conversation was interpreted as a name in the customers contact list. Alexa then asked out loud, "[contact name], right?" Alexa then interpreted backgro…

In my house, there is a design flaw in Alexa which encourages this failure mode: there is a single volume level for the system, instead of separate volume levels for Alexa voice vs. music/application audio. I keep my Alexa's volume at 2-3, because otherwise the music / podcasts are too loud for me. However at volume 2-3 the Alexa system voice is fairly quiet and I often miss things Alexa says- requested or accidental…

I would like a feature where Alexa detected the sound level in the room and responded with an appropriate volume. For example, in the quiet of the early morning I just want to hear the days weather forecast at a low volume, to avoid waking others in the house. However, during the afternoon, when kids are playing and the A/C is rattling and the dog is barking, I would like my responses in a higher volume.

Re: Amazon device recorded private conversation, sent it out to random contact

#685

Post by stephenr » Fri, May 25, 2018, 2:53 PM UTC

freehunter wrote: post #681

stephenr wrote: post #676

Earlier quoted context omitted.

Hey Siri is keyed to a specific persons voice. Is the google equivalent not?

It’s silly that radios and podcasts and random people have the ability to trigger Alexa just by saying the name. “Hey Siri” gets around the issue by only listening for your voice, not just anyone. It’s shocking that anyone would build a voice assistant with zero authentication or authorization built in, so much so that an advertisement on the radio or TV can trigger an action on your device like on an Echo.

s/silly/fucking ridiculous/

but sure, I agree. That's why I questioned it.

Re: Amazon device recorded private conversation, sent it out to random contact

#686

Post by andreyk » Fri, May 25, 2018, 2:53 PM UTC

titzer wrote: post #615

andreyk wrote: post #588

The explanation for this is fairly unremarkable ( https://techcrunch.com/2018/05/24/family-claims-their-echo-s... ): Echo woke up due to a word in background conversation sounding like “Alexa.” Then, the subsequent conversation was heard as a “send message” request. At which point, Alexa said out loud “To whom?” At which point, the background conversation was interpreted as a name in the customers contact list. Alexa…

Actually it's very remarkable on how bad the voice recognition is, and how Amazon can knowingly ship this crap to people, putting it in their homes, recording their private interactions. Ya'll recorded people having sex yet? It sounds like the machine learning models are tuned for specific phrases and have terrible, terrible false positive rates. "Evaluating options to make this less likely"? Now I don't even know wh…

It's a 1-in-a-million edge case.... I would hope people on HN at least would understand this stuff happens in software.

Re: Amazon device recorded private conversation, sent it out to random contact

#687

Post by illvm » Fri, May 25, 2018, 2:57 PM UTC

kinsomo wrote: post #274

tree_of_item wrote: post #264

Earlier quoted context omitted.

One reason you might want a server is to have more expressive power than "if someone literally enters the room, turn on the lights". You might want to turn on the lights and run the AC a little before you predict people will arrive, for example.

> One reason you might want a server is to have more expressive power than "if someone literally enters the room, turn on the lights". You might want to turn on the lights and run the AC a little before you predict people will arrive, for example. Maybe, but that seems like a lot of work for little gain. You probably don't actually have that much control over the AC in your rooms unless you have zoned heating and a l…

If you don't have multiple zones you can still effectively have zones by opening/close the registers/vents. This will prevent hot or cold air from entering the room. However, it will also increase duct pressure and that could be somewhat problematic. Either way, "smart vents" are on the market. They're ridiculously overpriced though and one could hack together an alternative for under $50 per vent (source: personal experience). Probably less if they are clever. Of course then there is the problem of running power to the vents or having sufficient battery. But that's an exercise left for the reader.

Re: Amazon device recorded private conversation, sent it out to random contact

#688

Post by sly010 » Fri, May 25, 2018, 3:10 PM UTC

hdkrgr wrote: post #680

robotmlg wrote: post #409

Earlier quoted context omitted.

> As it is I hate how I can't talk ABOUT "ok google" around my phone... because it always goes off. On several tech podcasts I listen to ( http://atp.fm , https://daringfireball.net/thetalkshow/ ) they use fillers like "Yo, dingus!" to talk about alexa/hey siri/ok google

Another convention seems to be 'Alexo' when you want to talk about Alexa but not to Alexa.

We just call her "you know who" (a.k.a. the thing who's name we are too afraid to mention)

Re: Amazon device recorded private conversation, sent it out to random contact

#689

Post by amzn_engineer1 » Fri, May 25, 2018, 3:34 PM UTC

snarf21 wrote: post #197

amzn_engineer1 wrote: post #66

What actually happened: Alexa misinterpreted some voice commands and activated a "call" skill. The people involved and local news got very excited and escalated this into a conspiracy story. Amazon takes customer privacy EXTREMELY seriously. There's no way a team would get the "ok" to build a skill that randomly records private conversations then sends them to a random contact. It also doesn't make any logical sense…

I get what you are saying but I would say that Amazon does NOT take privacy extremely seriously or this couldn't have happened. Let me be clear that I'm not saying they don't care at all or they are conspiring with the NSA. What I mean by the above is that the "call" skill is much different than the "weather" skill. All Alexa has to do is have a confirmation prompt in the "call" skill and this wouldn't have happened.…

I think you accurately identify the issue with this situation / skill; voice triggers need to weigh convenience (how easy should it be to activate this skill) against it's permissions or potential (how can this skill affect a customer). In this case, I think this was not done properly.

This does not mean Amazon does not take privacy seriously. It is a company of small teams with very few layers of management between an engineer and business decision. The error in judgement of one team does not reflect on all of Amazon.

As you say, Voice UI is still in its infancy and not lacking growing pains. However, because Amazon does take privacy very seriously, after this incident, I'm certain there will be actions taken internally to ensure teams properly weigh the gravity of a skill with its voice trigger (or adjustments made if there is already an existing policy).

My own thoughts, not Amazon's.

Re: Amazon device recorded private conversation, sent it out to random contact

#690

Post by titzer » Fri, May 25, 2018, 3:46 PM UTC

andreyk wrote: post #686

titzer wrote: post #615

Earlier quoted context omitted.

Actually it's very remarkable on how bad the voice recognition is, and how Amazon can knowingly ship this crap to people, putting it in their homes, recording their private interactions. Ya'll recorded people having sex yet? It sounds like the machine learning models are tuned for specific phrases and have terrible, terrible false positive rates. "Evaluating options to make this less likely"? Now I don't even know wh…

It's a 1-in-a-million edge case.... I would hope people on HN at least would understand this stuff happens in software.

1 in a million happens about 4000 times a second on a single CPU running at 4ghz...

Did you really want to talk concrete numbers? Because if so, I am wondering about the probability that Amazon's voice recognition mistakes random conversation as a valid entry in someone's contact list, as well as the other terms in this equation.

I don't know if you work for Amazon, and if so I don't want to single you out specifically, but this is a pretty bad screwup, and it does not inspire confidence. Please don't brush it away with "oh it's a 1-in-a-million edge case". That attitude is even worse, and if it is indeed that attitude inside of Amazon, then I am even more strongly against this and I hope that further, deeper scrutiny is applied here, because this screwup is actually illegal.