Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon device recorded private conversation, sent it out to random contact

kiro7.com

Post reply on HN
Print view View original

71–80 of 734 posts

Re: Amazon device recorded private conversation, sent it out to random contact

#71

Post by cortesoft »

DoofusOfDeath wrote: post #22
asn0 wrote: post #15

Summary of vague technical details (which may be all we hear about this): > an Alexa engineer investigated ... they said 'our engineers went through your logs, and they saw exactly what you told us, they saw exactly what you said happened, and we're sorry.' He apologized like 15 times in a matter of 30 minutes and he said we really appreciate you bringing this to our attention, this is something we need to fix!" > th…

> The device did not audibly advise that it was preparing to send the recording, something it’s programmed to do. Apparently it's not programmed to do that. Unless this was a hardware glitch or cosmic-ray event.

As developers, we use 'programmed' in that manner all the time, to mean what we were trying to get it to do rather than what it actually does.

Re: Amazon device recorded private conversation, sent it out to random contact

#72

Post by menacingly »

simion314 wrote: post #12

This devices should by law always have a big LED on when recording, you would have the option to set it's brightness though. Also why could it send data out without any confirmation from the user.

I think we can assume there was a light here. It's almost happened to me before. Alexa disastrously misinterprets an ambient conversation as "start transcribing and send to one of my contacts". During this, the light does blink to indicate it's gobbling up your input. You just wouldn't know to look for it if you didn't even know you'd activated the device.

Re: Amazon device recorded private conversation, sent it out to random contact

#73

Post by nicky0 »

michael_j_ward wrote: post #54

I received one of these messages just a week ago. Alexa sent me a message of my friend and his girlfriend having a private conversation. I immediately texted him to ask if he intended to do that and he did not- so weird.

Can you explain a bit more. What was the context of the message? was it an email? What was the subject line and the text? Was the audio an attachment or a link? A link to where...?

"Amazon sent me a message" is so frustratingly vague...

Re: Amazon device recorded private conversation, sent it out to random contact

#75

Post by newman8r »

untog wrote: post #6
newman8r wrote: post #2

The article doesn't explain much, but it sounds to me like the device mistakenly interpreted a voice command to record and send a message.

Sure, but isn't that still terrifying? It brings me back to something I wonder about with these voice activated devices: you have no idea what it can do. Voice UIs are so utterly opaque, and Amazon(/Google/whoever) pushes new software updates without informing you. So it'll add new commands all the time, and you won't know about it until you accidentally trigger it, making it do... whatever . But either way, they sho…

yeah it's still off-putting.

I haven't used a smartphone in years. Looking back on the days when I did use a smartphone, there was always this tiny fear in the back of my head that maybe I'd pocket-dialed someone and they'd be able to hear my conversation. It actually had a subtle chilling effect.

The Hawthorne effect (which discusses how people react to being observed) seems to be especially relevant nowadays as there's a real possibility that we're being recorded at any time. https://en.wikipedia.org/wiki/Hawthorne_effect

It's actually a big theme in 1984 where the mere possibility that you were being listened to via telescreen was enough to keep you in line and influence your behavior.

Re: Amazon device recorded private conversation, sent it out to random contact

#76

Post by function_seven »

DiabloD3 wrote: post #45

So when will such devices just be made illegal entirely? It doesn't seem they can be made to be safe for average people if stuff like this keeps happening and companies are unwilling to compensate people. What if this was a lawyer talking about clients or a doctor talking about patients? Is Amazon (or Google, or Microsoft) willing to deal with such legal liabilities? Is this what move fast and break things looks like…

You ever butt dial someone? Should all cell phones be outlawed?

I'd say the Echo needs to have a more difficult activation routine available. Maybe have the option of setting the wake word to "Alexa, can you please" or something similarly long and unlikely to appear in normal conversation.

Also, I wonder if they had their wake word set to "Computer". I did that once, and quickly reverted after so many false activations.

Re: Amazon device recorded private conversation, sent it out to random contact

#77

Post by mikeash »

williamscales wrote: post #43
tambourine_man wrote: post #30

People are volunteerelly paying for their houses to be tapped for the convenience of being able to shout: play me some song. I’m so out of touch with this world it’s scary.

Nevermind that we are already in a world where you can play any song you like from the wireless computer you carry in your pocket. That's not convenient enough. /s

That pocket computer also has all the same potential threats to privacy. Somehow nobody cares about them, though.

Re: Amazon device recorded private conversation, sent it out to random contact

#78

Post by parliament32 »

I feel that while these devices are great in principal, there still needs to be work done on whether the "trigger word" activation is the best way to enable them. Maybe we need longer / more complex trigger words? Maybe there's a different way we can activate them alltogether, avoiding the "always listening" problem?

Re: Amazon device recorded private conversation, sent it out to random contact

#79

Post by ElCapitanMarkla »

cortesoft wrote: post #63
joshstrange wrote: post #51

Earlier quoted context omitted.

It's a little more than just that. Every light in my house is controlled with Alexa which is a very nice and hard to explain until you have tried it. Also my home theater system is all controllable via Alexa as well. This kind of thinking makes no sense to me. > People are voluntarily paying for their houses to be tapped for the convenience of being able to shout: play me some song. > I’m so out of touch with this wo…

I never thought that voice control would be that useful until I had my daughter... being able to turn on the white noise, turn off the tv, play her music, etc while having my hands full of toddler is very useful.

That’s about all we use ours for

“Alexa... play white noise”

“ALEXA... stop”

Re: Amazon device recorded private conversation, sent it out to random contact

#80

Post by pdkl95 »

sjm-lbm wrote: post #24
Someone1234 wrote: post #20

Earlier quoted context omitted.

What does "recording" mean in this context? The way these devices work is they're always "recording" in a loop on a secondary processor looking for the wake word. If we take this request literally the LED would never be off.

I'm not entirely sure that this would be a bad thing. It would set the expectation in customers' minds that the device is always listening, which is at least somewhat true.

> expectation in customers' minds that the device is always listening

If, even with that expectation, enough people still use "voice assistant" technology for a judge to consider the tech to be "in general public use"[1], the bright line test defined by Kyllo v United States is triggered and the police no longer need a warrant to use the technology (in the abstract - they don['t need to use your hardware) to view "details of a private home that would previously have been unknowable without physical intrusion"[2].

Normalizing the expectation that previously private areas (like the inside of your home) might be recorded and sent to a remote 3rd party will eventually result in everyone losing some of their 4th amendment protection against search and seizure.

[1] http://caselaw.findlaw.com/us-supreme-court/533/27.html

[2] Ibid.

Post reply on HN