Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon device recorded private conversation, sent it out to random contact

kiro7.com

Post reply on HN
Print view View original

231–240 of 734 posts

Re: Amazon device recorded private conversation, sent it out to random contact

#231

Post by LinuxBender »

qyv wrote: post #99
joshstrange wrote: post #51

Earlier quoted context omitted.

It's a little more than just that. Every light in my house is controlled with Alexa which is a very nice and hard to explain until you have tried it. Also my home theater system is all controllable via Alexa as well. This kind of thinking makes no sense to me. > People are voluntarily paying for their houses to be tapped for the convenience of being able to shout: play me some song. > I’m so out of touch with this wo…

There is a massive difference between walking a few steps to turn on a light switch (which is only useful when you are in close proximity) vs having a accurate real-time mapping functionality when on the go. Similarly, there is a massive difference in the privacy implications of listening to every conversation everyone in your home (including guests) is having vs. having your current location known. Wiretapping laws…

There is a drinking water appliance in our breakroom that will not dispense water if it can not reach the internet.

This is not the world we should be engineering. Actually, I would say, this is not engineering.

Re: Amazon device recorded private conversation, sent it out to random contact

#232

Post by pcunite »

dustinmoorenet wrote: post #140

I have this idea of a system I would like to have in my house. It contains cameras in every room that are constantly watching where people are and relaying the coordinates to a central server. That server makes decisions on if lights should be on or if A/C should be running in that room. But I would never buy this system. I would have to make it myself. I am hopeful that open source software and hardware can produce…

The Personal Cloud is what we should be building for ourselves. Servers running in your closet at your house.

Re: Amazon device recorded private conversation, sent it out to random contact

#233

Post by toss1 »

amzn_engineer1 wrote: post #66

What actually happened: Alexa misinterpreted some voice commands and activated a "call" skill. The people involved and local news got very excited and escalated this into a conspiracy story. Amazon takes customer privacy EXTREMELY seriously. There's no way a team would get the "ok" to build a skill that randomly records private conversations then sends them to a random contact. It also doesn't make any logical sense…

And this is where "software" diverges from "engineering". It's not a conspiracy, it's negligence.

Bugs happen in architecture, aircraft, etc. too. the difference is that the actual engineers are paid to have a precautionary approach -- and spend significant resources -- to actively prevent bugs from making it into the final product.

Amazon and your team has built a great product (I have one and make moderate use of it, have even considered building some skills).

But, you have planted a full-on bugging device in millions of people's homes. Done by a government, this would be cause for war or revolution. This is serious, and you need to treat it much more seriously than you obviously are. Every 'skill' does not require the same minimal levels of security and verification, some, like this one, require much more, or should be forbidden outright until such security can be properly implemented (and yes, this should probably include calls only to pre-configured whitelists, intent confirmation, etc. and to any manager that says "that's too inconvenient to the user", the response is "screw you, it's critical").

You call yourself an "engineer" at least twice, and claim that you take privacy "extremely seriously". The evidence from this incident and others noted in this thread indicates otherwise. Clearly, insufficient resources were allocated to figuring out the potential failure modes of a "call skill", and preventing them.

All due respect, but your team needs more of an engineering approach than you have. This entire "it's gotta ship yesterday" mentality in the software industry used to be just inconvenient. Now it's getting dangerous. Please help stop it.

Re: Amazon device recorded private conversation, sent it out to random contact

#234

Post by wilburTheDog »

mundo wrote: post #146

I wonder how similar this is to https://www.cs.cmu.edu/~sbhagava/papers/face-rec-ccs16.pdf ? That's the "Facial recognition fooled by funny-colored glasses" study from Carngie Mellon, where researchers were able to make machine learning algorithms fail disastrously (e.g. mistake a man for Milla Jovovich) with a pair of glasses printed with what looks like a random assortment of colorful pixels, but is in fact a targe…

I recently stayed at a house which had an alexa device. In a conversation where I said the words light switch several times Alexa beeped and responded to me each time. I think most of us just believe Amazon when they say "Alexa responds to its name" and don't stop to consider the possible failure modes. We want to believe it can understand our words, when it's really just guessing. Over long enough time it's inevitable that it will misunderstand you and do something you don't want.

Still, I wonder how it heard "record this conversation and send it to someone on my contact list".

Re: Amazon device recorded private conversation, sent it out to random contact

#235

Post by BonesJustice »

amzn_engineer1 wrote: post #66

What actually happened: Alexa misinterpreted some voice commands and activated a "call" skill. The people involved and local news got very excited and escalated this into a conspiracy story. Amazon takes customer privacy EXTREMELY seriously. There's no way a team would get the "ok" to build a skill that randomly records private conversations then sends them to a random contact. It also doesn't make any logical sense…

What is this alleged 'conspiracy' story? An Amazon device recorded a private conversation and transmitted it to someone else, and it did so without the user's knowledge or intent. That actually happened.

How it happened is only relevant to the engineers who build and maintain the thing. I, on the other hand, could not care less how it happened, and the fact that it did happen is reason enough never to buy one of those infernal devices.

Re: Amazon device recorded private conversation, sent it out to random contact

#236

Post by kinsomo »

jclardy wrote: post #162
kinsomo wrote: post #59

Earlier quoted context omitted.

> Due to watchword detection, the microphone amp would always be on. I don't think that would be a bad thing; especially if there's a switch to disable the mic. When you turn it off, you'll get reliable feedback to know it's actually off. I think it's important that these kinds of devices have simple feedback and control mechanisms that can be independently verified and reasoned about. Software is too opaque and too…

I don't disagree, but the light ring is also important to know that it actually heard your command. Having it on all the time would make the UX much worse. Unless it was a separate light.

> I don't disagree, but the light ring is also important to know that it actually heard your command. Having it on all the time would make the UX much worse. Unless it was a separate light.

I see that, I think it would be best as a separate light.

I think product UX has drifted too far towards blank monoliths; I know I wouldn't mind a few more blinkenlights :)

Re: Amazon device recorded private conversation, sent it out to random contact

#237

Post by grkvlt »

tambourine_man wrote: post #152
vorpalhex wrote: post #119

Earlier quoted context omitted.

If I'm home and don't have my phone (or have my hands full, or it's dark, or whatever) I can easily turn lights on or off and without fumbling around for switches or trying to juggle a cat or a child. I can also control my tv, thermostat, get a news briefing, check my calendar and set reminders. Yes, it's ultimately a convenience, but so is indoor plumbing and store bought bread.

>Yes, it's ultimately a convenience, but so is indoor plumbing and store bought bread. I’m sorry, clean water and food are not conviniences. They are among the most basics of human needs. That’s the kind of thing that scares me. That someone would make such a comparison with a straight face.

This is the kind of argument that seems really common and I find very annoying. So 'indoor plumbing' is obviously used to provide us with clean water, but the convenience aspect is the indoor part - you can have clean water from an outdoor communal well, instead. Similarly 'store bought bread' is food, but so is home made bread. I just cannot work out if the commenter was being deliberately obtuse or really misunderstood?

Re: Amazon device recorded private conversation, sent it out to random contact

#238

Post by setquk »

jordigh wrote: post #138
DoofusOfDeath wrote: post #22

Earlier quoted context omitted.

> The device did not audibly advise that it was preparing to send the recording, something it’s programmed to do. Apparently it's not programmed to do that. Unless this was a hardware glitch or cosmic-ray event.

On the topic of cosmic rays... it's more likely than most people think, at least for 2007 hardware. https://lwn.net/Articles/219983/

It’s even worse. Sometimes the packages that the DRAM ICs are encapsulated in have minor radioactive elements in and alpha particles emitted can flip bits.

Interesting paper: https://media.blackhat.com/bh-us-11/Dinaburg/BH_US_11_Dinabu...

Re: Amazon device recorded private conversation, sent it out to random contact

#239

Post by ctrlc-root »

dustinmoorenet wrote: post #140

I have this idea of a system I would like to have in my house. It contains cameras in every room that are constantly watching where people are and relaying the coordinates to a central server. That server makes decisions on if lights should be on or if A/C should be running in that room. But I would never buy this system. I would have to make it myself. I am hopeful that open source software and hardware can produce…

Indeed, I'm working on something similar: https://github.com/CtrlC-Root/mdcs

Re: Amazon device recorded private conversation, sent it out to random contact

#240

Post by rm_-rf_slash »

This is why I unplug my Echo whenever I’m not using it.

Side note: why is it that after years and years of privacy breaches, there has been almost no support to use hardware switches for cameras and microphones?

I would feel 100000% safer using my internet-connected cameras and microphones knowing that I can turn off those devices independently of their host devices and that no hacker can monitor my cameras and microphones, even if they’ve rootkitted every device I own.

Post reply on HN