Live data from Hacker News

Amazon device recorded private conversation, sent it out to random contact

kiro7.com

191–200 of 734 posts

Re: Amazon device recorded private conversation, sent it out to random contact

#191
post #170
post #166

Earlier quoted context omitted.

Such a system would be a huge GDPR nightmare. You'd need consent from every visitor to your house to collect data on them, and also you have to delete it if they ever request you to. Best not to even try.

Does the GDPR even apply to private individuals like that?

No, definitely not.

Re: Amazon device recorded private conversation, sent it out to random contact

#192
post #167

Earlier quoted context omitted.

They should feel more comfortable than an equivalent system built by a company that is looking to profit off of your data - and additionally, you can give the guest stronger guarantees that when you say that the system is "off", it actually is.

The chance that a random implementer has a security vulnerability is much higher than that Jeff Bezos is listening to me watch TV. A private system is more vulnerable to target attack and an Amazon system is more vulnerable to mass surveillance.

That's true - and I suppose that some people are much more concerned about the former threat than the latter.

Re: Amazon device recorded private conversation, sent it out to random contact

#193

Earlier quoted context omitted.

That seems overly simplistic to me. It's like saying that someone buys a car that's powered by an explosive liquid, and then complains when all the gas in the tank explodes at the same time.

Both a car and an amazon echo have a switch that turns that functionality off.

I feel like you're being purposefully reductive here. You know they're different right? The entire value proposition (and marketing strategy) of Alexa and related devices is that they are voice activated, so you don't have to physically interact with the device to make a request.

Given that, it would be ridiculous to expect paying customers to switch it off when not in use. The onus on Amazon is to protect their customers' privacy while the device is being used as intended.

If you really want to pick apart the car analogy, maybe this is closer to having automatic start and driving capability on a car, but the car sometimes randomly starts and drives away. Are you really going to blame the customer for that, even though they could disable the feature every time they exit the car?

Re: Amazon device recorded private conversation, sent it out to random contact

#194
post #97

Earlier quoted context omitted.

Possible? Yes. On a Pi? .. Maybe? Probably not with current power I'd guess. Allot goes into the voice training and equivalent operations. There are a few OSS alternatives to build something similar; I don't think anything has a similar power yet though.

There’s a Mozilla project for everything https://hacks.mozilla.org/2018/02/how-to-build-your-own-priv... (voice is experimental and I don’t know if it’s on device) Looks cool, but I haven’t tried. I’m the kind of person that gets up and walks to the light switch.

There's Mycroft too: https://mycroft.ai/ We have a lot of alternatives but they are unable to get the sort of marketing and hence the attention mainstream devices like Alexa and Google Home do

Re: Amazon device recorded private conversation, sent it out to random contact

#195
post #73

I received one of these messages just a week ago. Alexa sent me a message of my friend and his girlfriend having a private conversation. I immediately texted him to ask if he intended to do that and he did not- so weird.

Can you explain a bit more. What was the context of the message? was it an email? What was the subject line and the text? Was the audio an attachment or a link? A link to where...? "Amazon sent me a message" is so frustratingly vague...

Yea, I wish the girl mentioned what the message started with. Something had to have been misinterpreted as a voice command and a contact name ... and the device UI has to be broken to the point where it doesn't give you any audio acknowledgement or confirmation.

Re: Amazon device recorded private conversation, sent it out to random contact

#196

I've told my roommate I'm moving out if he ever buys an Alex/Google/Apple assistant device. I have a microphone and I've been intending to get one of the open source solutions working and just tie it in to mpd, weather and a few other things. But all the processing should really be done on your own device, by hardware you own, software that's open and that you configure, and not send up to someone else's computer (ak…

Just out of curiosity - what open solutions are you referring to?

Re: Amazon device recorded private conversation, sent it out to random contact

#197

What actually happened: Alexa misinterpreted some voice commands and activated a "call" skill. The people involved and local news got very excited and escalated this into a conspiracy story. Amazon takes customer privacy EXTREMELY seriously. There's no way a team would get the "ok" to build a skill that randomly records private conversations then sends them to a random contact. It also doesn't make any logical sense…

I get what you are saying but I would say that Amazon does NOT take privacy extremely seriously or this couldn't have happened. Let me be clear that I'm not saying they don't care at all or they are conspiring with the NSA.

What I mean by the above is that the "call" skill is much different than the "weather" skill. All Alexa has to do is have a confirmation prompt in the "call" skill and this wouldn't have happened. That is what extremely serious looks like. This is exactly the same as the phantom laughter incident from a few months ago. Alexa "heard" someone say 'Alexa laugh' and laughed, but that wasn't the user's intent. It was fixed by moving to 'Sure, I can laugh,' followed by laughter.

Voice UI is very hard and still in its infancy but ability for personal harm (physical or emotional) must be considered in these interfaces. Turning off the lights may not need confirmation but unlocking the doors or turning off the alarm probably should. Sending recordings or answering calls or even calling people should require more hoops or at least allow the user to control the risk/reward.

Re: Amazon device recorded private conversation, sent it out to random contact

#198

Earlier quoted context omitted.

I don't. And in this particular scenario, it's an important distinction to make because there's a risk of Amazon using equivocation to deflect responsibility.

If this was intentional, then yes they shouldn't be able to deflect responsibility. However, if it was just a bug, I think it is a bit unfair to vilify either the company or the developers. Bugs happen, and hopefully they can learn what caused this and prevent this class of bugs in the future.

And this is where "software" diverges from "engineering".

Bugs happen in architecture, aircraft, etc. too. the difference is that the actual engineers are paid to have a precautionary approach and spend significant resources to actively prevent bugs from making it into the final product.

In contrast, software is often written to "ship first", be "agile", and "move fast and break things". Yet when it causes problems, they just say "bugs happen", and "it is unfair to vilify them".

Features are not better than reliable security.

And yes, negligence is less bad than malice, but it is still damaging and developers and managers need to be held to account.

If your manager is pressing you to do unsafe crap in too big a hurry, it's your responsibility to push back, and if unsuccessful, leave for saner pastures and make it more difficult for that management to proceed.

Re: Amazon device recorded private conversation, sent it out to random contact

#199
post #174

I have this idea of a system I would like to have in my house. It contains cameras in every room that are constantly watching where people are and relaying the coordinates to a central server. That server makes decisions on if lights should be on or if A/C should be running in that room. But I would never buy this system. I would have to make it myself. I am hopeful that open source software and hardware can produce…

You don’t need to use cameras for this. A simple speaker and microphone is all you need to make a functioning motion sensor. Just exploiting the doppler effect. And even better, you can do it all outside of the human audible spectrum. And it can work with capturing motion around corners, too, since sound bounces off walls. I’d recommend something like this in every room, or even IR sensors, over cameras. You don’t wa…

> A simple speaker and microphone is all you need to make a functioning motion sensor.

You could just use an infrared motion sensor. They are dirt cheap and widely used for controlling lights.

The AC unit at work has one, too.

So, all of this already exists. Without spying capabilities, that is.

Re: Amazon device recorded private conversation, sent it out to random contact

#200

What actually happened: Alexa misinterpreted some voice commands and activated a "call" skill. The people involved and local news got very excited and escalated this into a conspiracy story. Amazon takes customer privacy EXTREMELY seriously. There's no way a team would get the "ok" to build a skill that randomly records private conversations then sends them to a random contact. It also doesn't make any logical sense…

If a "call" skill is accidentally triggered, before it is sent to any email addresses, it should tell user "N seconds of voice was recorded and about to send to ....", please said "Send" to send, "Play" to play back the message, etc.

The default must always be voice recording will be auto deleted after 1 minute if no response is heard. It should let user know about that too.

Post reply on HN