Amazon device recorded private conversation, sent it out to random contact • Hacker News Board

Re: Amazon device recorded private conversation, sent it out to random contact

#361

Post by Clubber » Thu, May 24, 2018, 6:49 PM UTC

yason wrote: post #297

I am oversimplifying here but: * install device that is designed to listen to speech in the house * the device is connected to internet * the device is capable of contacting other internet peers/services/hosts * the device knows a bit about its owner's internet presence such as contacts * the device is equipped with simple conversational user interface based on fuzzy human speech-command detection These basically bou…

Sounds like the equivalent of a confirm popup would be a solution. Are you sure you want to send this to so and so?

Re: Amazon device recorded private conversation, sent it out to random contact

#362

Post by ocdtrekkie » Thu, May 24, 2018, 6:51 PM UTC

jacksmith21006 wrote: post #360

Not sure why people would be buying the Echo over the Google Home any longer? The GH offers a much better UX and now is well ahead of Alexa in sales last quarter. "Google takes top spot in global smart speaker market, HomePod nowhere to be found" https://appleinsider.com/articles/18/05/23/google-takes-top-... Came to the market 2 years later and already passed. This is before we even get Duplex. https://www.youtube.c…

Uh, dude, I know being a fan of Google is your thing here, but this was less than a year ago: http://money.cnn.com/2017/10/11/technology/google-home-mini-...

And it's a flaw so bad they had to outright remove the button entirely, because they couldn't fix it without replacing the hardware.

It's a little disingenuous to insinuate a bug with recording audio is a reason people shouldn't be buying company X's product over company Y's, which also has had a bug with recording audio.

Re: Amazon device recorded private conversation, sent it out to random contact

#363

Post by wilsonnb » Thu, May 24, 2018, 6:52 PM UTC

tambourine_man wrote: post #276

grkvlt wrote: post #237

Earlier quoted context omitted.

This is the kind of argument that seems really common and I find very annoying. So 'indoor plumbing' is obviously used to provide us with clean water, but the convenience aspect is the indoor part - you can have clean water from an outdoor communal well, instead. Similarly 'store bought bread' is food , but so is home made bread. I just cannot work out if the commenter was being deliberately obtuse or really misunder…

> you can have clean water from an outdoor communal well, instead Have you ever considered that this is not the case in most of the world? Resources are far from evenly distributed. It is not by chance that the availability of indoor plumbing correlates to increase in life expectancy. It's nice that I can just open the tap and clean water comes out, but it's also a lot more sanitary than people carrying and storing b…

The whole point is that they were not comparing anything to clean water or cheap food. They were comparing something to water that doesn't need to be moved inside manually and bread that doesn't have to be made at home.

Neither of which is needed to live. Both of which are conveniences.

Re: Amazon device recorded private conversation, sent it out to random contact

#364

Post by cortesoft » Thu, May 24, 2018, 6:52 PM UTC

toss1 wrote: post #338

cortesoft wrote: post #226

Earlier quoted context omitted.

> Features are not better than reliable security. Well, yes and no. Bugs, security, reliability, etc are all important things to consider, but they can't be the only focus. Security doesn't matter if the thing you are creating has no features; it would be the same as if it didn't exist at all. Instead, we must manage risk; the risk of bugs, the risk of security vulnerabilities, etc. Nothing we do is risk free. Even w…

I did not say >"security is always the most important thing". I did not suggest that anyone should develop such that >" the thing you are creating has no features". I certainly never suggested that no one get out of bed because they might slip in the bath. These are strawman arguments. You do not need to lecture me about risk. I've had a career in international downhill ski racing, have won auto racing championships,…

Ok, I never said that the 'analysis' should be purely based on dollar value. I never even said it should be a mathematical model. You accuse me of a strawman and then turn around and do the same to me.

I was simply pointing out that we can never get to zero risk, and since we can't, we have to weigh risks based on consequences and probability.

> 1) fully examine the system for potential critical failure points/modes

Sure, to the best of your ability. How can you know for certain you have found all potential critical failure points? You can get pretty sure, but never fully sure. We still have industrial accidents, in every single industry in the world.

You also have to define what a 'critical risk' is. I don't think it is an a priori fact that accidentally sending a recording of a conversation to a contact is a 'critical risk'.

Re: Amazon device recorded private conversation, sent it out to random contact

#365

Post by ocdtrekkie » Thu, May 24, 2018, 6:53 PM UTC

duxup wrote: post #327

yason wrote: post #297

I am oversimplifying here but: * install device that is designed to listen to speech in the house * the device is connected to internet * the device is capable of contacting other internet peers/services/hosts * the device knows a bit about its owner's internet presence such as contacts * the device is equipped with simple conversational user interface based on fuzzy human speech-command detection These basically bou…

I remember the first voice command stuff on google. Buddy of mine was trying it out and I couldn't help myself but shout: "NEW TEXT MESSAGE TO JENNY! FUCK YOU! SEND!" Fortunately the phone was too slow to take it all (even if it had Jenny would have found it amusing, she was a cube or two away). But yeah first thing I thought of was that if it's listening how easy is it for it to mistake intent or someone else or etc…

I got a touchscreen for my desktop PC once (it was the Windows 8.1 era, after all). I immediately discovered that it provided me no significant practical benefit, but it was now infinitely easier for someone walking by my desk to poke my screen and mess up what I was doing.

Re: Amazon device recorded private conversation, sent it out to random contact

#366

Post by gol706 » Thu, May 24, 2018, 6:58 PM UTC

dalbasal wrote: post #184

dustinmoorenet wrote: post #140

I have this idea of a system I would like to have in my house. It contains cameras in every room that are constantly watching where people are and relaying the coordinates to a central server. That server makes decisions on if lights should be on or if A/C should be running in that room. But I would never buy this system. I would have to make it myself. I am hopeful that open source software and hardware can produce…

On making, not buying... A lot of home automation stuff seems like something that would be fun to make, but not something I really want to have. I mean, the AC might be a good energy saver (a smarter thermostat) but that's not something I want , more like something that I would buy if it was cheap and practical enough. To some extent, this is a phenomenon of early technology. It feels full of potential, so you want t…

Another pain point with making is that it's really hard to get it integrated with stuff you have bought. I built my own garage door automation with a Particle Photon board. It works great and can do things like text me if I leave the house with the garage door open using the IFTT support from my WiFi router. The problem is that it's really hard to get it integrated with any other control system that the rest of my house uses like my ZWave light switches and Hue lights.

I've been working on a custom UI that sits on top of the Wink Hub API to unify everything, but I'be been stuck with their almost completely undocumented Pubnub event API.

Re: Amazon device recorded private conversation, sent it out to random contact

#367

Post by bluGill » Thu, May 24, 2018, 6:58 PM UTC

Anon1096 wrote: post #312

bluGill wrote: post #207

Earlier quoted context omitted.

You are on the right track, but I don't think you are quite right. Amazon like systems are more vulnerable to mass surveillance. Your vulnerability isn't a targeted attack: you are not valuable enough to be worth the effort to figure out your system. As an attacker on your system I'd have to figure out how to break in, and then how to use the hardware you have. You are more valuable as part of a botnet - attacks that…

And what exactly does this "vulnerability" mean in real terms? Let's be honest. No one cares what conversations are going on in your house _unless_ you're someone specifically targeted. There's little use in mopping up data with no goal.

If mopping up everybody's conversations is cheap enough Russia/Iran/China/(insert your favorite large evil) does. If you happen to run of political office 15 years from now having all your conversations available to analyze will be useful. If they don't like you, you might find some out of context snippet of "private conversation" all over social media killing your campaign. (or alternatively the blackmail threat if you don't X)

That is they will target everybody because they know in a few years that will include somebody who they currently think is a nobody.

Of course as AI gets better and cheaper they may eventually listen to everything to see what who can be targeted automatically for what.

Re: Amazon device recorded private conversation, sent it out to random contact

#368

Post by ninkendo » Thu, May 24, 2018, 6:59 PM UTC

squeaky-clean wrote: post #331

jadedhacker wrote: post #125

I feel creeped out by these home listening devices and I don't own one, but don't our phones already have this capability? You can turn "Ok Google" on on an android phone. I sometimes record audio, and the mic is incredibly good. Is there a substantial difference between our phones and these devices? EDIT: Just realized the substantial difference is that Google and Amazon own all of these things. They don't control a…

"Okay Google" and "Hey Siri" run locally on the device, but all other transcription is done "in the cloud". Try setting your phone on airplane mode and going "Hey Siri, What time is it?" It will recognize the Hey Siri, but give an error for everything else. Pretty sure "Okay Google" will behave the same way. Also if you turn on battery save on iOS it disables "Hey Siri".

This. It would be a tremendous battery drain if your phone had the microphone on at all times, with a constant connection to the server, sending all audio over it at all times, just so it can detect when somebody says "hey siri".

It's a custom low-power chip they added on the 6S and later to allow a local tight loop that only listens for that utterance using a hardware-assisted neural net, and only activates the rest of the software stack if it detected it with reasonable confidence.

There's a nice blog post about it here from the apple engineers (see the "Two-pass detection" section): https://machinelearning.apple.com/2017/10/01/hey-siri.html

Re: Amazon device recorded private conversation, sent it out to random contact

#369

Post by edwhitesell » Thu, May 24, 2018, 6:59 PM UTC

modzu wrote: post #349

edwhitesell wrote: post #337

Earlier quoted context omitted.

Most phone now have some kind of lock screen, which makes it pretty difficult to get to the butt dialing stage. Will speech command recognition get to that stage? The main reason I won't have any of those products in my house is because of that. I'd much rather have a confirmation of some kind before the system takes action. "...random talking..." Assistant "I am recording" Me: "Stop recording"

some kind of acoustic biometrics would be helpful here (ie respond only to account holder, or disable some actions for others) along with better heuristic recognition of directives thats not foolproof but much better than what we have now, and i think we are pretty close

Agreed, but I think vocal biometrics is a significantly more difficult problem to solve when even the best speech recognition still has issues like this.

Phonetics is hard. Especially with ambient noise, echo and such. I had a conversation with one of the speech engineers when I worked at a speech recognition company and the level of detailed problems to solve was impressive. Totally made sense after talking about it, but things I would not have thought about before.

I'd imagine the next thing to come in this area that would really make an improvement is an "on person" microphone. Maybe it's a pen in your pocket, or some kind of vibration detection (that could pick up the wearers voice), that would then allow some improvemnts in the domain of "who is talking" and how well the voice is processed.

Re: Amazon device recorded private conversation, sent it out to random contact

#370

Post by rectang » Thu, May 24, 2018, 7:01 PM UTC

wilsonnb wrote: post #356

rectang wrote: post #19

"Always listening" is a fundamentally unsafe design. Once recordings of private conversations leave the local environment and make it to the cloud, eventually they will leak. It's akin to data collection by law enforcement: once the data exists, eventually it will be abused.

So you're saying that all data that has made it to the cloud will eventually leak? That seems like a strange assumption to me.

Yes. Assume that all data uploaded to the cloud will eventually be compromised. That is the only safe assumption.

Not even the most responsible companies (e.g. Google) can hold out 100% of the time in the face of determined assault by government. Some of that data is going to leak to three-letter agencies, or similar.

Somewhat less responsible companies (e.g. Amazon) will leak data more often, to a wider range of threat agents.

So then we have to consider how valuable this data is. Random sampling of private conversations within the home? Sometimes innocuous -- but if the wrong moment gets leaked, the consequences are potentially life-shattering.