Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple suspends Siri response grading in response to privacy concerns

techcrunch.com

Post reply on HN
Print view View original

201–210 of 227 posts

Re: Apple suspends Siri response grading in response to privacy concerns

#201

Post by saagarjha »

ashelmire wrote: post #174
probably_wrong wrote: post #19

So far, most of the comments here are a good example of both why this happened and why it will happen again. When people complained that putting microphones in your home 24/7 was creepy, the response was "that's nonsense, Siri is not sending anything unless you specifically ask for it". When it turned out that it does send things you didn't intend, the response was "well, it's not like anyone is listening anyway". No…

> We really need mandatory ethics courses in CS. You think the CS grads / devs are the ones deciding to do this? We're not really given true autonomy. We're grunts. Someone above us decides to do the spying and we either do it or hit the job market.

That's the whole point of an ethics course: you don't just do what your boss tells you to do. This is how engineering works: if you're told to build a bridge you know will collapse, it's your fault if you go along with it.

Re: Apple suspends Siri response grading in response to privacy concerns

#202

Post by starsinspace »

inscionent wrote: post #189
starsinspace wrote: post #158

Earlier quoted context omitted.

Nope, it does not. Only the trigger phrase (“Hey Siri“) is recognised locally on the device (according to Apple), everything that comes after that is sent as audio recording snippet to Apple servers where the actual speech recognition happens. There are rumours [0] that Apple might enable local speech-to-text at some point, but as of now, that’s not how it works. [0] https://appleinsider.com/articles/18/11/15/apple-c…

https://support.apple.com/en-us/HT208343 > You can use dictation instead of your keyboard to enter text with many apps and features that use the iOS keyboard. > On iPhone 6s or later, and iPad, you can use dictation without being connected to the Internet. Earlier models of iPhone and iPad require an Internet connection.

Sadly, offline dictation is a separate feature, it is not the speech-recognition used for Siri.

Quote directly from Apple:

"The recording of the user’s spoken words is sent to Apple’s voice recognition server. If the task involves dictation only, the recognized text is sent back to the device. "

Source:

Page 69 in https://www.apple.com/business/site/docs/iOS_Security_Guide....

Re: Apple suspends Siri response grading in response to privacy concerns

#203

Post by pdkl95 »

misnome wrote: post #132
probably_wrong wrote: post #19

So far, most of the comments here are a good example of both why this happened and why it will happen again. When people complained that putting microphones in your home 24/7 was creepy, the response was "that's nonsense, Siri is not sending anything unless you specifically ask for it". When it turned out that it does send things you didn't intend, the response was "well, it's not like anyone is listening anyway". No…

> When people complained that putting microphones in your home 24/7 was creepy, the response was "that's nonsense, Siri is not sending anything unless you specifically ask for it". This seems like a distorted representation, to support your argument, of what most the complaints and responses were. The gulf between 24/7 recording and "occasionally unintentional sending" is absolutely massive and you just merge them in…

> The complaints were (and still are) conspiracy theories

The people making those complaints are probably not crazy; it might be a good idea to try to understand their actual concerns instead of jumping to this inaccurate (and insulting) misrepresentation.

> they were streaming constant recordings

That would be a waste of bandwidth. Obviously a constant stream isn't happening. While a trivial noise gate and high-efficiency encoding could easily[1] allow almost constant recording of voice for trivial amounts of bandwidth, that isn't really the concern.

> constantly listening

It has to be constantly listening, or it wouldn't hear the wake word. However, this too isn't really the concern.

> in a targeted, continual sense

Dan Geer, on "targeting"[2]:

>> There is no mechanistic difference whatsoever between personalization and targeting save for the intent of the analyst. To believe otherwise is to believe in the Tooth Fairy. To not care is to abandon your duty.

Targeted surveillance is by definition rare and often hard to avoid. I'm far more concerned about non-targeted, sparsely-gathered data that surveillance capitalists like Google/FB/etc can use to refine their model of your pattern-of-life[3]. While this touches on a few important issues, yet again this isn't why I consider always-on, remotely-controlled microphones to be a serious problem.

The big problem is that you're normalizing the idea that remote 3rd parties might be recording in areas traditionally considered private. Kyllo v United States[4] created a bright-line rule that asks if a technology (in general, not any specific product) is considered to be "in general public use"[5]. If it's not, using that technology is considered a "search" for 4th Amendment purposes, and a warrant is required if the police want to use it. When a technology is "in general public use"[5], the police no longer need a warrant to use their own voice-activated internet-controlled microphones to see the "details of a private home that would previously have been unknowable without physical intrusion"[6].

I don't care what these devices may or may not be recording right now. I do care that normalizing of the idea that voice might be recorded inside the home by a remotely-managed device is eroding my future 4th Amendment rights even if I never buy an Alexa/etc.

[1] https://news.ycombinator.com/item?id=16346416

[2] http://geer.tinho.net/geer.source.27iv17.txt

[3] https://en.wikipedia.org/wiki/Pattern-of-life_analysis

[4] https://caselaw.findlaw.com/us-supreme-court/533/27.html

[5] used multiple times in ruling[4] (esp. section II of Justice Stevens' dissent)

[6] see [4], 2nd paragraph

Re: Apple suspends Siri response grading in response to privacy concerns

#204

Post by tareqak »

jammygit wrote: post #200
tareqak wrote: post #73

Earlier quoted context omitted.

I graduated from a Canadian engineering undergraduate program in software engineering, and I have my Iron Ring. The ethics in the more established branches engineering (chemical, electrical, mechanical) are definitely not pretend, and are more rigorous. Software engineering is considerably newer so the codifying the ethics there are definitely a work in progress. One thing that I will say is that the engineering ethi…

I suspect you’re far more informed than I am. I should not have spoken of engineering so broadly, I think what I said may have been very software specific

No problem. A lot more Canadian engineering undergraduates find the whole Iron Ring and ethics stuff to be some kind of Koolaid than I'd like to admit. I am interested in learning more about that team that won a prize for the cellphone-tracking solution at airports though: specifically, I'm curious as to how they came across the problem and how said problem was framed. Framing is one of the tools in which entities get people to build surveillance tools like PRISM and the like by appealing to those people's sense of patriotism. The other big tool is money, and enough of it.

Re: Apple suspends Siri response grading in response to privacy concerns

#205

Post by Faark »

saagarjha wrote: post #199
blub wrote: post #185

Earlier quoted context omitted.

This was answered in another comment: https://news.ycombinator.com/item?id=20593642 One cannot know with absolute certainty, only with reasonable certainty. On the other hand, an assistant is listening all the time by design. If one needs to know with absolute certainty, then they cannot use a mobile phone, nor probably a reasonably modern regular phone.

Modern smartphones have assistants that "listen all the time" in the same way that you're saying that smart speakers do (namely, they look for a wakeword).

No, modern smartphones allow you to have assistants that listen all the time. Disabling those is one of the first things I do, and I'd be pretty unhappy if it turns out those are still listening and wasting battery (even aside of privacy issues).

Re: Apple suspends Siri response grading in response to privacy concerns

#206

Post by jerf »

tialaramex wrote: post #181
jerf wrote: post #167

Earlier quoted context omitted.

Cell phones, even now, can't be doing that much background processing unknownst to us, because they are connected to a battery, battery life is a big deal in a highly competitive industry, and we'd all notice if they drained very quickly. ISTR there are documented instances of law enforcement and/or intelligence doing that sort of thing to phones, and the battery drain causing it to get noticed. Hooking it up to an o…

Counterpoint: Google's actual smartphone line (Pixel) ships with passive song recognition, default disabled. I enabled it because I was curious and it sips so little power that I left it on, but to function it has to be listening to the audio all the time so that it can say to itself "Hey, that is Sabotage by the Beastie Boys". It doesn't use the network, or the screen (unless you power the screen on to find out what…

That's part of why I left my caveat in there about the future. We've had chips to detect wakeup words for a while now. From what I know about song detection, that's likely to be doing yet more stuff, but still nowhere near enough for full voice transcription yet. But it is only a matter of time before your phone can be doing full voice transcription, with good guesses as to who the speaker is, full time. Squirting a full day's transcript of your speech up would be very easy to hide in all the chatter a phone already has, and squirting up just selected "suspicious" speech could easily be hidden in just the metadata network traffic of a phone.

We have not historically been there, which informs people's about cell phones, and we are not quite there yet. 4 or 5 years maybe.

(Although I have some interesting thoughts on how to deal with that. It would be fun to take one of these "transcribing phones" into a movie theatre, then fire the MPAA at Google for copyright infringement.)

Re: Apple suspends Siri response grading in response to privacy concerns

#207

Post by andromeduck »

tialaramex wrote: post #181
jerf wrote: post #167

Earlier quoted context omitted.

Cell phones, even now, can't be doing that much background processing unknownst to us, because they are connected to a battery, battery life is a big deal in a highly competitive industry, and we'd all notice if they drained very quickly. ISTR there are documented instances of law enforcement and/or intelligence doing that sort of thing to phones, and the battery drain causing it to get noticed. Hooking it up to an o…

Counterpoint: Google's actual smartphone line (Pixel) ships with passive song recognition, default disabled. I enabled it because I was curious and it sips so little power that I left it on, but to function it has to be listening to the audio all the time so that it can say to itself "Hey, that is Sabotage by the Beastie Boys". It doesn't use the network, or the screen (unless you power the screen on to find out what…

Doesn't your phone have to be awake a few seconds for that?

Re: Apple suspends Siri response grading in response to privacy concerns

#208

Post by eridius »

Faark wrote: post #205
saagarjha wrote: post #199

Earlier quoted context omitted.

Modern smartphones have assistants that "listen all the time" in the same way that you're saying that smart speakers do (namely, they look for a wakeword).

No, modern smartphones allow you to have assistants that listen all the time. Disabling those is one of the first things I do, and I'd be pretty unhappy if it turns out those are still listening and wasting battery (even aside of privacy issues).

No the parent is right. They are not "listening all the time". They're essentially sleeping while a separate process handles the audio stream looking for the wakeword. When it finds the wakeword, it then "wakes up" the assistant which reprocesses the audio snippet to confirm the wakeword is really there (using more expensive analysis), before then actively listening to the rest of the audio.

Re: Apple suspends Siri response grading in response to privacy concerns

#209

Post by justin66 »

cheeseomlit wrote: post #148
criddell wrote: post #142

Earlier quoted context omitted.

Ethics should be a mandatory part of any STEM program.

Ethics isn't something you can learn in a classroom, a college professor can't teach you the difference between right and wrong or good and evil.

Ethical theory is a tough subject for high school-age kids, and even a Jesuit high school isn't going to get too deep into it. There's more to reasoning about this stuff than you think.

In this case we're talking about a professional engineering ethics class that will be somewhat lighter on the theory and heavier on the case studies, I would suspect.

Re: Apple suspends Siri response grading in response to privacy concerns

#210

Post by tinus_hn »

Angostura wrote: post #112
tinus_hn wrote: post #100

Earlier quoted context omitted.

All Apple devices that support Hi Siri play a notification and display lights or a UI when it activates and starts sending data outside of the device. I don’t see what more they can do. Of course a device that listens for an activation phrase is going to listen. And it’s never going to be 100% accurate. If you can’t accept that, just set it up so it only starts listening when you press the button.

I'll tell you what they can do and what they should have done. If, on your iOS device you go to Privacy > Analytics you will find this: https://imgur.com/a/LTV9bGc Where Apple explicitly asks you whether you are willing for your data to be used to improve a variety of services and you can opt-in or opt out. Apple dropped the ball by not specifically including Siri as a separate item in the list. Bad Apple.

So because some things are optional, everything has to be optional. Too bad that’s just not how it works. If you don’t want Apple to process the data you send to Siri your option is to turn off the service.

Here’s a little ‘secret’, you can’t use the location service either without Apple processing you data for analysis. If you don’t want that, your option is to turn off location services.

Post reply on HN