Soli • Hacker News Board

Re: Soli

#231

Post by drusepth » Wed, Oct 16, 2019, 7:55 PM UTC

schmichael wrote: post #225

eutropia wrote: post #178

Earlier quoted context omitted.

> In the case of Pixel 4, the model runs on device, never sends sensor data to Google servers, and helps it interpret the motion to Quick Gestures. From the "Technology" page linked at the top. Personally, I don't see how it is meaningfully any worse for privacy than the always on microphone in the google assistant. And I look forward to what it will enable for VR and AR tech!

"Never sending raw sensor data to Google" and "Never sending inferences made from sensor data to Google" are massively different claims. "Never sending raw sensor data to Google" is an efficiency claim that means little with regard to protecting privacy.

It's pretty trivial* to sniff traffic and find out what data is actually being sent to Google once people have devices in their hands, no?

It doesn't seem like the kind of thing a company would try to underhandedly sneak in after explicitly saying the sensor runs on-device and "never sends sensor data to Google servers".

* Trivial in the sense of someone who actually knows what they're doing -- of which we only really need one person in order to "leak" what's actually happening.

Re: Soli

#232

Post by codetrotter » Wed, Oct 16, 2019, 7:57 PM UTC

jasonhong wrote: post #119

I recently showed some videos of Soli in the HCI class I teach. Students immediately hit upon the two major issues I wanted to discuss (I was pretty proud!). The first is learnability. A big problem with gestures is that there is no clear affordance as to what kinds of gestures you can do, or any clear feedback. For feedback, one could couple Soli's input with a visual display, but at that point, it's not clear if th…

> A big problem with gestures is that there is no clear affordance as to what kinds of gestures you can do, or any clear feedback. For feedback, one could couple Soli's input with a visual display, but at that point, it's not clear if there is a big advantage over a touchscreen, unless the display is really small.

For the Google Pixel 4 that they are using in the video you already have a big display. It can instruct you how to gesture so that you learn it and later it can let you gesture without instructions.

> The second is what's known as the Midas touch problem. How can the system differentiate if you are intentionally gesturing as input vs incidentally gesturing?

Either an activation word like you said, or it could use the front-side camera to see whether or not you are looking at it.

Re: Soli

#233

Post by pstuart » Wed, Oct 16, 2019, 8:04 PM UTC

GistNoesis wrote: post #163

It is a nice piece of technology. It is a 60Ghz millimeter-wave radar. It is a privacy nightmare. It is already shipped. Radar uses electromagnetic waves (like Wifi but higher frequency) so it can go through walls, and even typical range for gesture recognition is less than meter, It probably can go at least 10 times as far by boosting the gain of the amplifier, it is not constrained like a theremin would be because…

And once you add in MicroExpression comprehension (https://www.paulekman.com/resources/micro-expressions/), and voice audio analysis and you've got a real working, portable lie detector.

That will be, um, interesting.

Re: Soli

#234

Post by modernerd » Wed, Oct 16, 2019, 8:04 PM UTC

warrenmiller wrote: post #129

jasonhong wrote: post #119

I recently showed some videos of Soli in the HCI class I teach. Students immediately hit upon the two major issues I wanted to discuss (I was pretty proud!). The first is learnability. A big problem with gestures is that there is no clear affordance as to what kinds of gestures you can do, or any clear feedback. For feedback, one could couple Soli's input with a visual display, but at that point, it's not clear if th…

Watch the keynote for more information on accidental gestures. They cater for it

There's a little on accidental input detection in this short video too: https://www.youtube.com/watch?v=QS8SW-ouM5w

Re: Soli

#235

Post by eptcyka » Wed, Oct 16, 2019, 8:06 PM UTC

summerlight wrote: post #207

jasonhong wrote: post #119

I recently showed some videos of Soli in the HCI class I teach. Students immediately hit upon the two major issues I wanted to discuss (I was pretty proud!). The first is learnability. A big problem with gestures is that there is no clear affordance as to what kinds of gestures you can do, or any clear feedback. For feedback, one could couple Soli's input with a visual display, but at that point, it's not clear if th…

The second one seems more of a technical one and can be solved if Soli can reliably recognize user attention, which can effectively be a "hotword" for gesture. This is hard and not sure even it's feasible with this tech, but given all the excitements in this thread on potential privacy issues I guess it's doable :D The first one seems more troublesome. This is less intuitive than touch screen based interface. The onl…

If you can use a hot-word, what's wrong with using voice recognition to achieve what you want to do anyway? Using voice takes less effort.

Re: Soli

#236

Post by sizzle » Wed, Oct 16, 2019, 8:06 PM UTC

jasonhong wrote: post #119

I recently showed some videos of Soli in the HCI class I teach. Students immediately hit upon the two major issues I wanted to discuss (I was pretty proud!). The first is learnability. A big problem with gestures is that there is no clear affordance as to what kinds of gestures you can do, or any clear feedback. For feedback, one could couple Soli's input with a visual display, but at that point, it's not clear if th…

Once we figure out (non-invasive) BCI and EEG type brain activity signature patterns for when our brains process our perceived intent of taking an action and can activate that action on the system side, prior to our brain sending those electrical impulses to our motor system.

How hard would it be to teach ourselves to inhibit the electrical impulses to our motor system when BCI can identify intent?

When would this level of BCI be possible if you had to make an educated guess?

Thanks for sharing, as a fellow HCI/Cog Sci graduate!

Re: Soli

#237

Post by victoro0 » Wed, Oct 16, 2019, 8:15 PM UTC

drusepth wrote: post #231

schmichael wrote: post #225

Earlier quoted context omitted.

"Never sending raw sensor data to Google" and "Never sending inferences made from sensor data to Google" are massively different claims. "Never sending raw sensor data to Google" is an efficiency claim that means little with regard to protecting privacy.

It's pretty trivial* to sniff traffic and find out what data is actually being sent to Google once people have devices in their hands, no? It doesn't seem like the kind of thing a company would try to underhandedly sneak in after explicitly saying the sensor runs on-device and "never sends sensor data to Google servers". * Trivial in the sense of someone who actually knows what they're doing -- of which we only reall…

Are you serious?

Because I'm pretty sure there is some random "user experience checkbox" checked by default that somehow means some of the data is sent to Google so they can "improve their products and services" and what not, but don't worry because it is probably "anonymized" and will only be seen by "humans" or "AIs" depending on what feels less bad for the general public once discovered.

Re: Soli

#238

Post by sizzle » Wed, Oct 16, 2019, 8:15 PM UTC

eptcyka wrote: post #235

summerlight wrote: post #207

Earlier quoted context omitted.

The second one seems more of a technical one and can be solved if Soli can reliably recognize user attention, which can effectively be a "hotword" for gesture. This is hard and not sure even it's feasible with this tech, but given all the excitements in this thread on potential privacy issues I guess it's doable :D The first one seems more troublesome. This is less intuitive than touch screen based interface. The onl…

If you can use a hot-word, what's wrong with using voice recognition to achieve what you want to do anyway? Using voice takes less effort.

Absolutely, you can use the same utterances to invoke the same intents in a car from a home setting.

"Alexa, set temperature to degrees"

"Alexa, set volume to or increase/decrease volume"

Re: Soli

#239

Post by Arelius » Wed, Oct 16, 2019, 8:20 PM UTC

drusepth wrote: post #231

schmichael wrote: post #225

Earlier quoted context omitted.

"Never sending raw sensor data to Google" and "Never sending inferences made from sensor data to Google" are massively different claims. "Never sending raw sensor data to Google" is an efficiency claim that means little with regard to protecting privacy.

It's pretty trivial* to sniff traffic and find out what data is actually being sent to Google once people have devices in their hands, no? It doesn't seem like the kind of thing a company would try to underhandedly sneak in after explicitly saying the sensor runs on-device and "never sends sensor data to Google servers". * Trivial in the sense of someone who actually knows what they're doing -- of which we only reall…

No, it's not. Between encryption and a high background communication noise of the cellphone already it's hard to discern exactly what is being sent.

I don't think anyone thinks that they are, or will be sending all the sensor data to the servers. But they don't need to, they only need to send back what is determined due to that data.

Specifically consider what is most likely for an ad company, a few differentiated markers to help with targeting, that can easily be packed into a few bits. And can be trivially stored to be sent later on with some other packet that has a valid reason to be sent. (Appstore update check, anyone?) Accounting for every bit that leaves the phone sounds like a nigh impossible task.

I'm not saying this is definitely happening, but I don't think it's practical to rule out the possibility, with any amount of packet sniffing.

Re: Soli

#240

Post by drusepth » Wed, Oct 16, 2019, 8:21 PM UTC

degenerate wrote: post #156

fasicle wrote: post #111

Earlier quoted context omitted.

Would be useful when cooking with mess on your hands and trying to scroll through a recipe.

This is the one and only suggestion so far that makes sense. It would be nice when cooking. If I was sitting in a bus/train/plane/concert/ , there is no chance in hell I am waving my hands at a phone. I would even feel silly doing it at home.

Controlling audio also seems pretty big with the assumption that gestures work "through various materials for seamless interaction" as the website says. The dial, slide, and swipe gestures look perfect for adjusting volume and skipping songs without taking your phone out of your pocket. Technically a smart watch could also do this (though I'm not sure if it'd be equally smooth), but I also can't remember the last time I saw someone wearing a smart watch (YMMV, I'm in the midwest).

It seems like it'd work well in the general case of "I want to control a device that I can't touch". There's lots of reasons you wouldn't want to touch a device (dirty/non-free hands, device is out of reach, device doesn't have a screen, etc) and a lot of devices you'd want to control from afar (televisions, radios, speakers, AC units, alarm clocks, or arguably phones, I guess). It'll be interesting to see what uses emerge from that intersection once other kinds of devices get Soli.

I wonder if it'd be useful for more crazy ideas that fit into "giving a device more information about physical actions nearby" (like sleep tracking), but I don't know how realistic those are.