How many microphones do cell phones typically have? I guess I assumed one, though background noise cancelling would certainly be improved by having more. For this kind of positioning it seems the minimum needed would be 3 - and the Android SDK can access those audio streams separately unprocessed? Pretty neat.
FingerIO: Using Active Sonar for Fine-Grained Finger Tracking
101–110 of 126 posts
Re: FingerIO: Using Active Sonar for Fine-Grained Finger Tracking
#102I have a feeling this is also really depending on the hardware. The demos were probably designed around the specific brand of watch and cellphone since they'd need to know exact distances between the microphones/speakers.
It's a really cool concept. I wish they'd open source what they have, or at least have plans to open source it. However if this came about via University funding, they'll probably claim IP on it. If it was a student's own fellowship, he/she/they might decide to create a start-up out of it.
Re: FingerIO: Using Active Sonar for Fine-Grained Finger Tracking
#103How about security? How does it protect other uses from controlling your device?
I wonder about the other kind of the security. I doubt there's any serious protection against a rogue background-running application tracking your finger and gathering sensitive information, like keystrokes.
You know, check for what's the foreground process, notice it's a password manager or bank application (or even a lock screen would do, who knows, maybe user had reused the same PIN elsewhere), blast the speakers with ultrasound and wait for the typing-like movement patterns.
Re: FingerIO: Using Active Sonar for Fine-Grained Finger Tracking
#104Earlier quoted context omitted.
Having been in this kind of situation before: because the software isn't ready for any kind of release. They probably spent a week gathering and tuning the parameters on the DSP for each individual phone they ran it on, and without their knowledge of the system it'd take you a month and a half to get it working on your phone.
If they are publishing a paper, their methods (including custom source code) are ready for release for peer review. Ftheit software sucks, thet should not rely on it for scientific results.
Here's a case that I have published under before: Their software is ready for their own use for research by them personally. However, it has no help, documentation, automation, magic, or even small amounts of assistance. It doesn't work until you have it spit out a page of numbers, which you feed into Matlab and hack on for a week before compiling a new copy of the app that has the resulting calibration matrix baked into the code. The instruction manual for the apk you're asking for would be equivalent to a bachelor's in computer science and half a PhD on DSP and machine learning. That's why they're not releasing it. It proves that their math works and the approach is valid. Replacing their expertise and making it fast enough and reliable enough for general release would require a startup, six developers three of whom need PhDs, a UX team, and a year and a half of work. The approach works and they can prove it. That's it.
Re: FingerIO: Using Active Sonar for Fine-Grained Finger Tracking
#105Regarding everyone's latency concerns, as someone who has done low-latency audio processing on Android -- in their defense I'd bet almost anything the demo is meant to only demonstrate the math behind this. Depending on the platform ( Android cough), low latency audio processing can be almost a dark art itself. And hey look, they're doing this on Android. My guess is that they decided to release the demo earlier inst…
A lot of production input devices for stuff like waving hand gestures and pen input also have unacceptably high latency. Getting latency down is hard, and makes a huge difference to usability.
Re: FingerIO: Using Active Sonar for Fine-Grained Finger Tracking
#106Regarding everyone's latency concerns, as someone who has done low-latency audio processing on Android -- in their defense I'd bet almost anything the demo is meant to only demonstrate the math behind this. Depending on the platform ( Android cough), low latency audio processing can be almost a dark art itself. And hey look, they're doing this on Android. My guess is that they decided to release the demo earlier inst…
I'm curious, besides doing things in C/C++, are there any "magic tricks" to doing low-latency audio processing on Android? Looking at the chart here [1] it still seems to be a good bit behind iOS.
[1] http://www.androidpolice.com/2015/11/13/android-audio-latenc...
Re: FingerIO: Using Active Sonar for Fine-Grained Finger Tracking
#107Regarding everyone's latency concerns, as someone who has done low-latency audio processing on Android -- in their defense I'd bet almost anything the demo is meant to only demonstrate the math behind this. Depending on the platform ( Android cough), low latency audio processing can be almost a dark art itself. And hey look, they're doing this on Android. My guess is that they decided to release the demo earlier inst…
I'm curious, besides doing things in C/C++, are there any "magic tricks" to doing low-latency audio processing on Android? Looking at the chart here [1] it still seems to be a good bit behind iOS. [1] http://www.androidpolice.com/2015/11/13/android-audio-latenc...
In the Java layer? About the only thing you can do is ensure that you're using the devices native sampling rate: typically 48 kHz for phones, and 44.1 kHz for tablets. Non-native sampling will induce a rather large latency hit. The buffering + buffer size stuff, unfortunately, is really only accessible in native IIRC.
To be completely honest, it's been a long time since I've messed with audio-stuff on the Java-side, so not sure if/how much things have changed for the better.
Re: FingerIO: Using Active Sonar for Fine-Grained Finger Tracking
#108Earlier quoted context omitted.
Not when it's a few inches away. It's less distance than voice interaction.
A lot of people are a few inches from my pocket every day. And I can easily imagine software listening for such sonars and sending back fake responses. I think debarshri has a valid point.
> A lot of people are a few inches from my pocket every day.
Really? And your phone is on? With the screen visible for them to see what they are interacting with. And as they are fumbling around in your personal space inches from your hand or pocket, no one notices.
Get real! If true you should be more worried about pickpockets than some random gestures.
Re: FingerIO: Using Active Sonar for Fine-Grained Finger Tracking
#109I wonder how accurate it really is. The demo video didn't match up with the movements at all and the on-screen drawings looked like prerecorded video that they were trying to sync to. It's a neat idea, but without a dedicated component or an extremely high-speed RTOS, you're not going to come close to the level accuracy that's really needed to do the math and still allow interaction. I don't mean to rain on the parad…
I would totally disagree. I don't think it would have to have much fidelity or low latency be insanely useful. The huge advantage is no extra hardware necessary. And maybe it might be hard for you to believe it could do what you want it to do but I'm guessing your vision is pretty narrow.
This reads like a personal attack.
Re: FingerIO: Using Active Sonar for Fine-Grained Finger Tracking
#110We need to immediately improve on PIN-code protection upon cash-withdrawal in ATMs. The problem has been there for a while, but man, it gets easier and easier.
Or be (not, it would seem) overly paranoid like me, my PIN patterns and entering do not involve moving my fingers horizontally, I put my hand down on the pad with my fingers on set keys, cover it with my wallet, and type it in. I also always double one key to make it that much harder to get by observation, a trick I learned from a sysadmin with major access to my school's systems back when we all had to use public te…
What does it mean to “double one key”?