Summary: Qualcomm wanted me to devise a computer vision solution that was nearly three orders of magnitude power-efficient than what they had then. There was a clear justification existing as to why such a drastic improvement was needed. Nobody had a solution in spite of trying for a long time. Most laughed it as impossible. I started by looking for a proof as to why it could not be done if it indeed could not be done. After some three months of pulling my hair, I started getting glimpses of how to do it. Some three months later, I could convince myself and a few others that it is doable. Some three months later, the local team was fully convinced. Some three months later, the upper management was convinced that the research phase was over.
Details:
I was told to reduce power consumption for an existing computer vision solution by nearly three orders of magnitude. This included end-to-end power consumption including camera as well as power for computer vision computations.
No one in the industry knew how to do this. Many industry experts laughed at the goal itself.
To make it worse, I by myself had no prior expertize in computer vision, nor in camera design, though I knew image processing. I was assigned to the project solely because of my brand name with the company. Being ignorant perhaps helped since else I may not even have accepted the assignment.
I was told this is to be a five-years type research program, given the agressive goal.
I was told that the company was ready to do change whatever is needed to make this happen, be it the camera design (including pixel circuit), processor architecture, computer vision algorithms, even optics and packaging.
With no starting point in hand (and in fact a false start assumed based on half-baked ideas from one university), I had head-aches daily for more than a month. Most of this was to learn and deep dive into the prior art.
The goal I assumed was to either solve the problem, or show a theoretical basis for why it cannot be solved. (In hindsight, to solve a challenging problem, try to prove why it cannot be solved, since even that is usually hard, and you may prove yourself wrong in the process by solving it!)
There was no assigned team, given that there was no solution in mind. There were people with varied skillsets (optics, packaging, digital circuits) to help as per need.
Three months in, I had some ideas for how it could work. Six months in, there were three more engineers on the project working under me on a near full-time basis. And some were complaining that we don't know what we were doing. Seven months in, I had Excel-level calculations to show that it can work, and actually a simpler solution than what I had in mind at the three-month point. Nine months in, with about seven people working, the team was convinced of the solution, as validated via rough circuit design and simulations. By the end of the year, senior management was convinced that we have solved the problem and the research phase is over. We had an estimated five years to solve the problem, and the solution achieved three times less power than the target!
The solution was a mix of several new inventions (one was given gold rating by the company) and significant power savings obtained just by careful design. Unfortunately, I cannot talk much about the solution itself.
We started showing early preview of the capabilities (under NDA and without disclosing how we solved it) at multiple Tier 1 companies, and saw their jaws drop. At one company, one whispered to their team members, "Are these guys kidding?!". At another company, someone smiled silently for half-an-hour till we started showing the system emulator.
Soon, MIT Technology Review talked about the work [1].
I have since then left the company, so do not know much about the current state of the project, other than additional news coverage received over the time. Additional references are there in my LinkedIn profile.
PS: I have solved many hard problems through my career, have written about the one that gave me the most head-aches. Here is a list of some other ones [2].
[1] https://www.technologyreview.com/s/603964/qualcomm-wants-you...