Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding Indirect Time-of-Flight Depth Sensing

devblogs.microsoft.com

Post reply on HN
Print view View original

21–30 of 40 posts

Re: Understanding Indirect Time-of-Flight Depth Sensing

#21

Post by rzw2 »

rsp1984 wrote: post #2

The article doesn't mention the main drawback of ToF depth sensing: Multipath Errors. They originate from light bouncing around in the scene before coming back to the detector, causing the resulting depth maps to have dents and distortions in the neighborhood of angled surfaces. They are a big problem in built environments which have lots of 90 deg angles that act as retroreflectors to the signal. To my knowledge non…

There has been some work on resolving multipath errors with indirect ToF, Chronoptics (I'm a cofounder) recently licensed our technology to Melexis in automotive.

Here's a blog post I wrote about resolving multipath https://medium.com/chronoptics-time-of-flight/multipath-inte...

And a link to the announcement from Melexis https://www.melexis.com/en/news/2021/4mar2021-melexis-announ...

Re: Understanding Indirect Time-of-Flight Depth Sensing

#22

Post by rzw2 »

stefan_ wrote: post #5
rsp1984 wrote: post #2

The article doesn't mention the main drawback of ToF depth sensing: Multipath Errors. They originate from light bouncing around in the scene before coming back to the detector, causing the resulting depth maps to have dents and distortions in the neighborhood of angled surfaces. They are a big problem in built environments which have lots of 90 deg angles that act as retroreflectors to the signal. To my knowledge non…

The article is also a bit optimistic in regards to outdoors use, with direct sunlight exposure. The sensors I tested in the past just didn't work at all.

There are two issues with sunlight and iToF. The sunlight photons saturate the pixel, and you get no useful measurement. The dominant noise source of iToF is photon shot noise [1], so sunlight photons contribute heavily to noise. Increase your laser power, use better optical notch filters, decrease the sensor integration time, and do more image filtering. I'm a cofounder at Chronoptics and we've developed a sunlight capable ToF camera, https://youtu.be/7vMI37S0w3Q

[1] https://en.wikipedia.org/wiki/Shot_noise

Re: Understanding Indirect Time-of-Flight Depth Sensing

#23

Post by whiw »

mscharrer wrote: post #6

How do these ToF systems deal with multiple sensors pointing at the same scene? I've seen it work with two ToF sensors, but haven't been able to find a good explanation for how it works.

If the other emitter/sensor pairs are uncorrelated with our own, and if we integrate over enough transmit/receive cycles, then the other pairs will contribute approximately equally to both of our own two sensor phase detectors. The method descibed here uses the difference between the sensor phase detectors, so any competing interference should cancel out. It will (just) raise the background noise floor a bit.

Re: Understanding Indirect Time-of-Flight Depth Sensing

#24

Post by echelon »

Are there any higher than VGA quality ToF sensors? Azure Kinect is not cutting it for us.

We'd like to do real time, full 360 depth capture of actors, but all of the sensors we've found are poorly suited for this task.

We might have to rig up our own optical system and use FPGAs to run them...

Re: Understanding Indirect Time-of-Flight Depth Sensing

#25

Post by rzw2 »

mscharrer wrote: post #6

How do these ToF systems deal with multiple sensors pointing at the same scene? I've seen it work with two ToF sensors, but haven't been able to find a good explanation for how it works.

There are three different methods. 1) TDMA (Time division multiplexing). Some sync system so only one camera is emitting light at a given time. The Microsoft Azure Kinect uses this with the 3.5mm sync cable, this enables up to 3 cameras to illuminate at different times.

2) Frequency domain. iToF cameras use different modulation frequencies, you can set different modulation frequencies and the signals won't interfere. The other cameras photon's will contribute to photon shot noise. Or randomly change modulation frequencies by a small amount during the integration time, which is supported by some sensors.

3) Randomly change timing during integration time, this is more common with pulsed ToF cameras. Analog devices had an example of this in their booth at CES in 2020.

Re: Understanding Indirect Time-of-Flight Depth Sensing

#26

Post by rzw2 »

echelon wrote: post #24

Are there any higher than VGA quality ToF sensors? Azure Kinect is not cutting it for us. We'd like to do real time, full 360 depth capture of actors, but all of the sensors we've found are poorly suited for this task. We might have to rig up our own optical system and use FPGAs to run them...

Microsoft licensed their sensor to Analog devices, and you can buy the mega-pixel iToF sensor and build a iToF camera. https://www.analog.com/en/products/ADSD3100.html?icid=ToF-ad...

Or build a camera with multiple sensors capturing different FoVs.

Samsung have published papers on a 1.2MegaPixel ToF sensors, but currently only have a VGA sensor available. https://ieeexplore.ieee.org/abstract/document/9365854 https://www.samsung.com/semiconductor/minisite/isocell/visio...

Re: Understanding Indirect Time-of-Flight Depth Sensing

#27

Post by rzw2 »

jcims wrote: post #11

I worked briefly with a company that manufactured surveying equipment back in the mid 2000's and they had a similar setup (single sensor though). They could get sub-millimeter precision as well, but they had to 'chirp' the light power modulation from very long (e.g. 1km) to short (sub millimeter) in order to remove any aliasing that would occur when the distance was a multiple of the measurement baseline. If these ar…

The microsoft kinect does 3 frequency measurements to get the higher distance precision of higher modulation frequencies and to extend the maximum distance so the phase wrapping (aliasing) can be solved.

Based on some Microsoft patent filings and other research papers I wrote a short article on possible phase unwrapping methods on iToF sensors, https://medium.com/chronoptics-time-of-flight/phase-wrapping...

Re: Understanding Indirect Time-of-Flight Depth Sensing

#28

Post by ilaksh »

CyberDildonics wrote: post #13
ilaksh wrote: post #10

I strongly feel that for computer vision to take the next step, it needs to work on a predictive basis, using directed sampling and disentangled semantic priors, etc. That would mean that before percepts are updated and available for applications (with knowledge like depth), a lot of information, not just the current frame, has already been integrated. Information such as previous frames, common shapes or surfaces, o…

What does this have to do with a microsoft page on indirect time of flight?

It's a different approach to the same problem. For starters, you don't assume that you can take one sample (or pair of samples) and just use that.

Because these depth readings are being fed to applications as if they were good representations. What I am saying is, for one thing, before you try to really use that type of information, you integrate more data and do a lot more work. And that's the perception level.

Re: Understanding Indirect Time-of-Flight Depth Sensing

#29

Post by ilaksh »

blueblisters wrote: post #16
ilaksh wrote: post #10

I strongly feel that for computer vision to take the next step, it needs to work on a predictive basis, using directed sampling and disentangled semantic priors, etc. That would mean that before percepts are updated and available for applications (with knowledge like depth), a lot of information, not just the current frame, has already been integrated. Information such as previous frames, common shapes or surfaces, o…

> a lot of information, not just the current frame, has already been integrated. Information such as previous frames, common shapes or surfaces, objects, the current full scene model, etc. What's the current state of the art on this?

Good question. Have seen some interesting papers and systems. MIT Kimera Semantics is interesting .

Re: Understanding Indirect Time-of-Flight Depth Sensing

#30

Post by CyberDildonics »

ilaksh wrote: post #28
CyberDildonics wrote: post #13

Earlier quoted context omitted.

What does this have to do with a microsoft page on indirect time of flight?

It's a different approach to the same problem. For starters, you don't assume that you can take one sample (or pair of samples) and just use that. Because these depth readings are being fed to applications as if they were good representations. What I am saying is, for one thing, before you try to really use that type of information, you integrate more data and do a lot more work. And that's the perception level.

> Because these depth readings are being fed to applications as if they were good representations

No they aren't. Depth is always extremely noisy and needs to be managed and filtered in lots of different ways.

> And that's the perception level.

Depth isn't always looked at directly. This is about camera techniques. What you are talking about seems like some pet project ideas that are loosely connected to depth cameras.

Post reply on HN