Understanding Indirect Time-of-Flight Depth Sensing • Hacker News Board

Re: Understanding Indirect Time-of-Flight Depth Sensing

#11

Post by jcims » Tue, Apr 20, 2021, 1:55 PM UTC

I worked briefly with a company that manufactured surveying equipment back in the mid 2000's and they had a similar setup (single sensor though). They could get sub-millimeter precision as well, but they had to 'chirp' the light power modulation from very long (e.g. 1km) to short (sub millimeter) in order to remove any aliasing that would occur when the distance was a multiple of the measurement baseline.

If these are 90 degrees out of phase I don't know how they eliminate that possibility without doing something similar. (e.g. imagine the modulation is at 30MHz and your measurement interval is 10m, how would you differentiate between 20.034m and 30.034m?

Re: Understanding Indirect Time-of-Flight Depth Sensing

#12

Post by jayd16 » Tue, Apr 20, 2021, 2:01 PM UTC

rsp1984 wrote: post #2

The article doesn't mention the main drawback of ToF depth sensing: Multipath Errors. They originate from light bouncing around in the scene before coming back to the detector, causing the resulting depth maps to have dents and distortions in the neighborhood of angled surfaces. They are a big problem in built environments which have lots of 90 deg angles that act as retroreflectors to the signal. To my knowledge non…

Would it help to use multiple sensors at different positions and use the common points to filter out the 2nd+ bounces?

I think this would work for, say, a mirror but what about something like brushed metal?

Are those other bounces scattered enough that multiple perspective still produce an error?

Re: Understanding Indirect Time-of-Flight Depth Sensing

#13

Post by CyberDildonics » Tue, Apr 20, 2021, 2:19 PM UTC

ilaksh wrote: post #10

I strongly feel that for computer vision to take the next step, it needs to work on a predictive basis, using directed sampling and disentangled semantic priors, etc. That would mean that before percepts are updated and available for applications (with knowledge like depth), a lot of information, not just the current frame, has already been integrated. Information such as previous frames, common shapes or surfaces, o…

What does this have to do with a microsoft page on indirect time of flight?

Re: Understanding Indirect Time-of-Flight Depth Sensing

#14

Post by opwieurposiu » Tue, Apr 20, 2021, 2:20 PM UTC

jcims wrote: post #11

I worked briefly with a company that manufactured surveying equipment back in the mid 2000's and they had a similar setup (single sensor though). They could get sub-millimeter precision as well, but they had to 'chirp' the light power modulation from very long (e.g. 1km) to short (sub millimeter) in order to remove any aliasing that would occur when the distance was a multiple of the measurement baseline. If these ar…

The kinect does do the "chirp", but this detail is omitted from TFA.

Re: Understanding Indirect Time-of-Flight Depth Sensing

#15

Post by cpgxiii » Tue, Apr 20, 2021, 2:21 PM UTC

stefan_ wrote: post #5

rsp1984 wrote: post #2

The article doesn't mention the main drawback of ToF depth sensing: Multipath Errors. They originate from light bouncing around in the scene before coming back to the detector, causing the resulting depth maps to have dents and distortions in the neighborhood of angled surfaces. They are a big problem in built environments which have lots of 90 deg angles that act as retroreflectors to the signal. To my knowledge non…

The article is also a bit optimistic in regards to outdoors use, with direct sunlight exposure. The sensors I tested in the past just didn't work at all.

You can absolutely make ToF sensors work outdoors in direct sunlight, but you need to operate at different frequencies that require more expensive emitters and detectors to avoid being saturated by light from the sun.

Re: Understanding Indirect Time-of-Flight Depth Sensing

#16

Post by blueblisters » Tue, Apr 20, 2021, 2:25 PM UTC

ilaksh wrote: post #10

I strongly feel that for computer vision to take the next step, it needs to work on a predictive basis, using directed sampling and disentangled semantic priors, etc. That would mean that before percepts are updated and available for applications (with knowledge like depth), a lot of information, not just the current frame, has already been integrated. Information such as previous frames, common shapes or surfaces, o…

> a lot of information, not just the current frame, has already been integrated. Information such as previous frames, common shapes or surfaces, objects, the current full scene model, etc.

What's the current state of the art on this?

Re: Understanding Indirect Time-of-Flight Depth Sensing

#17

Post by hairytrog » Tue, Apr 20, 2021, 2:28 PM UTC

rsp1984 wrote: post #2

The article doesn't mention the main drawback of ToF depth sensing: Multipath Errors. They originate from light bouncing around in the scene before coming back to the detector, causing the resulting depth maps to have dents and distortions in the neighborhood of angled surfaces. They are a big problem in built environments which have lots of 90 deg angles that act as retroreflectors to the signal. To my knowledge non…

It only matters if you need physically accurate data - ie if your brain can't process the multi path error and correct for it. A bat sees in multi path error and has no problem with it. I assume a machine vision system can learn to perform with multipath error and indirectly account for it. ie, it sees an apple with multipath error, and still knows it's an Apple.

There are options to fix multipath and recover the underlying ground truth

1) you can do a reverse raytrace and iteratively correct for the error - somewhat expensive, but there's tricks and shortcuts to accelerate

2) hardware fix to measure the multipath component separately and subtract / correct it - there's several ways to do this - there are some patents on it that I've worked on. The same methods also can remove background signal from ambient light.

Re: Understanding Indirect Time-of-Flight Depth Sensing

#18

Post by hairytrog » Tue, Apr 20, 2021, 2:31 PM UTC

mscharrer wrote: post #6

How do these ToF systems deal with multiple sensors pointing at the same scene? I've seen it work with two ToF sensors, but haven't been able to find a good explanation for how it works.

if the measurement time is short enough, (<1us capturing at 60 fps), the probability of interaction is low even with large number of cameras. Even then, some temporal filtering and intelligent time offsetting to separate signals, can usually fix the problem.

Re: Understanding Indirect Time-of-Flight Depth Sensing

#19

Post by hairytrog » Tue, Apr 20, 2021, 2:32 PM UTC

stefan_ wrote: post #5

rsp1984 wrote: post #2

The article doesn't mention the main drawback of ToF depth sensing: Multipath Errors. They originate from light bouncing around in the scene before coming back to the detector, causing the resulting depth maps to have dents and distortions in the neighborhood of angled surfaces. They are a big problem in built environments which have lots of 90 deg angles that act as retroreflectors to the signal. To my knowledge non…

The article is also a bit optimistic in regards to outdoors use, with direct sunlight exposure. The sensors I tested in the past just didn't work at all.

Which sensor was that stefan_? maybe I worked on it

Re: Understanding Indirect Time-of-Flight Depth Sensing

#20

Post by cpgxiii » Tue, Apr 20, 2021, 2:34 PM UTC

rsp1984 wrote: post #2

The article doesn't mention the main drawback of ToF depth sensing: Multipath Errors. They originate from light bouncing around in the scene before coming back to the detector, causing the resulting depth maps to have dents and distortions in the neighborhood of angled surfaces. They are a big problem in built environments which have lots of 90 deg angles that act as retroreflectors to the signal. To my knowledge non…

There are practical mitigations for the problem, especially if you want to filter away points from these surfaces (and are OK with dropout regions in the depth image). Some of these sensors produce useful per-pixel confidence values which do a reasonable job identifying regions with multipath errors, and various types of spatial/temporal filters work so-so in handling small distortions. The K4A sensors are perhaps a bit overeager in their spatial filter, leading to slightly over-smoothed edges, though.

You can always try combining ToF sensors with other types, like stereo, and hope that the failure modes of the different types are mostly distinct.

The EpiScan3D and EpiToF cameras are probably the closest to "solving" reflective subjects, but they are basically one-off benchtop prototypes and nowhere near products.