Understanding Indirect Time-of-Flight Depth Sensing • Hacker News Board

Re: Understanding Indirect Time-of-Flight Depth Sensing

#31

Post by echelon » Tue, Apr 20, 2021, 5:25 PM UTC

rzw2 wrote: post #26

echelon wrote: post #24

Are there any higher than VGA quality ToF sensors? Azure Kinect is not cutting it for us. We'd like to do real time, full 360 depth capture of actors, but all of the sensors we've found are poorly suited for this task. We might have to rig up our own optical system and use FPGAs to run them...

Microsoft licensed their sensor to Analog devices, and you can buy the mega-pixel iToF sensor and build a iToF camera. https://www.analog.com/en/products/ADSD3100.html?icid=ToF-ad... Or build a camera with multiple sensors capturing different FoVs. Samsung have published papers on a 1.2MegaPixel ToF sensors, but currently only have a VGA sensor available. https://ieeexplore.ieee.org/abstract/document/9365854 https://…

Interesting! I was completely unaware of these. Thanks for the info.

I'm glad to see we're steadily climbing in terms of sensor resolution. We're not anywhere close to 4K 120Hz sampling, but the field is progressing.

There's nothing stopping us from doing offline processing for our workflow, I suppose.

Re: Understanding Indirect Time-of-Flight Depth Sensing

#32

Post by lnsru » Tue, Apr 20, 2021, 6:39 PM UTC

rzw2 wrote: post #26

echelon wrote: post #24

Are there any higher than VGA quality ToF sensors? Azure Kinect is not cutting it for us. We'd like to do real time, full 360 depth capture of actors, but all of the sensors we've found are poorly suited for this task. We might have to rig up our own optical system and use FPGAs to run them...

Microsoft licensed their sensor to Analog devices, and you can buy the mega-pixel iToF sensor and build a iToF camera. https://www.analog.com/en/products/ADSD3100.html?icid=ToF-ad... Or build a camera with multiple sensors capturing different FoVs. Samsung have published papers on a 1.2MegaPixel ToF sensors, but currently only have a VGA sensor available. https://ieeexplore.ieee.org/abstract/document/9365854 https://…

How are Analog Devices selling these chips? Like Sony with Heavy NDA? Or these will be regular catalogue parts in the future?

Re: Understanding Indirect Time-of-Flight Depth Sensing

#33

Post by jcims » Tue, Apr 20, 2021, 6:49 PM UTC

rzw2 wrote: post #27

jcims wrote: post #11

I worked briefly with a company that manufactured surveying equipment back in the mid 2000's and they had a similar setup (single sensor though). They could get sub-millimeter precision as well, but they had to 'chirp' the light power modulation from very long (e.g. 1km) to short (sub millimeter) in order to remove any aliasing that would occur when the distance was a multiple of the measurement baseline. If these ar…

The microsoft kinect does 3 frequency measurements to get the higher distance precision of higher modulation frequencies and to extend the maximum distance so the phase wrapping (aliasing) can be solved. Based on some Microsoft patent filings and other research papers I wrote a short article on possible phase unwrapping methods on iToF sensors, https://medium.com/chronoptics-time-of-flight/phase-wrapping...

Fantastic article!!! They definitely need your animations, it's way more intuitive.

Re: Understanding Indirect Time-of-Flight Depth Sensing

#34

Post by ilaksh » Tue, Apr 20, 2021, 6:54 PM UTC

CyberDildonics wrote: post #30

ilaksh wrote: post #28

Earlier quoted context omitted.

It's a different approach to the same problem. For starters, you don't assume that you can take one sample (or pair of samples) and just use that. Because these depth readings are being fed to applications as if they were good representations. What I am saying is, for one thing, before you try to really use that type of information, you integrate more data and do a lot more work. And that's the perception level.

> Because these depth readings are being fed to applications as if they were good representations No they aren't. Depth is always extremely noisy and needs to be managed and filtered in lots of different ways. > And that's the perception level. Depth isn't always looked at directly. This is about camera techniques. What you are talking about seems like some pet project ideas that are loosely connected to depth camera…

True, they are actually kind of pet project ideas now. But they are drawn from popular neuroscience and similar to some existing approaches or beginnings. I believe that this type of predictive and generative system will be the paradigm for perception in the near future.

And of course it's not just the raw data being used at the application level in current real-world applications. But what is fed in doesn't take into account most of the type of information I am talking about.

Re: Understanding Indirect Time-of-Flight Depth Sensing

#35

Post by rzw2 » Tue, Apr 20, 2021, 10:51 PM UTC

echelon wrote: post #31

rzw2 wrote: post #26

Earlier quoted context omitted.

Microsoft licensed their sensor to Analog devices, and you can buy the mega-pixel iToF sensor and build a iToF camera. https://www.analog.com/en/products/ADSD3100.html?icid=ToF-ad... Or build a camera with multiple sensors capturing different FoVs. Samsung have published papers on a 1.2MegaPixel ToF sensors, but currently only have a VGA sensor available. https://ieeexplore.ieee.org/abstract/document/9365854 https://…

Interesting! I was completely unaware of these. Thanks for the info. I'm glad to see we're steadily climbing in terms of sensor resolution. We're not anywhere close to 4K 120Hz sampling, but the field is progressing. There's nothing stopping us from doing offline processing for our workflow, I suppose.

At those resolutions and frame rates multiview stereo is probably your best technology for the moment.

Re: Understanding Indirect Time-of-Flight Depth Sensing

#36

Post by rzw2 » Tue, Apr 20, 2021, 10:53 PM UTC

lnsru wrote: post #32

rzw2 wrote: post #26

Earlier quoted context omitted.

Microsoft licensed their sensor to Analog devices, and you can buy the mega-pixel iToF sensor and build a iToF camera. https://www.analog.com/en/products/ADSD3100.html?icid=ToF-ad... Or build a camera with multiple sensors capturing different FoVs. Samsung have published papers on a 1.2MegaPixel ToF sensors, but currently only have a VGA sensor available. https://ieeexplore.ieee.org/abstract/document/9365854 https://…

How are Analog Devices selling these chips? Like Sony with Heavy NDA? Or these will be regular catalogue parts in the future?

Im guessing like Sony, it's hard to support such chips because of the underlying complexities. Melexis has open datasheets and can purchase individual sensors off mouser.

Re: Understanding Indirect Time-of-Flight Depth Sensing

#37

Post by rzw2 » Tue, Apr 20, 2021, 11:09 PM UTC

jcims wrote: post #33

rzw2 wrote: post #27

Earlier quoted context omitted.

The microsoft kinect does 3 frequency measurements to get the higher distance precision of higher modulation frequencies and to extend the maximum distance so the phase wrapping (aliasing) can be solved. Based on some Microsoft patent filings and other research papers I wrote a short article on possible phase unwrapping methods on iToF sensors, https://medium.com/chronoptics-time-of-flight/phase-wrapping...

Fantastic article!!! They definitely need your animations, it's way more intuitive.

Thanks, got any references to how the surveying equipment de-aliasing method worked?

Re: Understanding Indirect Time-of-Flight Depth Sensing

#38

Post by jcims » Wed, Apr 21, 2021, 12:12 AM UTC

rzw2 wrote: post #37

jcims wrote: post #33

Earlier quoted context omitted.

Fantastic article!!! They definitely need your animations, it's way more intuitive.

Thanks, got any references to how the surveying equipment de-aliasing method worked?

I don’t unfortunately. I was there doing an information security assessment and the guy I was working with was one of the product engineers. When I wrapped my work we spent another hour or two just talking about their products and how they worked.

Re: Understanding Indirect Time-of-Flight Depth Sensing

#39

Post by jcims » Wed, Apr 21, 2021, 2:10 AM UTC

rzw2 wrote: post #21

rsp1984 wrote: post #2

The article doesn't mention the main drawback of ToF depth sensing: Multipath Errors. They originate from light bouncing around in the scene before coming back to the detector, causing the resulting depth maps to have dents and distortions in the neighborhood of angled surfaces. They are a big problem in built environments which have lots of 90 deg angles that act as retroreflectors to the signal. To my knowledge non…

There has been some work on resolving multipath errors with indirect ToF, Chronoptics (I'm a cofounder) recently licensed our technology to Melexis in automotive. Here's a blog post I wrote about resolving multipath https://medium.com/chronoptics-time-of-flight/multipath-inte... And a link to the announcement from Melexis https://www.melexis.com/en/news/2021/4mar2021-melexis-announ...

Could this be used with SAR as well?

Re: Understanding Indirect Time-of-Flight Depth Sensing

#40

Post by rzw2 » Wed, Apr 21, 2021, 2:50 AM UTC

jcims wrote: post #39

rzw2 wrote: post #21

Earlier quoted context omitted.

There has been some work on resolving multipath errors with indirect ToF, Chronoptics (I'm a cofounder) recently licensed our technology to Melexis in automotive. Here's a blog post I wrote about resolving multipath https://medium.com/chronoptics-time-of-flight/multipath-inte... And a link to the announcement from Melexis https://www.melexis.com/en/news/2021/4mar2021-melexis-announ...

Could this be used with SAR as well?

Maybe, would be very interesting to investigate.