Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Camera Is the Lidar

medium.com

Post reply on HN
Print view View original

31–40 of 77 posts

Re: The Camera Is the Lidar

#31

Post by deepnotderp »

flyinglizard wrote: post #27

Why is this better than separate LIDAR and camera? Because you're collecting NIR ambient light, your optics are wideband. Meaning that daylight would have a more pronounced negative effect on system range (easier to saturate the photocells). It's also low resolution (as most LIDARs are), and there is no color segmentation data. In an automotive application, I can't see a justification to unify both visual and LIDAR i…

It's better because there's no need for calibration, you always have perfect calibration.

Solid state lidar has issues. The cofounder of Ouster, Angus Pacala previously cofounded Quanergy, a solid state lidar startup.

Re: The Camera Is the Lidar

#32

Post by [deleted] »

barbegal wrote: post #14

I'm sure this works well in bright light but I'm sceptical that this can perform at all well on overcast days or at night. The OS-1 device spins at 10Hz and the LIDAR samples 2048 points over one 360 degree revolution. This means each column of pixels is sampled at 1/20480 of a second. To sample that fast requires a lot of light which is fine on a sunny day but on a cloudy day you can have 100 time less light. And at…

[deleted]

Re: The Camera Is the Lidar

#33

Post by dllu »

flyinglizard wrote: post #27

Why is this better than separate LIDAR and camera? Because you're collecting NIR ambient light, your optics are wideband. Meaning that daylight would have a more pronounced negative effect on system range (easier to saturate the photocells). It's also low resolution (as most LIDARs are), and there is no color segmentation data. In an automotive application, I can't see a justification to unify both visual and LIDAR i…

First, the optics are not wideband. It is only collecting narrow band NIR light. Saturation is not a problem. The CEO of Ouster explains this in a reddit comment [0]:

> We are not sacrificing lidar performance by adding ambient imaging functionality. The lidar subsystem has a short integration time that avoids saturation, and if anything our approach outperforms other lidars.

> As proof, the example videos linked to in the article show raw unedited point cloud data with the lidar operating in extremely sunny environments with plenty of specular reflectors. You can see lens flare in the ambient imagery as any camera would exhibit, but the lidar signal and range data are unaffected. In addition, if you point a velodyne directly at the sun its false positive rate increases significantly while our sensor's FPR does not. No lidar will return the distance to the sun so the only thing that matters is FPR in this scenario.

> We've independently verified the OS-1's range performance with customers under all levels of solar exposure and I guarantee you can't get a smaller, cheaper lidar with even close to this combination of resolution and performance. If you have any doubts, download the raw pcap files from our github page and play them back yourself. We stand behind our data, our pricing, and our spec!

Second, even if you do plan on adding extra cameras, the extrinsic calibration between camera and lidar may become easier if you have good quality ambient light measurement from the lidar. For example Jesse Levinson, cofounder of Zoox, computes extrinsic calibration between camera and Velodyne lidar by assuming that depth discontinuities are correlated with visual features [1]. But obviously the correlation between 850 nm images and visible light images would be way better.

[0] https://www.reddit.com/r/SelfDrivingCars/comments/9c60pe/the...

[1] http://www.roboticsproceedings.org/rss09/p29.pdf

Re: The Camera Is the Lidar

#34

Post by flyinglizard »

deepnotderp wrote: post #31
flyinglizard wrote: post #27

Why is this better than separate LIDAR and camera? Because you're collecting NIR ambient light, your optics are wideband. Meaning that daylight would have a more pronounced negative effect on system range (easier to saturate the photocells). It's also low resolution (as most LIDARs are), and there is no color segmentation data. In an automotive application, I can't see a justification to unify both visual and LIDAR i…

It's better because there's no need for calibration, you always have perfect calibration. Solid state lidar has issues. The cofounder of Ouster, Angus Pacala previously cofounded Quanergy, a solid state lidar startup.

Solid state LIDAR certainly has issues - but someone is going to solve those and this is what will get into automotive, definitely not $10k units with moving parts.

There was an announcement on a cooperation between BMW and Innoviz (an Israeli maker of solid state LIDARs) with Magna being their OEM sponsor.

I'm not sure calibration is that big of a deal for this application. Sensors are going to be calibrated and tested in the factory or at a module level regardless, and the accuracy requirements in automotive are much lower than consumer products using similar technology.

You can't overcome not having colors (traffic lights, anyone?), limited ranging distance or sensor saturation due to ambient conditions.

Re: The Camera Is the Lidar

#35

Post by flyinglizard »

dllu wrote: post #33
flyinglizard wrote: post #27

Why is this better than separate LIDAR and camera? Because you're collecting NIR ambient light, your optics are wideband. Meaning that daylight would have a more pronounced negative effect on system range (easier to saturate the photocells). It's also low resolution (as most LIDARs are), and there is no color segmentation data. In an automotive application, I can't see a justification to unify both visual and LIDAR i…

First, the optics are not wideband. It is only collecting narrow band NIR light. Saturation is not a problem. The CEO of Ouster explains this in a reddit comment [0]: > We are not sacrificing lidar performance by adding ambient imaging functionality. The lidar subsystem has a short integration time that avoids saturation, and if anything our approach outperforms other lidars. > As proof, the example videos linked to…

>> First, the optics are not wideband. It is only collecting narrow band NIR light. Saturation is not a problem. The CEO of Ouster explains this in a reddit comment [0]:

They are probably wider band than would be required to read only the sensor self illumination.

>> Second, even if you do plan on adding extra cameras, the extrinsic calibration between camera and lidar may become easier if you have good quality ambient light measurement from the lidar. For example Jesse Levinson, cofounder of Zoox, computes extrinsic calibration between camera and Velodyne lidar by assuming that depth discontinuities are correlated with visual features [1]. But obviously the correlation between 850 nm images and visible light images would be way better.

I agree with that - but you could probably go the other way around and coorelate the LIDAR depth map with depth obtained through stereo imaging. The temporal synchronization between NIR and depth provided by this unit is nice though.

Let me phrase this differently - while the videos are cool to watch, I don't think calibration is the problem in vehicles, and nor are baseline artifacts between sensors when operating at such far ranges (whether your camera and LIDAR are perfectly aligned or translated 10cm apart, it won't matter much looking 10m down the road).

Having moving parts, however, won't get this system into a production model.

Re: The Camera Is the Lidar

#36

Post by amelius »

maxerickson wrote: post #13
amelius wrote: post #12

Earlier quoted context omitted.

And after certification, what if the company wants to push a quick update to all of its cars every now and then, through a remote update? Would that be allowed? How would we even know that it happens?

Just make update transparency part of the initial certification. Of course they might cheat, but who really knows if their car has an airbag where it is supposed to.

> Just make update transparency part of the initial certification.

Somehow I doubt the authorities have the foresight.

Re: The Camera Is the Lidar

#37

Post by patrickg_zill »

haney wrote: post #15

So I'm not up on the Lidar industry but $12k for a sensor seems really expensive, but then again from my casual observations Lidar is really expensive. Is there a physical / first principles reason this is true or is it just really new technology?

The most general answer to your question is Peter Drucker's (famous management guru) observation that every doubling of production of an item (over the lifetime, not yearly) resulted in a cost reduction of 20-30%.

So right now, with very few LIDARs produced, we have a high price, which will start dropping as more are produced.

You might find this interesting: a single transistor used to sell for roughly the equivalent of $8 USD in today's money; today the cheapest ones are 6 Cents USD (price checked today from Mouser.com) in qty 1 pricing...

https://spectrum.ieee.org/tech-talk/semiconductors/devices/h...

Re: The Camera Is the Lidar

#38

Post by foota »

deepnotderp wrote: post #29
Animats wrote: post #26

Earlier quoted context omitted.

One might think that, except that Advanced Scientific Concepts, which Continental bought, has had it working for a decade.[1] Their units work fine, but are expensive. They're mostly sold to DoD and used for space applications. The Space-X Dragon spacecraft uses one for docking. There's a tradeoff between field of view and range. Automotive systems will probably include a long-range narrow field of view unit and a sh…

Also note that in space there's way less background noise.

This made me curious, how wide is Earth's shadow the the height the space station is at?

Re: The Camera Is the Lidar

#39

Post by nradov »

barbegal wrote: post #11
amelius wrote: post #4

The problem is that deep learning should not be allowed in safety critical systems, because (1) the accuracy is always less than 100% even in known test situations, and (2) we don't know how it works and under what conditions it breaks down.

I think it should be allowed but the tests it should pass must be far more strenuous than for traditional software. I'm happy with failure rates of around 1 catastrophic failure every million hours. Even humans sometimes fail catastrophically and black out at the wheel for no detectable reason. That level of testing is well beyond what today's software and hardware is capable of. Waymo has to override their cars (dis…

Perhaps, but it's going to be difficult to justify that logic to a jury in a wrongful death civil lawsuit.

Re: The Camera Is the Lidar

#40

Post by qualitative »

thanatropism wrote: post #2

Is it just me, or are we seeing yet another impressive leap in Computer Vision that's soon going to be hyped as an incremental step into Skynet?

None of this technology is new. It's been done to death as terrain following guidance systems for cruise missiles, now reapplied in a civilian context.

It's technology that already exists, but must be reinvented in a non-military context from scratch, since the tech transfer between weapons systems and civilian applications is likely locked up in policy. So, we know that this technology exists, and is proven, but we have to reinvent the wheel, because reasons.

The reason we see this interminable slow motion public struggle to bring it to consumer applications, is likely because there are no controls in place that can actually prevent "contemporaneous discovery" wink, wink.

Post reply on HN