Beautiful!
Extra kudos to the author for not calling the work done in Torch "learning".
Reprojecting the Perseverance landing footage onto satellite imagery
21–30 of 40 posts
Re: Reprojecting the Perseverance landing footage onto satellite imagery
#22The graph based approach is interesting... But I wonder if better and far simpler results might be had by simply using a few iterations of optical flow to perfect the alignment of each frame starting from the alignment of the previous frame?
As a benefit, the transformation could use images after being projected onto a deformable mesh to model the hills etc.
Re: Reprojecting the Perseverance landing footage onto satellite imagery
#23Not only impressively coded but a beautiful result as well fascinating to have that real time, frame by frame comparison. Great job! It only reinforces that I really need to learn my matrix math.
Plug: Just wrote a piece on this topic that you might enjoy reading: https://www.heinrichhartmann.com/posts/2021-03-08-rank-decom...
Re: Reprojecting the Perseverance landing footage onto satellite imagery
#24I've been having trouble finding the answer to this. How close to it's intended target did it land? Thanks
5 meters. However, the "intended target" is not simply defined. The landing ellipse for Perseverance was 7.7km by 6.6km. The goal is to land at a safe spot within the ellipse rather than land at a specific location. The new Terrain Relative Navigation capability determines the rovers position relative to the surface during descent by comparing camera images to onboard satellite imagery. On Earth you'd use GPS. No GPS…
There is an obvious case where you can't rely on GPS on Earth.
Pershing-2 missiles had radar correlation guidance back in the 80's.
An obvious consequence of Google maps imagery and open source is that a capable college student can make an optical terminal guidance unit out of a mobile phone.
Re: Reprojecting the Perseverance landing footage onto satellite imagery
#25It'd be pretty neat to lift this up into 3d - you could probably reverse the transforms to find the camera pose for each frame, then drop it into a scene alongside the camera frustum and the topography so we can see exactly how much steering the descent stage did to hit its target and how fast it was descending at every stage.
On the other hand, I'd fully expect NASA/JPL to have IMU telemetry from the EDL. If I'm not completely mistaken, they would eventually get published in PDS here: https://naif.jpl.nasa.gov/naif/data.html
For example for MSL (Curiosity, the previous rover) the EDL CK and SPK provides orientation and position data if I'm interpreting this description right: https://naif.jpl.nasa.gov/pub/naif/pds/data/msl-m-spice-6-v1...
The downside being that it'll take probably 6-12 months until the data is put to public
(EDL = entry, descent, landing; IMU = inertial measurement unit; PDS = planetary data system)
Re: Reprojecting the Perseverance landing footage onto satellite imagery
#26I've done this kind of stuff through a point and click UI in GIS software. It's really cool seeing a lot of the underlying math and concepts laid out like this.
I'm curious - what did you do, in what software, and how?
I think what he is referring to is georeferencing and reprojecting, you can read how it works in e.g. QGIS here https://docs.qgis.org/3.16/en/docs/user_manual/working_with_...
Re: Reprojecting the Perseverance landing footage onto satellite imagery
#27I've done this kind of stuff through a point and click UI in GIS software. It's really cool seeing a lot of the underlying math and concepts laid out like this.
I'm curious - what did you do, in what software, and how?
Georeferencing processes of various complexity in ArcGIS, PCI Geomatica, and ENVI.
Re: Reprojecting the Perseverance landing footage onto satellite imagery
#28[deleted]
Re: Reprojecting the Perseverance landing footage onto satellite imagery
#29I love that you can see the approach angle in the distortion of the field. It also helps to convey how thin the atmosphere is to see how long it takes for that to square up.
Re: Reprojecting the Perseverance landing footage onto satellite imagery
#30Very impressive.
A next step could be to leave the already projected images where they are, and only draw over them, while marking the latest frame with a border. Eventually use frame sections which cover multiple frames to perform multi-frame superresolution.