Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accurate Image Alignment and Registration Using OpenCV

magamig.github.io

Post reply on HN
Print view View original

21–30 of 35 posts

Re: Accurate Image Alignment and Registration Using OpenCV

#21

Post by magamig »

thebigman433 wrote: post #13

I currently working on an optical respiratory monitor and one of the challenges was image alignment/registration for an optical and a thermal image with wildly different resolutions, fovs. We ended up doing this by manually calibrating the images in matlab using a custom script and logging out a transformation matrix that we could then multiply the optical image by to get matching pixels in the thermal image. Really…

Maybe you could use this for when there are "enough" features, and keep using the manual method for the remaining situations. I'd have to take a look at the images to have a better idea of what you are dealing with.

Re: Accurate Image Alignment and Registration Using OpenCV

#22

Post by zwieback »

shrx wrote: post #8

The biggest issue in real life use cases is lens distortion - these alignment algorithms usually don't correct barrel, pin-cushion or more complex lens distortions, making the alignment imperfect.

OpenCV, which is what the OP is using, has higher order camera calibration. There are functions to calibrate based on checkerboards or dot grids and undistort images using those calibrations.

Sometimes it's better to match first, then undistort, though.

Re: Accurate Image Alignment and Registration Using OpenCV

#24

Post by account42 »

Note that this is exactly what panorama software like Hugin [0] does - it even comes with a convenient command-line tool for this simple use case (aligning a stack of images that are mostly overlapping): https://wiki.panotools.org/Align_image_stack

[0] http://hugin.sourceforge.net/

Re: Accurate Image Alignment and Registration Using OpenCV

#25

Post by xhkkffbf »

vanderZwan wrote: post #5
gliptic wrote: post #4

Earlier quoted context omitted.

This is called drizzle (with dithering, e.g. the raw photos are taken with offsets) in the astrophotography sphere and is very effective when the images are undersampled. Although, astronomy images have the advantage of being filled with stars that are relatively easy to align very accurately.

I imagine the minuscule vibrations of a shutter make it a natural fit to the already existing image stacking pipeline, correct?

Yes, I believe Google's team publicly says it helps add just enough random translation.

Re: Accurate Image Alignment and Registration Using OpenCV

#26

Post by IAmEveryone »

xhkkffbf wrote: post #25
vanderZwan wrote: post #5

Earlier quoted context omitted.

I imagine the minuscule vibrations of a shutter make it a natural fit to the already existing image stacking pipeline, correct?

Yes, I believe Google's team publicly says it helps add just enough random translation.

Do camera phones have mechanical shutters? I'm almost certain they don't.

Re: Accurate Image Alignment and Registration Using OpenCV

#27

Post by Syzygies »

legulere wrote: post #19
vanderZwan wrote: post #2

Image alignment allows for some fun image manipulations like these. I think one of the coolest novel applications of image alignment in recent years must be multi-frame super-resolution. For example, the Pixel phones use it to improve low-light and zoomed in photos[0]. [0] https://sites.google.com/view/handheld-super-res/

I wonder if you could use alignment of multiple samples for better vectorizing of letters of old books. You could create very nice vector only pdfs.

Yes! For a two-fer, take a pair of cell phone photos of the open book, compute a 3D model, and flatten it as if the book had been sliced and scanned. Now match up all the letter E's in the entire book for image enhancement.

Vectorization is nontrivial. Most academic journal articles are now poorly scanned, and it would be nice to simply improve the scans.

Re: Accurate Image Alignment and Registration Using OpenCV

#28

Post by SCUSKU »

versatran01 wrote: post #20
thebigman433 wrote: post #13

I currently working on an optical respiratory monitor and one of the challenges was image alignment/registration for an optical and a thermal image with wildly different resolutions, fovs. We ended up doing this by manually calibrating the images in matlab using a custom script and logging out a transformation matrix that we could then multiply the optical image by to get matching pixels in the thermal image. Really…

Direct methods could be useful in your case. https://pages.cs.wisc.edu/~dyer/ai-qual/irani-visalg00.pdf

At work I had to make a custom image registration pipeline, that uses only 2 degrees of freedom, so just x,y translation. OpenCV did not have anything that did this, but a python library called Kornia does this well.

https://kornia-tutorials.readthedocs.io/en/latest/image_regi...

Re: Accurate Image Alignment and Registration Using OpenCV

#29

Post by hikarudo »

SCUSKU wrote: post #28
versatran01 wrote: post #20

Earlier quoted context omitted.

Direct methods could be useful in your case. https://pages.cs.wisc.edu/~dyer/ai-qual/irani-visalg00.pdf

At work I had to make a custom image registration pipeline, that uses only 2 degrees of freedom, so just x,y translation. OpenCV did not have anything that did this, but a python library called Kornia does this well. https://kornia-tutorials.readthedocs.io/en/latest/image_regi...

The opencv_contrib repo does have a module, called "reg", for direct alignment.

Re: Accurate Image Alignment and Registration Using OpenCV

#30

Post by thebigman433 »

magamig wrote: post #21
thebigman433 wrote: post #13

I currently working on an optical respiratory monitor and one of the challenges was image alignment/registration for an optical and a thermal image with wildly different resolutions, fovs. We ended up doing this by manually calibrating the images in matlab using a custom script and logging out a transformation matrix that we could then multiply the optical image by to get matching pixels in the thermal image. Really…

Maybe you could use this for when there are "enough" features, and keep using the manual method for the remaining situations. I'd have to take a look at the images to have a better idea of what you are dealing with.

The main issue is that the thermal camera we're using is only a few hundred pixels tall/wide, so using it for feature detection was rough when we did try that for something. For our purposes, with cameras on a solid mount, the hardcoded transformation matrix is good enough, but I think if you had better cameras and more time/more compute power, something like this could be better.

A bigger issue is that the project is running on a Pi, so just getting the image alignment running along with our face detection wouldve also been a pretty tall task if we want to stay at around ~5fps.

Post reply on HN