Page Dewarping

#1

Post by pietrofmaggi » Thu, Aug 18, 2016, 1:45 PM UTC

Re: Page Dewarping

#2

Post by Syzygies » Thu, Aug 18, 2016, 5:00 PM UTC

This problem is fundamentally different with stereo images; there is a hope of reconstructing the exact 3D geometry of the page before flattening, rather than inferring from content. An iPhone app that did this would do well.

Re: Page Dewarping

#3

Post by radarsat1 » Thu, Aug 18, 2016, 5:13 PM UTC

Syzygies wrote: post #2

This problem is fundamentally different with stereo images; there is a hope of reconstructing the exact 3D geometry of the page before flattening, rather than inferring from content. An iPhone app that did this would do well.

I've always thought it should be possible using a short video of the camera orbiting the page from different angles. (Assuming a static scene of course.)

Re: Page Dewarping

#4

Post by niftich » Thu, Aug 18, 2016, 5:27 PM UTC

Recently, Dropbox wrote about dewarping prior to OCR in their app: https://news.ycombinator.com/item?id=12297944

This code had the same idea, and is open-source!

Re: Page Dewarping

#5

Post by cooper12 » Thu, Aug 18, 2016, 5:29 PM UTC

I love well-illustrated writeups. Even a reader without mathematics or programming knowledge can understand what steps the author took. His model actually seems to better represent the warped paper than a cylinder would. (though I don't know the actual specifics of the CTM model)

I wish he went into more details on the steps taken after dewarping. You can tweak the image levels to get good contrast, but surprisingly there aren't any shadows from underleveling or loss of detail from overleveling. I wonder if the author ran OCR on the scans after, and speaking of OCR, IIRC Leptonica is one of the dependencies of Tesseract so it must do some similar pre-processing.

Edit: reading more carefully, he mentions that he used adaptive thresholding from OpenCV.

Re: Page Dewarping

#6

Post by manmal » Thu, Aug 18, 2016, 6:44 PM UTC

radarsat1 wrote: post #3

Syzygies wrote: post #2

This problem is fundamentally different with stereo images; there is a hope of reconstructing the exact 3D geometry of the page before flattening, rather than inferring from content. An iPhone app that did this would do well.

I've always thought it should be possible using a short video of the camera orbiting the page from different angles. (Assuming a static scene of course.)

Or now, some frames of a live photo.

Re: Page Dewarping

#7

Post by Ciantic » Thu, Aug 18, 2016, 7:14 PM UTC

I wonder if someone has tried to dewarp whole book from a video when flipping through it. I imagine that could be handy way to copy the whole book.

Re: Page Dewarping

#8

Post by jgable » Thu, Aug 18, 2016, 7:18 PM UTC

This is great! My wife is a music teacher and often scans sheet music so that it's more portable. She has been asking me for a while for something exactly like this. I'll have to tweak it to work on sheet music, since I imagine his methods to identify lines of text won't work for the music staff out of the box.

Re: Page Dewarping

#9

Post by artursapek » Thu, Aug 18, 2016, 7:30 PM UTC

I've always been interested in getting into graphics programming, and stuff like this only makes me more interested. Really well written post.

Re: Page Dewarping

#10

Post by mzucker » Thu, Aug 18, 2016, 7:35 PM UTC

Thanks for the feedback, everyone -- happy to answer questions here or in the Disqus comments on my blog.