Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Page Dewarping

mzucker.github.io

Post reply on HN
Print view View original

21–30 of 75 posts

Re: Page Dewarping

#21

Post by claar »

troymc wrote: post #15

I use Microsoft's "Office Lens" app on my Android phone all the time, like a "smart camera" which automatically squares off and white-balances each photo of a page (usually mail or forms filled in by hand). It can't handle warped pages though, so I hope they add something like this!

Yes, Office Lens is great for that. I recently switched to CamScanner, which has an even better algorithm for deskewing pages (take a pic of a receipt at an odd angle, it automatically transforms it to flat and rectangular). Pro version was on sale -- I'm not affiliated, just a happy customer and highly related to this article.

Re: Page Dewarping

#22

Post by chrisfosterelli »

Jarwain wrote: post #20
voxic11 wrote: post #14

Earlier quoted context omitted.

Doesn't that describe most modern cell phone cameras?

I haven't heard of a phone camera that can do video at 120fps, they're usually 30, maybe 60 fps. The S7 has some high framerate slow-mo option, but it's limited to a 720p resolution.

The iPhone 6S records 1080p at 120 fps and 720p at 240 fps for their Slow-Mo feature [0].

[0]: http://www.apple.com/ca/iphone-6s/specs/

Re: Page Dewarping

#23

Post by mzucker »

saynsedit wrote: post #16

Is using a curve whose end points are fixed to zero to model the warping accurate? I can't see a rationale for why the end points should both be 0.

The main rationale is removing redundant degrees of freedom -- since the page is allowed to rotate freely, the edges of the page can still move around plenty.

Re: Page Dewarping

#24

Post by mzucker »

tomn wrote: post #17
mzucker wrote: post #10

Thanks for the feedback, everyone -- happy to answer questions here or in the Disqus comments on my blog.

This is really interesting, thanks! If you decide to try to make this faster, check out ceres[0] a non-linear least squares optimisation framework that does automatic differentiation using a clever C++ template hack. I've used it a few times to solve these kind of problems and found it to be very good! [0] http://ceres-solver.org/

Yep, I'm still waiting to use ceres for something - I didn't end up using it on my image approximation project https://mzucker.github.io/2016/08/01/gabor-2.html because it doesn't work well with inequality constraints.

Re: Page Dewarping

#25

Post by loeg »

Jarwain wrote: post #20
voxic11 wrote: post #14

Earlier quoted context omitted.

Doesn't that describe most modern cell phone cameras?

I haven't heard of a phone camera that can do video at 120fps, they're usually 30, maybe 60 fps. The S7 has some high framerate slow-mo option, but it's limited to a 720p resolution.

The iphone 6S can do 120 fps for a short period of time.

Re: Page Dewarping

#26

Post by mzucker »

jff wrote: post #13
jgable wrote: post #8

This is great! My wife is a music teacher and often scans sheet music so that it's more portable. She has been asking me for a while for something exactly like this. I'll have to tweak it to work on sheet music, since I imagine his methods to identify lines of text won't work for the music staff out of the box.

He mentions adding line detection for tables, which should be a good start for sheet music.

Yep, might just work out of the box -- give it a shot!

Re: Page Dewarping

#27

Post by tomn »

mzucker wrote: post #24
tomn wrote: post #17

Earlier quoted context omitted.

This is really interesting, thanks! If you decide to try to make this faster, check out ceres[0] a non-linear least squares optimisation framework that does automatic differentiation using a clever C++ template hack. I've used it a few times to solve these kind of problems and found it to be very good! [0] http://ceres-solver.org/

Yep, I'm still waiting to use ceres for something - I didn't end up using it on my image approximation project https://mzucker.github.io/2016/08/01/gabor-2.html because it doesn't work well with inequality constraints.

Ah yes, only supporting constraints on the parameters can be annoying if your problem requires that.

Re: Page Dewarping

#28

Post by jgable »

buovjaga wrote: post #18
jgable wrote: post #8

This is great! My wife is a music teacher and often scans sheet music so that it's more portable. She has been asking me for a while for something exactly like this. I'll have to tweak it to work on sheet music, since I imagine his methods to identify lines of text won't work for the music staff out of the box.

You could also try ScanTailor: http://scantailor.org/

Thanks for the suggestion! I'll check it out.

Re: Page Dewarping

#30

Post by kazinator »

This strikes close to home for me, because I've done this "by hand" quite a number of times, using some interactive Gimp filters, with very good results. I've been able to take a perspective photo of a curved page lying on a desk, get it almost perfectly straight, and threshold it to clean black and white or sharp gray scale.

Here is my very quick and dirty manual job of the same example page:

http://www.kylheku.com/~kaz/dewarp.png

Literally less than five minutes.

First I cropped the image. Then duplicated the layer. Blurred the top layer (Gaussian, 50 radius). Then flipped to Divide mode and merged the visible layers. This leveled the lightness quite well, almost completely eliminating the shadow over he right side of the page and all other lighting differences. There is a hint of the edge of the shadow still present because it is such a sharp contrast; but that can be eliminated in an adjustment of the intensity curves. In such cases it may be helpful to experiment with smaller blur radii, too.

I then did a perspective transform in the lateral direction, squeezing the left side top-bottom and expanding the right, resulting in the warp now being approximately horizontal. (The perspective transform is not just for adding a perspective effect; it is also useful reversing perspective!)

Finally, I used the Curve Bend (with its horrible interactive interface and awful preview) to warp in a compensating way. Basically, the idea is to draw an upper and lower curve which is the opposite of the curve on the page. I made two attempts, keeping the results of the second.

If the preview of this tool wasn't a ridiculous, inscrutable thumbnail, it would be possible to do an excellent job in one attempt, probably close to perfect.

Because the page is evenly light thanks to the divide-by-blurred layer trick, it will nicely threshold to black and white, or a narrow grayscale range.

Post reply on HN