Compressing and enhancing hand-written notes (2016) • Hacker News Board

Re: Compressing and enhancing hand-written notes (2016)

#61

Post by kazinator » Mon, Mar 12, 2018, 9:06 PM UTC

alister wrote: post #17

I have an observation about scanning documents that results in good quality and smaller files, but I can't satisfactorily explain why it works. Consider these two cases: (1) Scan document at very high resolution as a JPG and then use a third-party program (like Photoshop or whatever) to re-encode the JPG at your preferred low resolution. (2) Scan document at your preferred low resolution as a JPG straight away. Don't…

Of course you get better results with 1.

Here is why: if you go with (2), then (1) is still done: by the crap firmware in your printer or its driver. It scans at high resolution and then downsamples in some way over which you have no control. It might not even be done with floating-point math.

1. is a somewhat like getting the raw image from a camera: a higher quality source for your own processing.

Re: Compressing and enhancing hand-written notes (2016)

#62

Post by GrumpyNl » Mon, Mar 12, 2018, 9:11 PM UTC

MagerValp wrote: post #44

nayuki wrote: post #37

On the top image, I see that the back side of the page has clearly leaked through. In my experiences with scanning paper, I found a trick that essentially eliminates any visible backside content: Using a flatbed scanner, I would scan with the lid open, and the room darkened. The worst thing to do is to scan with the lid closed, with a lid that has a white background. This would increase the reflection from the backsi…

You achieve the same effect with a black paper on top of the document you’re scanning, or in between the pages if it’s a book. As a bonus you can leave the light on :)

Somebody is scratching his head now.

Re: Compressing and enhancing hand-written notes (2016)

#63

Post by nayuki » Mon, Mar 12, 2018, 9:28 PM UTC

MagerValp wrote: post #44

nayuki wrote: post #37

On the top image, I see that the back side of the page has clearly leaked through. In my experiences with scanning paper, I found a trick that essentially eliminates any visible backside content: Using a flatbed scanner, I would scan with the lid open, and the room darkened. The worst thing to do is to scan with the lid closed, with a lid that has a white background. This would increase the reflection from the backsi…

You achieve the same effect with a black paper on top of the document you’re scanning, or in between the pages if it’s a book. As a bonus you can leave the light on :)

Indeed, https://en.wikipedia.org/wiki/Vantablack would be great here.

Re: Compressing and enhancing hand-written notes (2016)

#64

Post by l9k » Mon, Mar 12, 2018, 9:34 PM UTC

Softcadbury wrote: post #12

I wonder if your technique could remove some lines for the paper we use in France [1]. I never really understood why they were so many lines... [1]: https://images-na.ssl-images-amazon.com/images/I/815WQQdAHBL...

It helps children learning how to write. Lowercase letters start from the thicker line to the first thinner line above. Uppercase letters and taller lowercase letters like "t" or "d" go to the second line. And the tails of letters like "g" or "y" go the first line below.

Re: Compressing and enhancing hand-written notes (2016)

#65

Post by [deleted] » Mon, Mar 12, 2018, 9:51 PM UTC

derefr wrote: post #59

dtech wrote: post #19

Earlier quoted context omitted.

> My theory is that scanner manufacturers are deliberately choosing the JPG encoding profile that gets them the fastest result. This is more-or-less correct. The chips in the printers have a lot less power than your CPU, and the algorithms are a lot worse than those in Photoshop.

I'm surprised there aren't any high-quality JPEG encoder ASICs. Would scanners benefit from just using some of the plentiful, cheap, excellent-quality H.264 encoder ASICs—and treating the output as HEIF?

[deleted]

Re: Compressing and enhancing hand-written notes (2016)

#66

Post by striking » Mon, Mar 12, 2018, 11:25 PM UTC

goerz wrote: post #51

ramses0 wrote: post #49

Earlier quoted context omitted.

`German and Swiss regulators have subsequently (in 2015) disallowed the JBIG2 encoding in archival documents.[19]`

Yeah, but I'm not archiving for the German or Swiss government. For scanned handwritten notes, JBIG2 still beats out anything else by an order of magnitude at least

JBIG2 has been known to swap 6s and 8s. Use it at your own risk. There's a reason nothing uses it anymore.

Re: Compressing and enhancing hand-written notes (2016)

#67

Post by goerz » Mon, Mar 12, 2018, 11:54 PM UTC

striking wrote: post #66

goerz wrote: post #51

Earlier quoted context omitted.

Yeah, but I'm not archiving for the German or Swiss government. For scanned handwritten notes, JBIG2 still beats out anything else by an order of magnitude at least

JBIG2 has been known to swap 6s and 8s. Use it at your own risk. There's a reason nothing uses it anymore.

Is there an alternative that provides comparable compression?

Re: Compressing and enhancing hand-written notes (2016)

#68

Post by Scaevolus » Tue, Mar 13, 2018, 12:18 AM UTC

sp332 wrote: post #26

alister wrote: post #17

I have an observation about scanning documents that results in good quality and smaller files, but I can't satisfactorily explain why it works. Consider these two cases: (1) Scan document at very high resolution as a JPG and then use a third-party program (like Photoshop or whatever) to re-encode the JPG at your preferred low resolution. (2) Scan document at your preferred low resolution as a JPG straight away. Don't…

If you're scanning at a lower resolution, the scanner has fewer samples to work with when trying to make a visual representation of your document. If you scan at a higher resolution, the algorithm could at the very least average together nearby samples. It could also detect sharp lines vs fuzzy borders and decide whether the low-res version has a sharp transition or an averaged color between areas.

Low-resolution scans are faster than high-resolution scans for this exact reason: the sampling is worse. There might be the same number of samples per line captured by the linear CCD (and then downsampled), but the distance between scanlines varies by DPI.

Re: Compressing and enhancing hand-written notes (2016)

#69

Post by herpaderp_33 » Tue, Mar 13, 2018, 1:11 AM UTC

It would be interesting to see this paired with potrace.

Re: Compressing and enhancing hand-written notes (2016)

#70

Post by Myrmornis » Tue, Mar 13, 2018, 2:07 AM UTC

This is awesome, and a depressingly large factor better than any blog post I’ll ever write.

I totally identify with the need for this. I also want to archive images of notes and whiteboards, and they must be kept small as so far my life fits in google drive and github.

Currently I use Evernote to do this. I don’t use any other functionality in Evernote but the “take photo” action does processing and size reduction very like the blog post.