Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Xerox responds to the recent character substitution issue

realbusinessatxerox.blogs.xerox.com

Post reply on HN
Print view View original

41–50 of 72 posts

Re: Xerox responds to the recent character substitution issue

#41

Post by ToothlessJake »

Groxx wrote: post #27
ToothlessJake wrote: post #14

Earlier quoted context omitted.

O-SubC-R then perhaps. Still is recognizing shapes/symbols which is the very basis of OCR. This seems a bit hair splitty when the end result is the same as invalid OCR dictionaries.

Well sure, but then why don't we just call it "lossy GZIP"? OCR is a pretty specific subset, and produces characters - this does not produce computer-readable characters, therefore not OCR.

What are you on about? What does it produce if not computer-readable characters? Computer illegible characters? Are you saying it cannot read from the dictionary it creates? Or from the characters it is later optically recognizing off that dictionary?

Again from the JBIG2 wiki[1]:

"Textual regions are compressed as follows: the foreground pixels in the regions are grouped into symbols. A dictionary of symbols is then created and encoded.."

It seems not only is JBIG2 being deployed as OCR by Xerox for whatever reason, its implementation in this case is an absolute failure.

[1] http://en.wikipedia.org/wiki/JBIG2

Re: Xerox responds to the recent character substitution issue

#43

Post by Groxx »

ToothlessJake wrote: post #41
Groxx wrote: post #27

Earlier quoted context omitted.

Well sure, but then why don't we just call it "lossy GZIP"? OCR is a pretty specific subset, and produces characters - this does not produce computer-readable characters, therefore not OCR.

What are you on about? What does it produce if not computer-readable characters? Computer illegible characters? Are you saying it cannot read from the dictionary it creates? Or from the characters it is later optically recognizing off that dictionary? Again from the JBIG2 wiki[1]: "Textual regions are compressed as follows: the foreground pixels in the regions are grouped into symbols. A dictionary of symbols is then…

Does it produce ASCII? UTF? If no, it's not OCR.

edit: by the definition you seem to be going on, any facial recognition is also OCR, since you could consider a face a 'glyph' (edit: 'symbol'). The only 'text' thing here that I can see is that it is intended to be used on text, which lends some optimizations, nothing that it's actually text-based in any way.

Re: Xerox responds to the recent character substitution issue

#44

Post by ChuckMcM »

That is an astonishing response. Reminds me a bit of the first time EMC pointed out that while it was possible to have your data corrupted in their hash based storage system, it probably would never happen.

I was expecting "Here is new firmware and we apologize for using JBIG2, won't happen again."

One wonders if JBIG2 is used in the storing of checks by banks (my bank these days only sends me images of my checks, never the actual check any more) or DMV records, or any number of things.

So in the previous thread I suggested a JBIG2 test image, now I want to build one that if you copy it, it goes from one thing to something else entirely!

Re: Xerox responds to the recent character substitution issue

#45

Post by bestham »

ToothlessJake wrote: post #41
Groxx wrote: post #27

Earlier quoted context omitted.

Well sure, but then why don't we just call it "lossy GZIP"? OCR is a pretty specific subset, and produces characters - this does not produce computer-readable characters, therefore not OCR.

What are you on about? What does it produce if not computer-readable characters? Computer illegible characters? Are you saying it cannot read from the dictionary it creates? Or from the characters it is later optically recognizing off that dictionary? Again from the JBIG2 wiki[1]: "Textual regions are compressed as follows: the foreground pixels in the regions are grouped into symbols. A dictionary of symbols is then…

OCR per definition gives out text. Not binary data that resemble the bitmap of the input image.

Re: Xerox responds to the recent character substitution issue

#46

Post by nsxwolf »

When you read a scanned or copied document, your confidence in the information is based on its quality.

There comes a point when the quality is so poor that you no longer trust your interpretation. Is that a 3? An 8? If you can't tell, you will not act on that information without further clarification.

This compression algorithm destroys this process.

How can you trust what you are reading anymore? How do we know there isn't a bug that sometimes causes the content substitution when the source text is large and perfectly legible?

Disk space is not at enough of a premium to justify this.

Re: Xerox responds to the recent character substitution issue

#47

Post by eCa »

ToothlessJake wrote: post #41
Groxx wrote: post #27

Earlier quoted context omitted.

Well sure, but then why don't we just call it "lossy GZIP"? OCR is a pretty specific subset, and produces characters - this does not produce computer-readable characters, therefore not OCR.

What are you on about? What does it produce if not computer-readable characters? Computer illegible characters? Are you saying it cannot read from the dictionary it creates? Or from the characters it is later optically recognizing off that dictionary? Again from the JBIG2 wiki[1]: "Textual regions are compressed as follows: the foreground pixels in the regions are grouped into symbols. A dictionary of symbols is then…

It produces symbols, not characters.

Say that the scanner internally splits the scan into regions of 10x10 pixels that it saves in memory. If another region differs on less than (say) 10% of the pixels it is assumed that the two zones are identical and the first one is used in the second place too. The regions have no semantic meaning.

OCR translates the scan into a character set.

Re: Xerox responds to the recent character substitution issue

#48

Post by 205guy »

This is an interesting story with lots of odd comments.

First of foremost, I agree that Xerox putting their name on a product which creates an unfaithful copy is corporate suicide. Such an ancient paragon of computer innovation should be able to come up with a clever algorithm that compresses but doesn't substitute image bits.

But...

- The original story[1] didn't mention that the product itself warns against the very thing they are reporting. Did they ignore that warning, did the copier not show it, did they use a setting that did not have the warning? Their further posts cover the issue, so it looks like somebody else set the resolution and ignored the warning.

- Calling what the JBIG2 algorithm does "OCR" is misleading. OCR is pretty much understood to be analog text (image) to digital text (ASCII, UTF-32). Matching to a real character set and outputting those characters is a defining part of true OCR. It's also confusing because the copiers have a true OCR function, and this is not related. What JBIG2 does, I would call it "sub-image matching and substitution."

- Calling JBIG2 "lossy" is also misleading. I suppose it is lossy by definition, but lossy is usually limited to pixel effects as seen in JPG, no image blocks.

- JBIG2 seems like an algorithm that shouldn't be used on low-res text documents. You might say it's just a configuration of the algorithm, but if engineers can't take it as a tool and use it correctly, you start to wonder if it's a problem with the tool.

[1] http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...?

Re: Xerox responds to the recent character substitution issue

#49

Post by smtddr »

gpvos wrote: post #21
xahrepap wrote: post #11

Earlier quoted context omitted.

> We do not normally see a character substitution issue with the factory default settings however, the defect may be seen at lower quality and resolution settings. I might have read it wrong, but from how I understood it the default settings don't have this problem. It's when people adjust the quality settings to be lower. Am I wrong?

You are correct. The default setting is "high" or "higher"; I don't know which. The setting that may copy blocks of characters around is the lowest setting and is called "normal", and comes with some small print on the screen that actually warns you for the character substitution.

Oh wow, why didn't anyone mention this before? Or have I been missing it? I'm not being sarcastic, the fact that the warning about char-substitution is displayed to the user like that changes this whole story. I still think it a bad idea to even have that setting at all and Xerox should just remove it from future devices - but the user was warned, in as much as the average user ever reads warnings on computer screens......

Re: Xerox responds to the recent character substitution issue

#50

Post by morsch »

I was curious to see how JBIG2 fares compared to JPEG, and found a benchmark from 2010 [1] comparing the file size of the resulting PDF:

  convert *.jpg JPEG.pdf -- 43777 kb
  convert *.png PNG.pdf -- 6907 kb
  jbig2 -b J -d -p -s *.jpg; pdf.py J > JBIG2.pdf -- 947 kb
  jbig2 -b J -d -p -s -2 *.jpg; pdf.py J > 2xJBIG2.pdf -- 1451 kb
Quite a difference. I don't quite understand how JPEG fares so poorly compared to (lossless) PNG, maybe because it doesn't do monochrome?

[1] http://ssdigit.nothingisreal.com/2010/03/pdfs-jpeg-vs-png-vs...

Post reply on HN