Live data from Hacker News

Should you normalize RGB values by 255 or 256?

30fps.net

141–150 of 156 posts

Re: Should you normalize RGB values by 255 or 256?

#141
As an argument for 255, take the extreme case of a black and white image, single bit, 0 is black and 1 is white.

It seems pretty obvious that you want 0 to map to 0.0 and 1 to 1.0. It is black and white, not light gray (0.25) and dark gray (0.75). It means that you normalize a black and white image by 1, not 2.

For 2 bits, you would normally have 0 = black, 1 = light grey, 2 = dark grey, 3 = black. So it is natural to map it to 0.0, 0.33, 0.66, 1.0, again, you want your black to be black and your white to be white, you also want equal spacing, so normalize by 3.

Follow the logic to 8 bits and you will normalize by 255, because you want black to be 0.0 and white to be 1.0, even though the difference starts to get really small with 8 bits.

Another way to see things is that with the alterative normalization (256 for 8 bits), the output range depends on the number of bits: [0.25, 0.75] for 1 bit, [0.125, 0.875] for 2 bits, etc... You typically don't want that, more bits should mean more nuances, but the contrast should stay the same.

Re: Should you normalize RGB values by 255 or 256?

#142
post #81

Earlier quoted context omitted.

Sorry, you seem to be confused. >There are only 255 bins, not 256 There are 256 bins because there are 256 values. The questions are: 1. What are the boundaries of these bins? 2. Which sample represents a particular bin? With 1-bit color, we have sample values {0, 1}. What bins do they represent? Here's one choice: [0, 1), [1, 2) Two equally sized bins, spanning the interval [0, 2] of length 2, each defined by its sa…

Thank you for the thoughtful reply. Maybe bins is the wrong word to use, so I'll try with intervals. Starting with 1 bit data, there are two numbers and one interval. I think where bins makes it confusing is that inside the interval there are two big rounding errors mapping everything to either 0 or 1 and many people seem to be considering those the bins. Taking a step back, remember we're ultimately mapping these di…

>Maybe bins is the wrong word to use, so I'll try with intervals

Same thing, in both how you use it and how the author does.

>Taking a step back, remember we're ultimately mapping these discrete numbers to some real world continuous variable

I think this is where you have a misconception.

There are two maps.

The important one goes the other way: FROM a continuous variable TO a finite set.

It's not 1-to-1: it maps entire ranges of numbers (intervals, bins, whatever) to discrete values (samples, integers, whatever).

The bins are preimages of that map.

The discussion in the article comes from two ways of defining that map: FROM continuous signal TO discrete variable.

The map that goes the other way, from the integers into floats, has to be CONSISTENT with it.

The article presents this backwards, putting the cart before the horse. This causes confusion.

>All we know is that those are the numbers we have available to represent the real values we're measuring.

Each of those numbers doesn't represent any one value; it represents a range.

Think about it this way: if we have a continuous signal that we're discretizing into a finite number of bits, we're invariably smashing ranges into single values (what you call "rounding error").

When we're reading this data — say, we read number 5 — we don't know which continuous variable value it came from.

To display it on a screen, we make a choice; we pick some number from the interval it came from, and call it a day.

>The important part is that 0 represents the minimum and 1,3, and 7 all represent the same maximum real value

The important part is that this is a choice you make about what those point samples represent.

It's a convenient choice. Which is why we all use it.

Some people prefer a different choice, that's all.

> If you normalize it by 4, you get [0, 1/4, 2/4, 3/4]

That's one way to do it, and not the way the article uses (re-read my previous comment, it has both).

Still, I'm with you here.

>and you're effectively throwing away some of the range of the ADC.

The map you're describing (FROM discrete INTO continuous) is approximating the DAC.

So, yes, with this scheme you're never getting 0.0 and 1.0.

Think of it this way. Say, you convert an image to a 1-bit representation, and render it on a screen in grayscale.

One choice is to render 0 as 0.0 and 1 as 1.0 (black and white).

Another is to render 0 as 0.25 and 1 as .75 (dark grey and light grey).

That's the "alternative" (divide by 2^n) approach. The formula here is x→ (x + 0.5)/2^n.

Neither is inherently wrong or better than the other; especially when you ask which rendering is closer to the original image.

Plus: one man's "you're not using the entire range of DAC" is another's "you leave a tiny bit of headroom".

In any case, you're not losing data in either [ discrete → continuous → discrete ] chain because you get the discrete values back perfectly.

What you divide by in the first step is dictated by what you do in the second.

>If you normalize 2 bit data by 3 you get [0, 1/3, 2/3, 1].

Let's see what this says about how we should go in the other direction to be consistent with this scheme.

Which continuous values get sent to 0 and 3? Which get sent to 1 and 2?

You wrote : {0, 1, 2, 3} → [0, 1/3, 2/3, 1]

So you can see that going in the other direction (discretizing):

    0 ← [0   ... 1/6) 
    1 ← [1/6 ... 5/6)
    2 ← [3/6 ... 1/6) 
    3 ← [5/6 ... 1  ]
Some people don't like that 0 and 3 get smaller ranges than the rest.

>So the answer is, if you have N bit data, you normalize by 2^N-1.

The answer is: it doesn't matter in practice, so use what's simpler in your context.

That's going to be dividing by 2^N - 1 for pretty much everyone.

Re: Should you normalize RGB values by 255 or 256?

#143
post #56

Earlier quoted context omitted.

That's only valid to do if the reciprocal is representable exactly.

That's not totally true. It's sufficient to be exactly representable, but you only need the reciprocal rounding error to be small enough to guarantee the multiplication rounding step fixes it across the entire range of numerators. For IEEE754 f16 values, there are 28 such extra values, the positive and negative sides of 1705/x where x is a power of 2 at least as great as 2048.

Interesting, but pretty limited corner case. Would compilers even identify those 28 values and do the transformation?

Re: Should you normalize RGB values by 255 or 256?

#144
post #141

As an argument for 255, take the extreme case of a black and white image, single bit, 0 is black and 1 is white. It seems pretty obvious that you want 0 to map to 0.0 and 1 to 1.0. It is black and white, not light gray (0.25) and dark gray (0.75). It means that you normalize a black and white image by 1, not 2. For 2 bits, you would normally have 0 = black, 1 = light grey, 2 = dark grey, 3 = black. So it is natural t…

I appreciate that the author concluded you should use 255, because this approach here (making sure the minimum and maximum of the range actually map to 0 and 1) makes so much sense I never considered you could or ever would want to do it any other way.

Re: Should you normalize RGB values by 255 or 256?

#145

This was a genuinely thought provoking article. I had to challenge some personal assumptions. Coming from an electrical engineering background, I disagree with how the author presented "Two types of quantizers". Mathematically rigorous, but not grounded in practical systems. In ADCs, there is always an inherent +-1/2 LSB of quantisation uncertainty. The transfer characteristic is always mid-tread sampling, or at leas…

All ADCs I have looked at document that they can't represent the positive full scale. For instance, for an 8 bit ±1 V ADC, -128 represents -1 V, +127 represents 127/128=0.99219 V. The transition from 126 to 127 happens at 1.5 LSB from the positive full range. 1 LSB difference represents 1/128 = 0.00781 V difference, and not 2 / 255 = 0.00784 V. But if you actually care about what the voltage (and uncertainty) is, mos…

You've made me do even more digging and now I'm even more confused.

ATMega328P, data sheet specifically calls out using that it cannot represent full range, ADC = VIN * 1024 / VREF. The STM32F4 datasheet shows an idealised transfer function which is mid-tread, but the "actual transfer function" is sort of a shifted mid-tread (lowest code is larger than a single LSB, highest code is VREF and 1/4 LSB). Other STM32 references I found show that it should follow an ideal mid-tread, unless you are using VADC as your voltage reference. High resolution differential input ADCs are, as best as I can tell, always mid-tread with both end codes representing the positive and negative voltage reference.

The best data sheet I've found was for a PIC32, very detailed transfer characteristic diagram. This shows that each LSB is VREF/1024, but also why! The transfer function shows that is is still a mid-tread transfer, but the highest code is assigned to (1023/1024) * VREF, hence the divide by 1024, but it doesn't solve the 1/2LSB end bin problem.

This is different to the ideas presented the article, there is no reason a single ended ADC couldn't use mid-tread quantisation with VREF as the end code. What it does show is that by using divide by 256, you are truncating the range and arbitrarily deciding where the end points should be. This doesn't fix any inherent 1/2LSB quantisation uncertainty, which was the main argument against using divide by 255.

As you say, in ADCs it's a moot point with all of the other sources of error.

Re: Should you normalize RGB values by 255 or 256?

#146
post #133

Earlier quoted context omitted.

The 0.5 comes from debiasing the round-trip. If your conversion from high precision -> 8-bit is just multiplication by 256 and then truncation, then you’ve got the mid-riser quantizer. The +0.5 comes from interpreting a value of 0 as bucket from 0-1, just like the value of 255 is the bucket from 255-256. It’s introduced in the conversion back from 8-bit to high precision.

You’re still misinterpreting what I proposed. I didn’t propose a +0.5 debias at all. I proposed that the bias is removed by scaling. In this case you divide by 255.0 but multiply by 256. But again, the likely reason no one does this is because it introduces a bias in the other direction, toward 255.

It’s unbiased, but amplifies the signal.

Re: Should you normalize RGB values by 255 or 256?

#148

Earlier quoted context omitted.

Thank you for the thoughtful reply. Maybe bins is the wrong word to use, so I'll try with intervals. Starting with 1 bit data, there are two numbers and one interval. I think where bins makes it confusing is that inside the interval there are two big rounding errors mapping everything to either 0 or 1 and many people seem to be considering those the bins. Taking a step back, remember we're ultimately mapping these di…

>Maybe bins is the wrong word to use, so I'll try with intervals Same thing, in both how you use it and how the author does. >Taking a step back, remember we're ultimately mapping these discrete numbers to some real world continuous variable I think this is where you have a misconception. There are two maps. The important one goes the other way : FROM a continuous variable TO a finite set. It's not 1-to-1: it maps en…

* Correction: fixing typo

    0 ← [0   ... 1/6)     
    1 ← [1/6 ... 3/6)
    2 ← [3/6 ... 5/6) 
    3 ← [5/6 ... 1  ]

Re: Should you normalize RGB values by 255 or 256?

#149

This is like a 1D version of what a scientific computing person might describe as the distinction between node-centered ("standard" or "mid-tread" in this post) vs cell-centered ("alternative", "mid-riser") samples: do consider values to be at the middles of bins (or middles of triangles, or middle of tetrahedra), or the boundaries between intervals (or vertices of triangles, or of tets). In a scientific computing se…

In 8-bit (per channel) color there is a standard for when the context is unspecified: sRGB. It was developed to match the characteristics of a "typical" display at the time of its creation. It won't be 100% accurate, but it represents the best you can assume. You're not forced to throw up your hands and give up.

Re: Should you normalize RGB values by 255 or 256?

#150
post #56

Earlier quoted context omitted.

That's not totally true. It's sufficient to be exactly representable, but you only need the reciprocal rounding error to be small enough to guarantee the multiplication rounding step fixes it across the entire range of numerators. For IEEE754 f16 values, there are 28 such extra values, the positive and negative sides of 1705/x where x is a power of 2 at least as great as 2048.

Interesting, but pretty limited corner case. Would compilers even identify those 28 values and do the transformation?

Maybe for f16. The compiler's implementation could just be checking all numerators to see if the transformation is safe. The corner cases are messy and not quickly brute-forceable for f32 or brute-forceable at all for f64 though, so I doubt they'd bother, especially when I bet those constants have showed up literally zero times across all programs.
Post reply on HN