Live data from Hacker News

Show HN: How to store a set of four 5-bit values in one 16-bit value

github.com

101–110 of 149 posts

Re: Show HN: How to store a set of four 5-bit values in one 16-bit value

#101
post #80

Conversely: the order for 5 4-bit values itself has 4-bits of information.

The order for five four bit values has about 6.91 bits of information. log_2(5!)=6.91

More relevant to the problem, the order for four five bit values has about 4.59 bits of information (log_2(4!)=4.59). Not all of the sets actually have four numbers- many have duplicate numbers, like the list [1 1 2 3] is a set of numbers {1 2 3}- and that gets us down to 4.08 bits of information which is small enough to make this example work.

At least that's how I understand this. I could be wrong.

Re: Show HN: How to store a set of four 5-bit values in one 16-bit value

#102
I haven't learned about compression of data, but from what I know, one has to make some assumptions to store n bit data into m bit space, where m What I fail to understand is how this helps to compress the data in this case. The code isn't really that nice, and the comments seem to assume prior knowledge is this field, which I don't have.

So can someone give a ELI-Noob?

Re: Show HN: How to store a set of four 5-bit values in one 16-bit value

#103
post #44

how are you getting 3876 unique values for a set of 4 4 bit values (16, 16, 16, 16) ?

He doesn't care about the order so 1, 2, 3, 4 is stored the same as 4, 3, 2, 1. That's where the savings is coming from.

I really don’t understand why not caring about the order would save space. Could you explain in simple terms please?

Re: Show HN: How to store a set of four 5-bit values in one 16-bit value

#104
post #69

Earlier quoted context omitted.

Arithmetic coding will work too, without any special trick to code duplicates.

Can you elaborate? I am not very familiar with arithmetic coding, but I thought it required arbitrary precision floating point numbers, so not O(1) memory.

You don't need arbitrary precision, just enough precision. Specifically, in this case you only need 16 bits. You can start with an upper and lower bound, and narrow it down for each number.

Here's an example with numbers dumped from a quick prototype encoder. The exact range numbers aren't that important, but note how the range gets smaller for every symbol we encode:

Say we want to encode the ordered numbers 10, 15, 31, 31.

We start with the full range [0, 65536)

There are C(32 - 10 + 2, 3) = 2024 combinations that start with 10, out of 52360 possible, so that narrows it down to around 4% of the full range, we'll use [49702, 52236)

For the next number we only allow numbers 10 and above, 15 has 153 out of 2024 possibilities, range [51022, 51214)

For the next number, 31 is only 1 of the 153 possibilities, so it gets a tiny range, [51212, 51214).

And now 31 is the next number in 1 out of 1 of cases, so the range is again [51212, 51214)

We can code the sequence as either 51212 or 51213, since we used a range that is slightly larger than needed some combinations have multiple codes. We could have started with the smaller range [0, 52360) instead to get a bijection.

Re: Show HN: How to store a set of four 5-bit values in one 16-bit value

#105
post #43
post #14

Earlier quoted context omitted.

Usually, when you store a value, you want to be able to get exactly that value back. If you store multiple values without keeping the ordering, you lose that capability. I.e. you can't replace all possible uses of four 5-bit variables with one 16-bit variable, only those where the variables are interchangeable.

Many common data structures fail to preserve order. Maps (aka dictionaries) based on hash tables come to mind. Slightly adrift of the topic... I have a HAMT (hash array map trie) implementation that uses a 32-bit unsigned int as a bitfield to indicate which of the 32 possible children nodes are populated. With this trick I could encode any node with 5 or fewer bits flagged with a 16-bit unsigned int instead. I just c…

The number of bags of k N-bit numbers is 2^N + k - 1 choose k:

https://en.wikipedia.org/wiki/Multiset#Counting_multisets

The number of bits required to uniquely represent such a bag is thus ⌈log2((2^N+k-1) choose k)⌉. For (N, k) we have:

(6, 6) -> ⌈log2(69 choose 6)⌉ = ⌈log2(119877472)⌉ = 27 bits (6, 7) -> ⌈log2(70 choose 7)⌉ = ⌈log2(1198774720)⌉ = 31 bits

So you can store seven 6-bit integers without order in one int32. I don't know if it'll be slow, though. It should be possible to infer an encoding algorithm from the proof of the counting formula but it might be slow.

Re: Show HN: How to store a set of four 5-bit values in one 16-bit value

#107

I haven't learned about compression of data, but from what I know, one has to make some assumptions to store n bit data into m bit space, where m What I fail to understand is how this helps to compress the data in this case. The code isn't really that nice, and the comments seem to assume prior knowledge is this field, which I don't have. So can someone give a ELI-Noob?

It’s not perfect compression; it’s lossy. Per the top comment, this method treats it as a set i.e. ignore the order. So you’re throwing out some information.

https://news.ycombinator.com/item?id=16249092

Re: Show HN: How to store a set of four 5-bit values in one 16-bit value

#108
post #2

Ah, without preserving order. That's a set of four 5-bit values, not a sequence of four 5-bit values.

Yup so no ARGB5555 16 bit type :-)

But an MSAA 4-fragment each 5-bit 16-bit type.

Or MSAA4x ARGB5555 in 64-bit (instead of 80 bit)

Re: Show HN: How to store a set of four 5-bit values in one 16-bit value

#109

Earlier quoted context omitted.

He doesn't care about the order so 1, 2, 3, 4 is stored the same as 4, 3, 2, 1. That's where the savings is coming from.

I really don’t understand why not caring about the order would save space. Could you explain in simple terms please?

Consider the simplest case where your set members are just bits.

If you have three bits and you care about the order, that's a 3-bit number -- it can take 8 different values.

If you have those same bits but you don't care about the order, the operation is a "bit count" -- you can only have 4 different values.

Re: Show HN: How to store a set of four 5-bit values in one 16-bit value

#110

Taking this to the extreme, you could "store" 65535 1-bit values in one 16-bit value. You just have to keep track of how many of them are equal to 1.

Thank you - this is a much better explanation of how this 'storage' works.
Post reply on HN