Over on http://golf.horse/ there are leaderboards for finding the smallest Javascript programs that output various word lists, including the Wordle list. I've found it to be a fun and educational challenge. I would be excited to see more submissions!
Game Boy Wordle clone: How to compress 12972 five-letter words to 17871 bytes
21–30 of 170 posts
Re: Game Boy Wordle clone: How to compress 12972 five-letter words to 17871 bytes
#22I wonder why the first letter is used on its own. Getting the first letter in 5 bits + 3 bits out of the second should make the first array much more densely populated. Maybe making the further lists more sparse offsets the gains?
Re: Game Boy Wordle clone: How to compress 12972 five-letter words to 17871 bytes
#23Bug? See screenshot. My 5th guess shouldn’t have gone down like that for a conventional Wordle clone. https://ibb.co/93CdVWv
Black is right letter right spot. Green is right letter wrong spot.
Re: Game Boy Wordle clone: How to compress 12972 five-letter words to 17871 bytes
#24And far better than gzip compression. Nice work. My takeaway is that context matters - this is not General Purpose Compression, but compression made specifically for this case. Good Stuff.
To improve compression of a sorted list of words you can replace the (initial) letters repeated from the word above with spaces before compression and add them back as an extra step after decompression. For example, if the previous word was "apple", the next entry will be " y" ("apply", edit: HN removes extra spaces, so this should be four spaces + "y") ("apple" will probably already be entered as " le" (three spaces…
You might want to look at locatedb and frcode - https://manpages.ubuntu.com/manpages/bionic/man5/locatedb.5....
If you want to skim, check out the EXAMPLE section toward the bottom.
Re: Game Boy Wordle clone: How to compress 12972 five-letter words to 17871 bytes
#25And far better than gzip compression. Nice work. My takeaway is that context matters - this is not General Purpose Compression, but compression made specifically for this case. Good Stuff.
Is task specific compression a thing in real life practical software engineering? As far as reducing data loads go I only came across the keyword "SQL optimization".
Re: Game Boy Wordle clone: How to compress 12972 five-letter words to 17871 bytes
#26 $ grep '^[a-z]\{5\}$' /usr/share/dict/words | python -c '
> from sys import stdin
> from os import write
> N = 26 ** 5
> data = bytearray((N+7)//8)
> for l in stdin:
> b = 0
> for c in l.strip():
> b *= 26
> b += ord(c)-ord("a")
> data[b//8] |= 1 write(1, data)
> ' | gzip | wc -c
12126
Re: Game Boy Wordle clone: How to compress 12972 five-letter words to 17871 bytes
#27Over on http://golf.horse/ there are leaderboards for finding the smallest Javascript programs that output various word lists, including the Wordle list. I've found it to be a fun and educational challenge. I would be excited to see more submissions!
Hasegawa Sayuri wrote up some notes about their submissions at http://sayuri.tx0.org/golfhorse/, including an extremely elegant and compact encoding of huffman trees.
Re: Game Boy Wordle clone: How to compress 12972 five-letter words to 17871 bytes
#28And far better than gzip compression. Nice work. My takeaway is that context matters - this is not General Purpose Compression, but compression made specifically for this case. Good Stuff.
Is task specific compression a thing in real life practical software engineering? As far as reducing data loads go I only came across the keyword "SQL optimization".
One example that comes to mind is some work nvidia has done compressing “video” streams. They do so by capturing your facial movements and reconstructing them on the other side, resulting and massively less bandwidth.
Re: Game Boy Wordle clone: How to compress 12972 five-letter words to 17871 bytes
#29And far better than gzip compression. Nice work. My takeaway is that context matters - this is not General Purpose Compression, but compression made specifically for this case. Good Stuff.
Is task specific compression a thing in real life practical software engineering? As far as reducing data loads go I only came across the keyword "SQL optimization".
Depends how specific you want "specific" to be I guess? I.e. we have loads of compression algorithms for different kinds of data. Whereas here we are looking at almost dataset-specific compression (i.e. the only benchmark is how it works for one specific set of data to compress), and there's a sliding scale between the two ends.
Similarly, having to contain the decompression code in the measured result size and it being a relevant contribution is something that only applies in some use cases of compression.
Re: Game Boy Wordle clone: How to compress 12972 five-letter words to 17871 bytes
#30And far better than gzip compression. Nice work. My takeaway is that context matters - this is not General Purpose Compression, but compression made specifically for this case. Good Stuff.
Is task specific compression a thing in real life practical software engineering? As far as reducing data loads go I only came across the keyword "SQL optimization".
Scarcity breeds innovation. Why hand tune a compression scheme for a specific dataset when it is less storage efficient than LZ? You opt to not use LZ when all the memory and compute you have can barely run prefix codes.
That's why people still write for the Z80: it's a fun toy.