Huffman trees are really cool and a neat little example program to put together.
Agreed, I had a lot of fun putting a toy one together, even if my compression format was worse in every way than a real format and my implementation quite inefficient
A student’s desire to get out of a exam led to a compression algorithm
31–40 of 132 posts
Re: A student’s desire to get out of a exam led to a compression algorithm
#32On a related note, I got an email today saying that Cloudflare will automatically turn on Brotli compression later this month. Does anyone know if Brotli really makes a difference beyond gzip?
Brotli and ZSTD are both a bunch better (~20% file size while being faster to decompress). Brotli does especially well on javascript/html because they hard-code some of the keywords in the spec.
Re: A student’s desire to get out of a exam led to a compression algorithm
#33> to the ubiquitous software utility PKZip
Huh? Ubiquitous?
Re: A student’s desire to get out of a exam led to a compression algorithm
#34missed opportunity to explain how compression and prediction are related, and that the better you can predict the next token the better your compression gets, then your article gets to mention GPT hey
That being said: the best results currently on the large text compression benchmark utilize ML: http://www.mattmahoney.net/dc/text.html
Re: A student’s desire to get out of a exam led to a compression algorithm
#35Re: A student’s desire to get out of a exam led to a compression algorithm
#36Earlier quoted context omitted.
In today's post-GPT world it's important to recognize the role of embodiment in intelligence as well.
Why?
The modality of the data contains an amount of information comparable to the data itself. Telling ChatGPT that it's hearing music rather than a story would probably help it reason about it a lot.
On a lower level, you can't tell ChatGPT to reason about an image when its only input is a microphone.
Re: A student’s desire to get out of a exam led to a compression algorithm
#37Caches drive the internet. They are orders of magnitude more important than compression. 99.99% of requests hit a local cache. (1) Compression is important too. (1) I worked in a telco. Check it out for yourself!
Ok how do they work then?
They store frequently-requested items in a place that permits faster retrieval.
Re: A student’s desire to get out of a exam led to a compression algorithm
#38Please pardon my ignorance... My understanding is that there are 1000s of different compression algorithms, each with their own pros/cons dependent on the type and characteristics of the file. And yet we still try to pick the "generically best" codec for a given file (ex. PNG) and then use that everywhere. Why don't we have context-dependent compression instead? I'm imagining a system that scans objects before compre…
Most image codecs are made for natural images; those of the real world, not synthetic ones like the one you proposed. Lossy codecs like JPEG use perceptual coding to maximize perceived quality for a given file size.
Re: A student’s desire to get out of a exam led to a compression algorithm
#39Caches drive the internet. They are orders of magnitude more important than compression. 99.99% of requests hit a local cache. (1) Compression is important too. (1) I worked in a telco. Check it out for yourself!
Yeah it's cache all the way down. CPU has L1, L2, L3 for instructions cache. Server has ramcache, then SSD cache. Then apps with SQL almost always use redis/memcache as first level cache. Then for network side we have CDN which is basically a glorified webcache, even at router/switch there is cache for all the stuffs.
Re: A student’s desire to get out of a exam led to a compression algorithm
#40Caches drive the internet. They are orders of magnitude more important than compression. 99.99% of requests hit a local cache. (1) Compression is important too. (1) I worked in a telco. Check it out for yourself!
I don't think your caches would help very much if all media were sent uncompressed. Everything is already compressed, so you might not think about it much, but I'd say that it is at least as important as caching.