51–60 of 83 posts
Lo and behold, a nice arithmetic coding implementation that wasn't written by an LLM! A sight for sore eyes – a treat, even. Looks like it was written by someone else though. Check it out: https://github.com/samyak112/pym-particles/blob/main/arithme...
Its not an issue is it? I am not sure.
Somewhat related is stavros's method to compress 500KB to something like 50 bytes https://www.stavros.io/posts/compressing-images-with-stable-... main drawback is that it's not lossless ;-) but this is great. I hope this actually becomes a format that wraps the weights and transformer module (maybe this can also be NAS-optimized too?). Maybe it would even work for video? It's like calling gzip but instead of compress…
Just amazing, wow
What do those compress to with conventional approaches? For comparison. I am curious. A classic machine learning ensemble approach is to overfit a collection of small models then bag them (e.g. voting) allowing the models to generalize. I'm sure someone's tried to overfit a bunch of transformers for compression like this, then bag them to see how well it does?
Ensembling is not compute or parameter-efficient, so compression per se is a terrible application. (This is related to why people train ever larger LLMs like 1 10t-parameter LLM, rather than 100 GPT-3-scale LLMs.)
Somewhat related is stavros's method to compress 500KB to something like 50 bytes https://www.stavros.io/posts/compressing-images-with-stable-... main drawback is that it's not lossless ;-) but this is great. I hope this actually becomes a format that wraps the weights and transformer module (maybe this can also be NAS-optimized too?). Maybe it would even work for video? It's like calling gzip but instead of compress…
While clearly satirical, it's definitely quite thought-provoking from various angles including the basis of information, representation of data, and even copyright. It's like watching a movie, writing a book based on it, and then making another movie based on that book.
Earlier quoted context omitted.
Maybe everyone should compress the 1st 100MB worth of digits of pi, for an apples-to-apples comparison? Edit: oh wait that's too easy. Need to generate /publish random digits so everyone can use it.
random digits aren't compressible though?
Fabrice Bellard may have been the first to do this, 7 years ago: https://news.ycombinator.com/item?id=27244004
Yeah yeah, I just found the idea kinda interesting so wanted to implement it
Earlier quoted context omitted.
Random digits are compressible though. Random data does not mean it does not match a pattern in your dictionary for example.
No.. they're not. Do you understand random (the apparent or actual lack of definite patterns or predictability[0]) or compression (reduces bits by identifying and eliminating statistical redundancy[1])? [0]: https://en.wikipedia.org/wiki/Randomness [1]: https://en.wikipedia.org/wiki/Data_compression
Three questions: 1. How much was AI used to generate documentation for this project? 2. The 100MB CSV data sources are not provided in the repo so it doesn't seem possible to reproduce your results. The enwik9 dataset says it is a "slice" of the larger data set, and there are many NYC taxi trip record datasets that exist. Can you provide the datasets used to generate your results? 3. I am surprised to see performance…
1. I wrote the content as what i want to mention in the documentation and just used AI to polish it so that its easy to understand, is it hard to understand the documentation right now? 2. Have added the link for downloading both the enwik9 slice and the nyc dataset. Apologies I forgot to add it. You can get it from here - https://github.com/samyak112/pym-particles/blob/main/README.... 3. Other than zip i tested it w…