31–40 of 83 posts
So the codec would be something like:
I've seen experiments where people have a "fixed" pipeline but I think having something more dynamic would work quite well.
Earlier quoted context omitted.
And just for comparison, my absolute best compression method managed to get down to 10s of KB, but the real unlock got to the ~1KB figures. Note these numbers are ALL post-compression numbers. This is not raw data vs compressed data. The ~100KB figure IS POST COMPRESSION. For context these numbers are for a grid based game where players can perform 4 actions per second, and the numbers I’m sharing are for 30 minutes…
I was working on a multiplayer game a while ago, and one of the iterations of the netcode was "thin client" where clients just sent input, server simulated the game, and it dumped world state onto the pipe at 60hz. I didn't ship that version but I estimated a $3000 bandwidth bill with that approach! I started looking into diffing the state, compression, etc... until I realized, wait a minute! My player movement is li…
Clever insight :) yes a specialised solution usually wins! Good effort
Did you end up publishing your game?
What do those compress to with conventional approaches? For comparison. I am curious. A classic machine learning ensemble approach is to overfit a collection of small models then bag them (e.g. voting) allowing the models to generalize. I'm sure someone's tried to overfit a bunch of transformers for compression like this, then bag them to see how well it does?
Check it out: https://github.com/samyak112/pym-particles/blob/main/arithme...
Earlier quoted context omitted.
No.. they're not. Do you understand random (the apparent or actual lack of definite patterns or predictability[0]) or compression (reduces bits by identifying and eliminating statistical redundancy[1])? [0]: https://en.wikipedia.org/wiki/Randomness [1]: https://en.wikipedia.org/wiki/Data_compression
by this definition, a random dataset could apparently present no patterns, while presenting non apparent patterns.
A non-general compression algorithm (model - I don't mean a distinct llm, but "modeling data") targeted at a specific dataset will always do better than a general algorithm.
The reason I mentioned the "encoder" doesn't matter - arithmetic coding, for the data it is presented, will beat huffman/adaptive huffman every day, but it's the model that is where the real "compression" comes into play.
I've implemented enough "coders" over the years, including arithmetic for both commercial and research purposes (was a student of Glen Langdon).
So apply this same logic to compressing a bigger model within a smaller model
I know this is absolutely regarded, but humour me please
Dumb question: can you train a model to predict the next byte of ANOTHER MODEL So apply this same logic to compressing a bigger model within a smaller model I know this is absolutely regarded, but humour me please
Dumb question: can you train a model to predict the next byte of ANOTHER MODEL So apply this same logic to compressing a bigger model within a smaller model I know this is absolutely regarded, but humour me please
Not dumb at all. It's a whole field of active research - Speculative Decoding. A recent paper goes one level deeper with Speculative Speculative Decoding - https://arxiv.org/abs/2603.03251
Compression is such an interesting field