Viewing profile — topwalktown

I wrote it myself it python. The model runs in a loop and can either read the terminal or input keys. If it does neither the loop exits.

Sorry! It should work now.

I test if qwen3.5-35B-A3B can exit vim when running in a harness that allows it to read the raw terminal buffer and send raw key presses to the terminal. Enjoy!

Transformers like Llama use rotary embeddings which are applied in every single attention layer https://github.com/huggingface/transformers/blob/222505c7e4d...

I'm trying to train a variable resolution ViT using IJEPA. I'm currently topping out at about 30% on imagenet1k after training for 20 epochs (6 hours) It'd be cool to have some hel…

i experiment with using a (mostly) unmodified llama model to generate images, by training on the bits from a lossy compression algorithm. It turns out the key is having a decoder w…

yeah, check out the Emu paper by meta. They basically do all of what is mentioned in the above comment

Quantization also works as regularization; it stops the neural network from being able to use arbitrarily complex internal rules. But really it's only really useful if you absolute…

I wrote a short article about jpg and if we could use concepts from how jpg works to make an image autoencoder that has a left-to-right positional bias and variable compression Bas…