Viewing profile — joelburget

PDF: https://www.whitehouse.gov/wp-content/uploads/2025/07/Americ...

I wrote an OCaml implementation of this paper a few years ago, which I've now extracted into its own [repo]( https://github.com/joelburget/constructive-reals/blob/main/C... ) The l…

And more recently, [Language Models Use Trigonometry to Do Addition]( https://arxiv.org/abs/2502.00873 )

This is a good idea, though one problem is that Einsum notation (as realized in Numpy and Pytorch) doesn't support the notion of co-contravariance, and the site is based on their E…

A couple of these I'd like references on if anyone happens to have them. 1. "current science suggests that the actual health impact from consuming most types of plastic might well …

o1 is an application of the Bitter Less. To quote Sutton: "The two methods that seem to scale arbitrarily in this way are search and learning." (emphasis mine -- in the original Su…

Vision Transformers do a shocking amount of compression in the tokenizer. In the [Chameleon paper]( https://arxiv.org/pdf/2405.09818 ) they say the tokenizer "encodes a 512 × 512 i…

Vision transformers should be our default guess as to how GPT-4o works, yet this article never mentions them.

It works on all human languages, just inefficiently. I ran it over a sample I found on wikipedia: sample = "ฟองมันฟันหนู, ฟันหนูฟองมัน, ฝนทองฟองมัน" len(sample), len(enc.encode(sam…

A few interesting findings: * the cl100k_base tokenizer has ~100k tokens -- previous tokenizers had ~50k. (enc.n_vocab gives 100277 but some numbers in that range don't work, start…

For those looking to run this on a Mac, the following seems to have worked for me (M1, Big Sur 11.2.3): ``` brew tap gcenx/wine brew install --cask --no-quarantine wine-crossover b…

You have a keen eye -- thanks for the close reading.

Thanks for the heads up. It never occurred to me the svg might render differently. I'll see if we can fix this.