Viewing profile — taliesinb

Interesting. It all seems very brittle, though. And that something has gone very wrong with our ecosystem of tools, languages, and processes when it becomes advisable to massage so…

Amusingly, you need only have circuits of prime depth, though you should probably adjust their widths using something principled, perhaps Euler's totient function.

And i bet these would be useful in initial and final parts of transformer too. Because syntactic parsing and unparsing of brackets, programming language ASTs, etc is highly recursi…

There is an obvious implication: since the initial models were trained without loops, it is exceedingly unlikely that a single stack of consecutive N layers represents only a singl…

If you could link to where this is implemented I'd be very grateful!

Hello! Yes I am curious, how does one deal with cycles in the code hash graph? Mutually recursive functions for example?

I’ve been super interested in these kinds of cosmic turduckens. See also https://en.wikipedia.org/wiki/Thorne%E2%80%93%C5%BBytkow_obj... and https://en.wikipedia.org/wiki/Quasi-sta…

Given that whole name binding thing is ultimately a story of how to describe a graph using a tree, I was primed to look for monoidal category-ish things, and sure enough the S and …

Yes, this is a very cool story. But, fascinatingly, integration does in fact have a meaning. First, recall from the OP that d/dX List(X) = List(X) * List(X). You punched a hole in …

Cool! Looks pretty professional. I explored a similar idea once (also implemented in Python, via decorators) to help speed up some neuroscience research that involved a lot of hype…

The end-game is just dissolving any distinction between compile-time and run-time. Other examples of dichotomies that could be partially dissolved by similar kinds of universal aci…

An earnest question: can you elaborate on what Zig got wrong in that respect?

In another life!

Yes, get in touch with me.

Symbolica.ai | London, Australia | REMOTE, INTERNS, VISA We're trying to apply the insights of category theory, dependent type theory, and functional programming to deep learning. …

No worries! Yes, exactly. It's also similar to doing arithmetic with and without units. Sure, you can do arithmetic without units, but when you are actually working with real-world…

Looks interesting, but your link is broken, can you try again or give us a direct PDF or Arxiv link?

Thanks for highlighting XArray in your other comments. Yup, XArray is great. As are Dex and the various libraries for named axis DL programming within PyTorch and Jax. I never said…

While I applaud the OP's exposition, imagine that instead of having axis names live as single-letter variables within einsum, our arrays themselves had these names attached to thei…

Hey there, a bit late but I've read your other comments and I'd like to get in touch. I happen to be very focused on type derivatives, in the context of applying category theory to…

That is an excellent explanation full of great intuition building! If anyone is interested in a kind of tensor network-y diagrammatic notation for array programs (of which transfor…

Amazing, but that's a lot of channels for a cheap oscilloscope. I'm curious how he did that, I don't see any wiring. Also, he missed an opportunity to bring in the rudiments of tra…

Wow, I love the interactive wizzing around and the animation, very neat! Way more explanations should work like this. I've recently finished an unorthodox kind of visualization / e…

QE did not lead to historically high levels of inflation. Most of the money stayed in the financial sector, on banks balance sheets.

I guess we should distinguish money from commodities, though? For money to mean something different to commodities, it must enjoy a privileged legal and social role, by virtue of i…