Viewing profile — jsenn

I don’t see Tao suggesting what you have suggested there. Instead he suggests that humans responsibly disclose AI use, and that mathematicians develop a set of norms to deal with a…

I can see this being important if you only care about the results as evaluations of AI progress, but if what you care about is the math itself why should you care about the prompt …

This looks cool, but I wonder how well their trained compiler generalizes to new task families . They trained on 29 specific types of tasks, with 800 sub tasks and many rephrasings…

Their demo is almost unbelievably fast, but as I understand it, the limitation of Taalas's strategy is KV-cache. This grows with context length, so either needs to be stored in SRA…

> isn't the verification code going to be sloppy as well The beauty of formal methods is it doesn't matter if your proof is sloppy. As long as it passes verification, it is correct…

The article you are responding to showed that a strange LLM behaviour was caused by a training signal that was explicitly designed to produce that type of behaviour. They were able…

Is this parlour trick so different from useful tasks like “implement this feature while following the naming conventions of my project”?

> Section 2.6 gives the hidden state size per token, which, on first read, is strictly larger than the hidden state in normal attention This is where you’ve gone off track. The “hi…

You can find papers discussing "cubic" attention, i.e. each token gets to interact with each pair of other tokens, but always in very theoretical settings with single-layer transfo…

If you remove the terms "self", "agency", and "trivially reducible", it seems to me that a classical robot/game AI planning algorithm, which no one thinks is conscious, matches the…

I don’t know the post you’re referring to but I highly recommend How the Immune System Works by Lauren Sompayrac. It explains the interesting parts without getting bogged down in t…

A string fixed at both ends produces harmonic sounds because of its particular structure. In order to have a non-integer overtone the ends would have to move up and down, which by …

> It will have overtones that are integer multiples of the fundamental that give it its characteristic sound. What I’m wondering is why would the overtones go in integer multiples …

Is the drone of a fan harmonic? I would’ve thought it’s more like a repetition pitch so its overtones would not be harmonic and would not exhibit a missing fundamental. Agree with …

Apple's implementation of this is discussed here: https://news.ycombinator.com/item?id=43685714

yeah, the linked paper [1] has more detail--basically they seem to start with a seed set of "class labels" and subcategories (e.g. "restaurant review" + "steak house"). They ask an…

> This approach works by randomly polling participating devices for whether they’ve seen a particular fragment, and devices respond anonymously with a noisy signal. By noisy, we me…

I think the main advantage is that you can compute the extra parameters (the PRNG seeds) from the network weights alone, whereas most other quantization methods require simulating …

> What makes this technique particular to LLM weights This is my understanding as a non-expert. LLM activations tend to be relatively sparse with large outliers. With linear quanti…

This doesn’t answer your question, but one thing to keep in mind is that past the very first layer, every “token” position is a weighted average of every previous position, so adja…