Viewing profile — measured_step

How do our base reptilian brains reason? We don't know the specifics, but unless it's magic, then it's determined by some kind of logic. I doubt that logic is so unique that it can…

This is a great idea but it is only possible if the model(s) can actually reason. Currently, even GPT-4 struggles with: - Scope - Abduction (compared to deduction and induction whi…

You were safe until you shared that with us, now they can just include your keyboard setup as an input embedding if they target you.

How would this scale for a use case like writing code? I could imagine that some inputs would require a large number of neurons. Would this architecture be able to do that if it we…

For chess engines such as Stockfish to beat weaker opponents, you would want to turn up the contempt parameter.[1] To be able to do this, you would want to use a UCI-compliant ches…

It appears that RAG actually dominates for 2k context lengths compared to this method, but that this method outperforms it more and more the longer the context gets (see the graph …

I disagree. Mistral actually being open source, as well as the new criteria for how open source models are, is pretty worth being aware of. Plus, if they make a larger Mistral mode…