Viewing profile — adebayoj

Hey HN! We're Guide Labs and we just launched Clarity, an AI platform powered by our Steerling-8B model. You can: Click any chunk of output the model generates and see the concepts…

Hey HN we recently released Steerling-8B, an 8B model designed to be interpretable from the ground up. The model has ~100K concept slots it fills on its own during training, and we…

This part of the claim is involved, so we have future posts to clarify this. And yes, you can remove a prototype and generate again. We show examples in that prism post. In prism, …

It is not hype. You can try the model on huggingface yourself to see its capabilities. My reply here was clarifying that the examples we showed were ones where the model didn't mak…

We train the model with `explanations`. Most training asks the model to predict the next token or group of tokens. Our training says, predict the next group of tokens (causal diffu…

Take a look at the link in the blogposts. Here is a github link as well: https://github.com/guidelabs/steerling . The model weights are on huggingface, so you can play with it.

Great questions. We weren't quite explicit about the training data attribution process. We'll discuss this in more detail in future work. We can track down which parts of the train…

You got it exactly right :) And you can update the attribution.md to have it NOT rely on opensource projects that have been compromised. Imagine asking claude code to write a packa…

Down to the very exact text chunk in a document! Check this out for an idea of what smaller versions of this style of model can do: https://www.guidelabs.ai/post/prism/ . We'll hav…

Yes, that is the post that has the most up to date details of the model architecture. Take a look at this: https://github.com/guidelabs/steerling . It has the scaffolding for what …

You are exactly right, it is guiding the model, during training, with concepts and the dictionary. This is important because dictionary learning for interpretability (post hoc) is …

It is impossible to completely get rid of hallucinations. However, this can tell you exactly why the model hallucinated.

Exactly! We will have a future post that shows this more granularly over the coming weeks. Here is a post we wrote on how this works at smaller scale: https://www.guidelabs.ai/post…

Our decomposition allows us to answer question like: for 84 percent of the model's representation, we know it is relying on this concept to give an answer. We can also trace its be…

Thanks for pointing this out. LLama 3 8B was trained on ~15T tokens. The Qwen models on 15-18T tokens as well. We trained on 1.35T tokens, and are within shot of these models on be…

Good point. Historically, people have thought that there is a interpretability vs quality/performance tax. This is not true; at least not in this case. Here are a bunch of question…

sounds great! Will follow up via email.

Thanks, it is certainly a first step.

You are missing a few things, but you got some things right. 1) The is not an SAE in the way you think. It is a combination of a supervised + unsupervised layer that is constrained…

It does :) We constrained the model to do exactly this during training: https://www.guidelabs.ai/post/scaling-interpretable-models-8... .

You got it exactly right. Guilty as charged. Over the coming weeks, we will be showcasing exactly how you can debug all of these examples. I agree that attribution is most useful f…