Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Steerling-8B, a language model that can explain any token it generates

guidelabs.ai

Post reply on HN
Print view View original

81–90 of 104 posts

Re: Show HN: Steerling-8B, a language model that can explain any token it generates

#81

Post by monocasa »

kamranjon wrote: post #78

I'm really interested in using this but wonder if the unique architecture means that it will not be able to be converted to a GGUF and used by ollama or llama.cpp? I certainly would understand that the observability features would require some custom tweaks, but I'd just like to try it out on my local ai server (basically just ollama + tailscale) and see how it works as a regular model.

Not immediately, but it's not a much larger amount of work for llama than a new foundational model which typically has a tweaked compute graph.

Re: Show HN: Steerling-8B, a language model that can explain any token it generates

#82

Post by rao-v »

deepdarkforest wrote: post #50

Just wanted to say i think most interpretability research it's just a smoke show nowadays but this is actually the first one that i think has a very serious potential. I love that the SAE is actually constrained and not just slapped unsupervised posthoc. How granular can you get the source data attribution? Down to individual let's say Wikipedia topics? Probably not urls? Would be interested to see this scale to 30/7…

+1 this does seem to be a genuine attempt to actually build an interpretable model, so nice work!

Having said that, I worry that you run into Illusion of Conscious issues where the model changes attrition from “sandbagging” to “unctuous” when you control its response because the response is generated outside of the attribution modules (I don’t quite understand how cleanly everything flows through the concept modules and the residual). Either way this is a sophisticated problem to have. Would love to see if this can be trained to parity with modern 8B models.

Re: Show HN: Steerling-8B, a language model that can explain any token it generates

#84

Post by idiotsecant »

msteffen wrote: post #77

In the recent HN thread announcing the new Gemini coding agent ( https://news.ycombinator.com/item?id=47074735 ), a lot of people complained about Gemini’s tendency to do unwanted refactors, not perform requested actions, etc. It made me cautiously optimistic that all of Anthropic’s work on alignment, which they did for AI safety, is actually the cause of Claude code’s comparatively superior utility (and their presen…

I wonder the opposite, if actual AGI would need to be less aligned. Alignment is basically the process of pruning interesting behavior out of the model to make a product.

Re: Show HN: Steerling-8B, a language model that can explain any token it generates

#85

Post by dwohnitmok »

dwohnitmok wrote: post #8
brendanashworth wrote: post #3

Is there a reason people don't use SHAP [1] to interpret language models more often? The in-context attribution of outputs seems very similar. [1] https://shap.readthedocs.io/en/latest/

SHAP would be absurdly expensive to do for even tiny models (naive SHAP scales exponentially in the number of parameters; you can sample your coalitions to do better but those samples are going to be ridiculously sparse when you're talking about billions of parameters) and provides very little explanatory power for deep neural nets. SHAP basically does point by point ablation across all possible subsets, which really…

Note that the parameters to SHAP can be things other than the model parameters (e.g. model inputs), it's very not obvious what those should be. Indeed that's often the central problem for interpretability (what are my actual features) and SHAP is entirely silent on what those features should be. SHAP could work as a final step if you have a small feature set. But I doubt that LLMs will have a small set of features for any reasonable interpretation of what they do.

Re: Show HN: Steerling-8B, a language model that can explain any token it generates

#86

Post by jzig »

msteffen wrote: post #77

In the recent HN thread announcing the new Gemini coding agent ( https://news.ycombinator.com/item?id=47074735 ), a lot of people complained about Gemini’s tendency to do unwanted refactors, not perform requested actions, etc. It made me cautiously optimistic that all of Anthropic’s work on alignment, which they did for AI safety, is actually the cause of Claude code’s comparatively superior utility (and their presen…

What does alignment even mean? What is being aligned and what is it aligning to?

Re: Show HN: Steerling-8B, a language model that can explain any token it generates

#87

Post by 0xdeadbeefbabe »

adebayoj wrote: post #18
ottah wrote: post #10

It's a neat party trick, but explainability it's not solution to any AI safety issue I care about. It's a distraction from real problems, which is everything else around the model. The inflexible bureaucratic systems that make it hard to exercise rights and deflect accountability.

op here. Important point, but I disagree. We see explainability/interpretability as a CORE need for AI safety. We believe you can't align/audit/debug/fix a system that you don't understand. Just to give you some answers for what we can do: 1) We can find the training data that is causing a model to output toxic/unwanted text and correct it. 2) We know what high level concepts the model is relying on for any group of…

Hmm so like git blame?

Re: Show HN: Steerling-8B, a language model that can explain any token it generates

#89

Post by jacquesm »

adebayoj wrote: post #56
crimsonnoodle58 wrote: post #53

So maybe one day we'll see coding agents like Claude Code create and update an ATTRIBUTION.md, citing all the open source projects and their licenses used to generate code in your project?

You got it exactly right :) And you can update the attribution.md to have it NOT rely on opensource projects that have been compromised. Imagine asking claude code to write a package/function in the style of a codebase that you care about or force it to ALWAYS rely on some internal packages that you care about. The possibilities are endless when you insert such knobs into models.

I would rather see that it does not rely on open source projects that have not given permission to be used to train that particular AI on.

Re: Show HN: Steerling-8B, a language model that can explain any token it generates

#90

Post by killerstorm »

This seems to be too coarse-grained to be useful: all sciency content will be "analytical" and associate with sources like ArXiv.

But there might be bad, malicious articles on ArXiv, so it doesn't really say anything about veracity.

Perhaps this might help to detect some problems like prompt injection - but then it might be more interesting to see those examples.

Post reply on HN