Show HN: LLM Attention Visualization
ishamf.dev
Show HN: LLM Attention Visualization
1–10 of 30 posts
Re: Show HN: LLM Attention Visualization
#2[dead]
Re: Show HN: LLM Attention Visualization
#3I highly question this simplistic idea of high vector magnitude = high influence.
Re: Show HN: LLM Attention Visualization
#4I highly question this simplistic idea of high vector magnitude = high influence.
I don't disagree with that. I did add an entire caveat paragraph there.
To me, it's more of a neat visualization, not something that can be used to interpret LLM behavior. Even with a lot of simplification, it can show some interesting patterns.
Re: Show HN: LLM Attention Visualization
#5I highly question this simplistic idea of high vector magnitude = high influence.
You get what you pay for. If you want to think harder and get more, https://transformer-circuits.pub/2025/attention-qk/index.htm...
Re: Show HN: LLM Attention Visualization
#6I like the visualisation. Pretty cool
Re: Show HN: LLM Attention Visualization
#7I highly question this simplistic idea of high vector magnitude = high influence.
Same I dont get it just, could you clarify it
Re: Show HN: LLM Attention Visualization
#8neat, combining info from two phrases is hard to see without such a tool.
are you worried later-layer attention gets drowned out by earlier layers just because there are more of them contributing to the sum?
Re: Show HN: LLM Attention Visualization
#9I am curious what's the actual formula.
I mean, there so many headers and layers, it is tricky to make a choice that will resonate with our intuition . Is it some weighted average? Or maybe ablation test?
Re: Show HN: LLM Attention Visualization
#10This is great, I've read multiple books and watched videos about the attention mechanism. Now that I understand it, this is the clearest example I've seen on how attention works.