Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: LLM Attention Visualization

ishamf.dev

Post reply on HN
Print view View original

11–20 of 30 posts

Re: Show HN: LLM Attention Visualization

#11

Post by ifz »

stared wrote: post #9

I am curious what's the actual formula. I mean, there so many headers and layers, it is tricky to make a choice that will resonate with our intuition . Is it some weighted average? Or maybe ablation test?

It's really simple, basically just the magnitude of the value vector, weighted by QK dot product, summed across all attention heads and layers.

When I started, I expected I'd have to experiment a lot to find something comprehensible. But this simple computation can already show some patterns.

Re: Show HN: LLM Attention Visualization

#12

Post by ifz »

wopak wrote: post #8

neat, combining info from two phrases is hard to see without such a tool. are you worried later-layer attention gets drowned out by earlier layers just because there are more of them contributing to the sum?

Hmm, I might try to add some controls to limit which layers get summed up. It might be able to reveal more patterns.

Right now only simple correlations are visible.

Re: Show HN: LLM Attention Visualization

#13

Post by visarga »

stared wrote: post #9

I am curious what's the actual formula. I mean, there so many headers and layers, it is tricky to make a choice that will resonate with our intuition . Is it some weighted average? Or maybe ablation test?

If you want quick access look at google images for "transformer attention formula" there are some interesting depictions

Re: Show HN: LLM Attention Visualization

#14

Post by stared »

ifz wrote: post #11
stared wrote: post #9

I am curious what's the actual formula. I mean, there so many headers and layers, it is tricky to make a choice that will resonate with our intuition . Is it some weighted average? Or maybe ablation test?

It's really simple, basically just the magnitude of the value vector, weighted by QK dot product, summed across all attention heads and layers. When I started, I expected I'd have to experiment a lot to find something comprehensible. But this simple computation can already show some patterns.

Nice! Sometimes the simplest approaches work the best.

Re: Show HN: LLM Attention Visualization

#18

Post by TomatoCo »

ex-aws-dude wrote: post #15

I don't know much about LLMs but does that mean you have N^2 computation with the context size since every token needs to track how it relates to every other token?

Yes, except no with the KV cache. Because tokens aren't modified by future tokens you can cache the meaning of previous tokens. This makes the total effort linear over the entire context (or constant per forward pass).

Re: Show HN: LLM Attention Visualization

#19

Post by ex-aws-dude »

TomatoCo wrote: post #18
ex-aws-dude wrote: post #15

I don't know much about LLMs but does that mean you have N^2 computation with the context size since every token needs to track how it relates to every other token?

Yes, except no with the KV cache. Because tokens aren't modified by future tokens you can cache the meaning of previous tokens. This makes the total effort linear over the entire context (or constant per forward pass).

I see and is there only 1 layer of relations?

Or does it accumulate the relations like A relates to B, so also add in B's relations

Post reply on HN