Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLM Visualization

bbycroft.net

Post reply on HN
Print view View original

61–70 of 138 posts

Re: LLM Visualization

#61

Post by CyberDildonics »

kaimac wrote: post #54
kmeisthax wrote: post #43

Earlier quoted context omitted.

None of this seems obvious just reading the original Attention is all you need paper. Is there a more in-depth explanation of how this adaptive weighting works?

I found these notes very useful. They also contain a nice summary of how LLMs/transformers work. It doesn't help that people can't seem to help taking a concept that has been around for decades (kernel smoothing) and giving it a fancy new name (attention). http://bactra.org/notebooks/nn-attention-and-transformers.ht...

It's just as bad a "convolutional neural networks" instead of "images being scaled down"

Re: LLM Visualization

#62

Post by BOOSTERHIDROGEN »

albertzeyer wrote: post #55
kmeisthax wrote: post #43

Earlier quoted context omitted.

None of this seems obvious just reading the original Attention is all you need paper. Is there a more in-depth explanation of how this adaptive weighting works?

The audience of this paper are other researchers who already know the concept of attention, which was very well known already in the field. In such research papers, such things are never explained again, as all the researchers already know this or can read other sources, which are cited, but focus on the actual research questions. In this case, the research question was simply: Can we get away by just using attention…

What books cover exclusively about this topic ? Thanks

Re: LLM Visualization

#63

Post by bobbylarrybobby »

airstrike wrote: post #56
WhitneyLand wrote: post #44

Earlier quoted context omitted.

In case it’s confusing for anyone to see “weight” as a verb and a noun so close together, there are indeed two different things going on: 1. There are the model weights, aka the parameters. These are what get adjusted during training to do the learning part. They always exist. 2. There are attention weights. These are part of the transformer architecture and they “weight” the context of the input. They are ephemeral.…

I always thought the verb was "weigh" not "weight", but apparently the latter is also in the dictionary as a verb. Oh well... it seems like it's more confusing than I thought https://www.merriam-webster.com/wordplay/when-to-use-weigh-a...

“To weight” is to assign a weight (e.g., to weight variables differently in a model), whereas “to weigh” is to observe and/or record a weight (as a scale does).

Re: LLM Visualization

#65

Post by russellbeattie »

I've wondered for a while if as LLM usage matures, there will be an effort to optimize hotspots like what happened with VMs, or auto indexed like in relational DBs. I'm sure there are common data paths which get more usage, which could somehow be prioritized, either through pre-processing or dynamically, helping speed up inference.

Re: LLM Visualization

#66

Post by nlh »

rvz wrote: post #46

Rather than looking at the visuals of this network, it is more better to focus on the actual problem with these LLMs which the author already has shown: With in the transformer section: > As is common in deep learning, it's hard to say exactly what each of these layers is doing, but we have some general ideas: the earlier layers tend to focus on learning lower-level features and patterns, while the later layers learn…

I find this problem fascinating.

For decades we’ve puzzled at how the inner workings of the brain works, and thought we’ve learned a lot we still don’t fully understand it. So, we figure, we’ll just make an artificial brain and THEN we’ll be able to figure it out.

And here we are, finally a big step closer to an artificial brain and once again, we don’t know how it works :)

(Although to be fair we’re spending all of our efforts making the models better and better and not on learning their low level behaviors. Thankfully when we decide to study them it’ll be a wee less invasive and actually doable, in theory.)

Re: LLM Visualization

#67

Post by DavidSJ »

bobbylarrybobby wrote: post #63
airstrike wrote: post #56

Earlier quoted context omitted.

I always thought the verb was "weigh" not "weight", but apparently the latter is also in the dictionary as a verb. Oh well... it seems like it's more confusing than I thought https://www.merriam-webster.com/wordplay/when-to-use-weigh-a...

“To weight” is to assign a weight (e.g., to weight variables differently in a model), whereas “to weigh” is to observe and/or record a weight (as a scale does).

A few other cases of this sort of thing:

affect (n). an emotion or feeling. "She has a positive affect."

effect (n). a result or change due to some event. "The effect of her affect is to make people like her."

affect (v). to change or modify [X], have an effect upon [X]. "The weather affects my affect."

effect (v). to bring about [X] or cause [X] to happen. "Our protests are designed to effect change."

Also:

cost (v). to require a payment or loss of [X]. "That apple will cost $5." Past tense cost: "That apple cost $5."

cost (v). to estimate the price of [X]. "The accounting department will cost the construction project at $5 million." Past tense costed. "The accounting department costed the construction project at $5 million."

Re: LLM Visualization

#68

Post by gtoubassi »

kmeisthax wrote: post #43
atgctg wrote: post #39

A lot of transformer explanations fail to mention what makes self attention so powerful. Unlike traditional neural networks with fixed weights, self-attention layers adaptively weight connections between inputs based on context. This allows transformers to accomplish in a single layer what would take traditional networks multiple layers.

None of this seems obvious just reading the original Attention is all you need paper. Is there a more in-depth explanation of how this adaptive weighting works?

I struggled to get an intuition for this, but on another HN thread earlier this year saw the recommendation for Sebastian Raschka's series. Starting with this video: https://www.youtube.com/watch?v=mDZil99CtSU and maybe the next three or four. It was really helpful to get a sense of the original 2014 concept of attention which is easier to understand but less powerful (https://arxiv.org/abs/1409.0473), and then how it gets powerful with the more modern notion of attention. So if you have a reasonable intuition for "regular" ANNs I think this is a great place to start.

Re: LLM Visualization

#69

Post by holtkam2 »

The visualization I've been looking for for months. I would have happily paid serious money for this... the fact that it's free is such a gift and I don't take it for granted.

Re: LLM Visualization

#70

Post by CardenB »

BOOSTERHIDROGEN wrote: post #62
albertzeyer wrote: post #55

Earlier quoted context omitted.

The audience of this paper are other researchers who already know the concept of attention, which was very well known already in the field. In such research papers, such things are never explained again, as all the researchers already know this or can read other sources, which are cited, but focus on the actual research questions. In this case, the research question was simply: Can we get away by just using attention…

What books cover exclusively about this topic ? Thanks

I doubt any.
Post reply on HN