LLM Visualization • Hacker News Board

Re: LLM Visualization

#121

Post by modriano » Mon, Dec 04, 2023, 2:46 PM UTC

nocoder wrote: post #94

block_dagger wrote: post #38

Earlier quoted context omitted.

To know, you must perform intellectual work, not merely be smart. I bet you are smart enough.

What a nice comment!! This has been a big failing of my mental model. I always believed if I was smart enough I should understand things without effort. Still trying to unlearn this....

Unfortunately you must look closely at the details to deeply understand how something works. Even when I already have a decent mental heuristic about how an algorithm works, I get a much richer understanding by calculating the output of an algorithm by hand.

At least for me, I don't really understand something until I can see all of the moving parts and figure out how they work together. Until then, I just see a black box that does surprising things when poked.

Re: LLM Visualization

#122

Post by jimmySixDOF » Mon, Dec 04, 2023, 2:46 PM UTC

taliesinb wrote: post #98

warkanlock wrote: post #10

This is an excellent tool to realize how an LLM actually works from the ground up! For those reading it and going through each step, if by chance you get stuck on why 48 elements are in the first array, please refer to the model.py on minGPT [1] It's an architectural decision that it will be great to mention in the article since people without too much context might lose it [1] https://github.com/karpathy/minGPT/blob…

Wow, I love the interactive wizzing around and the animation, very neat! Way more explanations should work like this. I've recently finished an unorthodox kind of visualization / explanation of transformers. It's sadly not interactive, but it does have some maybe unique strengths. First, it gives array axis semantic names , represented in the diagrams as colors (which this post also uses). So sequence axis is red, ke…

You might also like this interactive 3D walk through explainer from PyTorch :

https://pytorch.org/blog/inside-the-matrix/

Re: LLM Visualization

#123

Post by stareatgoats » Mon, Dec 04, 2023, 3:09 PM UTC

SiempreViernes wrote: post #112

Anyone know if there is a name for this 3D control schema? This feels like one of the most intuitive setups I've ever used.

Not sure if it has a name, but you might find out in the github repository: https://github.com/bbycroft/llm-viz/tree/main

Re: LLM Visualization

#124

Post by lubesGordi » Mon, Dec 04, 2023, 3:48 PM UTC

albertzeyer wrote: post #76

BOOSTERHIDROGEN wrote: post #62

Earlier quoted context omitted.

What books cover exclusively about this topic ? Thanks

This is frequently a topic here on HN. E.g.: https://udlbook.github.io/udlbook/ ( https://news.ycombinator.com/item?id=38424939 ) https://fleuret.org/francois/lbdl.html ( https://news.ycombinator.com/item?id=35767789 ) https://www.fast.ai/ ( https://news.ycombinator.com/item?id=24237207 ) https://d2l.ai/ ( https://news.ycombinator.com/item?id=38428225 ) Some more: https://news.ycombinator.com/item?id=35543774 There i…

Thanks for the udl book (Understanding Deep Learning), that looks like a really great starting point.

Re: LLM Visualization

#125

Post by crimsoneer » Mon, Dec 04, 2023, 4:21 PM UTC

arikrak wrote: post #41

This looks pretty cool! Anyone know of visualizations for simpler neural networks? I'm aware of tensorflow playground but that's just for a toy example, is there anything for visualizing a real example (e.g handwriting recognition)?

I like this one: https://aegeorge42.github.io/

Re: LLM Visualization

#126

Post by itslennysfault » Mon, Dec 04, 2023, 4:26 PM UTC

flockonus wrote: post #64

Twitter thread by the author sharing some extra context on this work: https://twitter.com/BrendanBycroft/status/173104295714982714...

Thanks for sharing. This is a great thread.

Since X now hides replies for non-logged in user here is a nitter link for those without an account (like me) that might want to see the full thread.

https://nitter.net/BrendanBycroft/status/1731042957149827140

Re: LLM Visualization

#127

Post by Exuma » Mon, Dec 04, 2023, 4:39 PM UTC

This is really awesome but I at least wish there were a few added sentences around how I'm supposed to intuitively think about the purpose of why it's like that. For example, I see a T x C matrix of 6 x 48... but at this step, before it's fed into the net, what is this supposed to represent?

Re: LLM Visualization

#128

Post by singularity2001 » Mon, Dec 04, 2023, 5:56 PM UTC

Exuma wrote: post #127

This is really awesome but I at least wish there were a few added sentences around how I'm supposed to intuitively think about the purpose of why it's like that. For example, I see a T x C matrix of 6 x 48... but at this step, before it's fed into the net, what is this supposed to represent?

Also later why 8 and why is "A" expected in the sixth position

Re: LLM Visualization

#129

Post by Workaccount2 » Mon, Dec 04, 2023, 6:23 PM UTC

Such an amazing tool

Re: LLM Visualization

#130

Post by SubiculumCode » Mon, Dec 04, 2023, 7:47 PM UTC

block_dagger wrote: post #38

wills_forward wrote: post #33

My jaw drop to see algorhythmic complexity laid out so clearly in a 3d space like that. I wish I was smart enough to know if it's accurate or not.

To know, you must perform intellectual work, not merely be smart. I bet you are smart enough.

99% persperation, 1% inspiration, as the addage goes...and I completely agree.

The frustration for the curious is that there is more than you can ever learn. You encounter something new and exciting, but then you realize that to really get to the spot where you can contribute will take at least a year or six, and that will require dropping other priorities.