Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLM Visualization

bbycroft.net

Post reply on HN
Print view View original

101–110 of 138 posts

Re: LLM Visualization

#101

Post by jayveeone »

warkanlock wrote: post #10

This is an excellent tool to realize how an LLM actually works from the ground up! For those reading it and going through each step, if by chance you get stuck on why 48 elements are in the first array, please refer to the model.py on minGPT [1] It's an architectural decision that it will be great to mention in the article since people without too much context might lose it [1] https://github.com/karpathy/minGPT/blob…

Yes yes it was the 48 elements thing that got me stuck. Definitely not everything from the second the page loaded.

Re: LLM Visualization

#102

Post by garte »

nlh wrote: post #66
rvz wrote: post #46

Rather than looking at the visuals of this network, it is more better to focus on the actual problem with these LLMs which the author already has shown: With in the transformer section: > As is common in deep learning, it's hard to say exactly what each of these layers is doing, but we have some general ideas: the earlier layers tend to focus on learning lower-level features and patterns, while the later layers learn…

I find this problem fascinating. For decades we’ve puzzled at how the inner workings of the brain works, and thought we’ve learned a lot we still don’t fully understand it. So, we figure, we’ll just make an artificial brain and THEN we’ll be able to figure it out. And here we are, finally a big step closer to an artificial brain and once again, we don’t know how it works :) (Although to be fair we’re spending all of…

Is it a brain, though? As far as I understand it it's mostly stochastic calculations based on language or image patterns whose rule sets are static and immutable. Every conceivable idea of plasticity (and with that: a form of fake consciousness) is only present during training.

Add to that the fact that a model is being trained actively and the weights are given by humans and the possible realm of outputs is being heavily moderated by an army of faceless low paid workers I don't see any semblance of a brain but a very high maintenance indexing engine sold to the marketing departments of the world as a "brain".

Re: LLM Visualization

#107

Post by tysam_and »

Another visualization I would really love would be a clickable circular set of possible prediction branches, projected onto a Poincare disk (to handle the exponential branching component of it all). Would take forever to calculate except on smaller models, but being able to visualize branch probabilities angularly for the top n values or whatever, and to go forwards and backwards up and down different branches would likely yield some important insights into how they work.

Good visualization precludes good discoveries in many branches of science, I think.

(see my profile for a longer, potentially more silly description ;) )

Re: LLM Visualization

#108

Post by crotchfire »

nandhinianand wrote: post #105
crotchfire wrote: post #103

Application error: a client-side exception has occurred (see the browser console for more information).

Seems brave blocks some js scripts .. this works in Firefox

Not using brave.

I get the same fail in both firefox and (chromium-based) qutebrowser.

The web is where useful error messages go to die.

Re: LLM Visualization

#110

Post by fulafel »

altilunium wrote: post #99
johnklos wrote: post #87

Am I the only one getting "Application error: a client-side exception has occurred (see the browser console for more information)." messages?

It is possible that your machine does not yet support WebGL2. Check here : https://get.webgl.org/webgl2/

Or that you've blocked it (some sources recommend this to avoid fingerprinting so various extensions and privacy configuration receipes do it).
Post reply on HN