Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLM Visualization

bbycroft.net

Post reply on HN
Print view View original

31–40 of 138 posts

Re: LLM Visualization

#31

Post by sgnelson »

Arson9416 wrote: post #11
reqo wrote: post #4

I feel like visualizations like this are what is missing from univeristy curricula. Now imagine a professor going trough each animation describing exactly what is happening, I am pretty sure students would get a much more in-depth understanding!

Isn't it amazing that a random person on the internet can produce free educational content that trumps university courses? With all the resources and expertise that universities have, why do they get shown up all the time? Do they just not know how to educate?

Because Faculty are generally paid very poorly, have many courses to teach, and what takes up more and more of their time are the broken bureaucratic systems they have to deal with.

Add that at research universities, they have to do research.

Also add in that at many schools, way too many students are just there to clock in, clock out and get a piece of paper that says they did it. Way too few are there to actually get an education. This has very real consequences on the moral of the instructors. When your students don't care, it's very hard for you to care. If your students aren't willing to work hard, why are you willing to work hard? Because you're paid so well?

I know plenty of instructors who would love to do things like this, but when are they going too? When are they going to find the time to learn the skills necessary to build an interactive web app? You think everyone outside of comp sci and like disciplines just naturally know how to build these types of apps?

I could go on, but the tl;dr of it is: Educators are over worked, underpaid and don't have enough time in the day.

Re: LLM Visualization

#32

Post by jwilber »

reqo wrote: post #4

I feel like visualizations like this are what is missing from univeristy curricula. Now imagine a professor going trough each animation describing exactly what is happening, I am pretty sure students would get a much more in-depth understanding!

If you enjoy interactive visuals of machine learning algorithms, do check out https://mlu-explain.github.io/

Re: LLM Visualization

#34

Post by erosenbe0 »

Arson9416 wrote: post #11
reqo wrote: post #4

I feel like visualizations like this are what is missing from univeristy curricula. Now imagine a professor going trough each animation describing exactly what is happening, I am pretty sure students would get a much more in-depth understanding!

Isn't it amazing that a random person on the internet can produce free educational content that trumps university courses? With all the resources and expertise that universities have, why do they get shown up all the time? Do they just not know how to educate?

I second the reply about incentives. Funding curriculum materials and professional curriculum development is often seen as more of a K-12 thing. There is not even enough at the vocational level.

If big competitive grants and competitive salaries went to people with demonstrated ability like the engineer of this viz, there would be less stem dropouts in colleges and more summer learning! Also, in technical trades like green construction, solar, hvac, building retrofits, data center operations and the like, people would get farther and it would be a more diverse bunch.

Re: LLM Visualization

#35

Post by lusus_naturae »

Arson9416 wrote: post #11
reqo wrote: post #4

I feel like visualizations like this are what is missing from univeristy curricula. Now imagine a professor going trough each animation describing exactly what is happening, I am pretty sure students would get a much more in-depth understanding!

Isn't it amazing that a random person on the internet can produce free educational content that trumps university courses? With all the resources and expertise that universities have, why do they get shown up all the time? Do they just not know how to educate?

The person who made this went to university for maths education (if I found the right profile).

Re: LLM Visualization

#36

Post by webnrrd2k »

Arson9416 wrote: post #11
reqo wrote: post #4

I feel like visualizations like this are what is missing from univeristy curricula. Now imagine a professor going trough each animation describing exactly what is happening, I am pretty sure students would get a much more in-depth understanding!

Isn't it amazing that a random person on the internet can produce free educational content that trumps university courses? With all the resources and expertise that universities have, why do they get shown up all the time? Do they just not know how to educate?

This is the result of a single person on the internet, who was not chosen randomly. it's not a fair characterization to call this the product of some random person on the internet. You can't just choose anyone on the internet at random and get results this good.

Also, according to his home page, Mr. Bycroft earned a BSc in Mathematics and Physics from the University of Canterbury in 2012. It's true that this page isn't the direct result of a professor or a university course, and it's also true that it's not a completely separate thing. It seems clear that his university education had a big role to play in creating this.

Re: LLM Visualization

#37

Post by drdg »

Very cool. The explanations of what each part is doing is really insightful. And I especially like how the scale jumps when you move from e.g. Nano all the way to GPT-3 ....

Re: LLM Visualization

#39

Post by atgctg »

A lot of transformer explanations fail to mention what makes self attention so powerful.

Unlike traditional neural networks with fixed weights, self-attention layers adaptively weight connections between inputs based on context. This allows transformers to accomplish in a single layer what would take traditional networks multiple layers.

Post reply on HN