Show HN: Visual intuitive explanations of LLM concepts (LLM University)
11–20 of 37 posts
Re: Show HN: Visual intuitive explanations of LLM concepts (LLM University)
#12> We've just published a lot of original, visual, and intuitive explanations of concepts to introduce people to large language models. Kinda frustrating that the main link dumps me onto what reads like a university syllabus, and nothing original, visual, or intuitive. If I click through the sections in order, there are 5 "preamble" sections describing logistical and other meta-information about the course. All text.…
Re: Show HN: Visual intuitive explanations of LLM concepts (LLM University)
#13> We've just published a lot of original, visual, and intuitive explanations of concepts to introduce people to large language models. Kinda frustrating that the main link dumps me onto what reads like a university syllabus, and nothing original, visual, or intuitive. If I click through the sections in order, there are 5 "preamble" sections describing logistical and other meta-information about the course. All text.…
The landing page is technically the course overview. I'd love to hear what you think would've made it more engaging for you. We can probably pull up some of the visuals to it as a preview. Let me see what we can do on that front.
Re: Show HN: Visual intuitive explanations of LLM concepts (LLM University)
#14What type of representations are being used internally in these models ? We've got token embeddings going in, and it seems like some type of semantic embeddings internally perhaps, but exactly what ? OTOH it's outputting words (tokens) with only a linear layer between the last transformer block and the softmax, so what does that say about the representations at that last transformer block ?
Re: Show HN: Visual intuitive explanations of LLM concepts (LLM University)
#15Re: Show HN: Visual intuitive explanations of LLM concepts (LLM University)
#16> We've just published a lot of original, visual, and intuitive explanations of concepts to introduce people to large language models. Kinda frustrating that the main link dumps me onto what reads like a university syllabus, and nothing original, visual, or intuitive. If I click through the sections in order, there are 5 "preamble" sections describing logistical and other meta-information about the course. All text.…
That’s just you. I find the apple thing obvious on first sight and cements an intuition that talk of vectors does not or differently. Why choose?
EDIT: in some sense, the whole idea and usefulness of embedding comes from it working like the inverse of this kind of "intelligence"/"logic" tests - tests that ask you to which, out of several groups, a new symbol belongs. Usually there are couple competing answers, but the test has you guess the one that's the Right One. Embedding is about subverting this - it's about telling the test giver, "you know what, it actually belongs to all of them", and adding enough dimensions to the problem space that you can have all the groups be far away, from each other, and the new thing close to all of them - to each along a different dimension.
Re: Show HN: Visual intuitive explanations of LLM concepts (LLM University)
#17I'm not sure how much is actually known to write about, but what I'd like to see explained is how transformer-based LLMs/AI really work - not at the mechanistic level of the architecture, but in terms of what they learn (some type of world model ? details, not hand waving!) and how do they utilize this when processing various types of input ? What type of representations are being used internally in these models ? We…
Re: Show HN: Visual intuitive explanations of LLM concepts (LLM University)
#18Minor nitpick: The intercom button obscures the topic expansion button for the final appendix in the nav menu. Maybe move intercom to the bottom right instead?
Re: Show HN: Visual intuitive explanations of LLM concepts (LLM University)
#19I'm not sure how much is actually known to write about, but what I'd like to see explained is how transformer-based LLMs/AI really work - not at the mechanistic level of the architecture, but in terms of what they learn (some type of world model ? details, not hand waving!) and how do they utilize this when processing various types of input ? What type of representations are being used internally in these models ? We…
> not at the mechanistic level of the architecture, but in terms of what they learn (some type of world model ? details, not hand waving!) https://imgs.xkcd.com/comics/tasks.png
I'm sure some of key players know at least a little, but they don't seem inclined to share. In his Lex Fridman interview Sam Altam said something along the lines of "a LOT of knowledge went into designing GPT-4", and there's a time gap between GPT-3 (2020) and GPT-4 (2022) where it seems they spent a lot of time probably trying to understand it, among other things.
It seems the way values are looked up via query/key and added must constrain representations quite a bit, and comparing internal activations for closely related types of input might be one way to start to understand what's going on.
A high level understanding of what the model has learnt may be the last thing to fall, but understanding the internal representations would go a long way towards that.
Re: Show HN: Visual intuitive explanations of LLM concepts (LLM University)
#20> We've just published a lot of original, visual, and intuitive explanations of concepts to introduce people to large language models. Kinda frustrating that the main link dumps me onto what reads like a university syllabus, and nothing original, visual, or intuitive. If I click through the sections in order, there are 5 "preamble" sections describing logistical and other meta-information about the course. All text.…
The landing page is technically the course overview. I'd love to hear what you think would've made it more engaging for you. We can probably pull up some of the visuals to it as a preview. Let me see what we can do on that front.
Like the grandparent comment mentioned, the pitch is "visual, intuitive explanations", but I don't see that on the landing page. I'm looking for a way to get to the start of your content, but the top and left hand menus don't help and are, if anything, confusing until I realize that I'm now inside of a larger set of documentation unrelated to the course.
Below the fold we see a "Let's get started!", but the link I see, Structure of the Course" doesn't sound like getting started. It sounds like more front matter. From the nav menu I see that after that I still won't get to the content, but instead a page about the instructors. Do I really need to read blurbs of the instructors before I get to the meat of the course?
It just feels like too much wrapping paper and packaging to get to the good stuff--and it really does seem like good stuff! And I think the way that you've embedded this course into the rest of your documentation prevents you from presenting it in a structure that is more familiar and easy to navigate (e.g. an 'About' link at the top that talks about the instructors and Cohere).
It might be frustrating to put a lot of time and effort into high quality materials, only for people to not want to spend a few minutes looking around, but from the audience perspective, there's a sea of LLM-related content out there. I want to quickly determine if this is worth adding to my already-too-long list of LLM related bookmarks of things I want to read.