Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Microgpt is a GPT you can visualize in the browser

microgpt.boratto.ca

Post reply on HN
Print view View original

1–10 of 27 posts

Show HN: Microgpt is a GPT you can visualize in the browser

#1

Post by b44 »

very much inspired by karpathy's microgpt of the same name. it's (by default) a 4000 param GPT/LLM/NN that learns to generate names. this is sorta an educational tool in that you can visualize the activations as they pass through the network, and click on things to get an explanation of them.

Show HN: Microgpt is a GPT you can visualize in the browser
microgpt.boratto.ca

Re: Show HN: Microgpt is a GPT you can visualize in the browser

#2

Post by kfsone »

Minor nit: In familiarity, you gloss over the fact that it's character rather than token based which might be worth a shout out:

"Microgpt's larger cousins using building blocks called tokens representing one or more letters. That's hard to reason about, but essential for building sentences and conversations.

"So we'll just deal with spelling names using the English alphabet. That gives us 26 tokens, one for each letter."

Re: Show HN: Microgpt is a GPT you can visualize in the browser

#3

Post by b44 »

kfsone wrote: post #2

Minor nit: In familiarity, you gloss over the fact that it's character rather than token based which might be worth a shout out: "Microgpt's larger cousins using building blocks called tokens representing one or more letters. That's hard to reason about, but essential for building sentences and conversations. "So we'll just deal with spelling names using the English alphabet. That gives us 26 tokens, one for each let…

hm. the way i see things, characters are the natural/obvious building blocks and tokenization is just an improvement on that. i do mention chatgpt et al. use tokens in the last q&a dropdown, though

Re: Show HN: Microgpt is a GPT you can visualize in the browser

#6

Post by mips_avatar »

kfsone wrote: post #2

Minor nit: In familiarity, you gloss over the fact that it's character rather than token based which might be worth a shout out: "Microgpt's larger cousins using building blocks called tokens representing one or more letters. That's hard to reason about, but essential for building sentences and conversations. "So we'll just deal with spelling names using the English alphabet. That gives us 26 tokens, one for each let…

Using ascii characters is a simple form of tokenization with less compression

Re: Show HN: Microgpt is a GPT you can visualize in the browser

#8

Post by krackers »

There used to be this page that showed the activations/residual stream from gpt-2 visualized as a black-white image. I remember it being neat how you could slowly see order forming from seemingly random activations as it progressed through the layers.

Can't find it now though (maybe the link rotted?), anyone happen to know what that was?

Post reply on HN