Viewing profile — b44

the untrained model is literally just generating random characters, whereas your examples are at least pronouncable. you can add more layers to get progressively better results.

good catch - i intentionally cap node visualizations at 16 so it doesn't get super long, but the sidebar shouldn't have that

not many. diminishing returns start before 1000 and past that you should just add a second/third layer

hm. the way i see things, characters are the natural/obvious building blocks and tokenization is just an improvement on that. i do mention chatgpt et al. use tokens in the last q&a…

very much inspired by karpathy's microgpt of the same name. it's (by default) a 4000 param GPT/LLM/NN that learns to generate names. this is sorta an educational tool in that you c…