TokenTown: A visual way to understand how LLMs work

Post by JojiDoesCode » Wed, Jul 29, 2026, 3:20 PM UTC

hk__2 wrote: post #9

laurentiurad wrote: post #2

You can paste text and immediately see how it's split into tokens, along with token IDs, counts, and other details. It's useful for understanding why prompts cost what they do, why context limits behave the way they do, and why models sometimes split words in unexpected places. I made it because I kept explaining tokenization to friends and realized there wasn't a simple, interactive tool that focused on learning rat…

> I made it *I asked a LLM to make it

Post by simpledevraf » Wed, Jul 29, 2026, 3:22 PM UTC

Post by GCUMstlyHarmls » Wed, Jul 29, 2026, 3:32 PM UTC

Post by dan_gee » Wed, Jul 29, 2026, 3:35 PM UTC

foltik wrote: post #7

I read through the explanations and they’re all typical hand wavy LLM prose, full of jargon, and random tangential details. If you’re going to publicly present this as a learning tool, it would be great if you spent some actual human brain cycles writing and polishing it by hand, rather than just chucking the first thing that claude spits out over the fence. I can pretty much guess the prompt you used and probably ge…

Post by overtaxed » Wed, Jul 29, 2026, 3:53 PM UTC

foltik wrote: post #7

I read through the explanations and they’re all typical hand wavy LLM prose, full of jargon, and random tangential details. If you’re going to publicly present this as a learning tool, it would be great if you spent some actual human brain cycles writing and polishing it by hand, rather than just chucking the first thing that claude spits out over the fence. I can pretty much guess the prompt you used and probably ge…