The non-linear scaling on the slider is an excellent UX.
I noticed this as well, glad I'm not the only one
How fast is N tokens per second really?
71–80 of 105 posts
Re: How fast is N tokens per second really?
#72Isn't the thinking part the part that burns the tokens? You're just outputting tokens.
Re: How fast is N tokens per second really?
#73This is nice. Please tell you agent to make the site "light" or give an option for light/dark.
Re: How fast is N tokens per second really?
#74Isn't the thinking part the part that burns the tokens? You're just outputting tokens.
Totally depends, but I think this is mostly just an illustration of overall speed, regardless of the content.
Re: How fast is N tokens per second really?
#75It's interesting how even 5 tok/s is still much faster than you'd typically type, but feels glacially slow for an agent. On the other hand, I've been using Mimo and Minimax a lot recently. They routinely reach 100-150 tokens per second and that feels too fast , to the point where it's hard to keep up with what it's actually doing. Great for subagents though.
isn't 5 tok/s like 100wpm? Pretty standard typing speed.
You also would need to compare token generation not with the actual output, but with the thoughts and deleted and edited parts.
Re: How fast is N tokens per second really?
#76The visualiser seems to be quite naive with what it defines as a token. I don't think a token is an entire word as often as the demo shows, and when it gets to the `def estimate_tokens` method, the entire `# Rough heuristic: ~1 token per 4 chars of English` comment is printed all at once as one token, which is certainly not accurate.
This is not a realistic replay of what a common LLM might actually print out - it's entirely fabricated. But for the purpose of estimating the feel of tokens per second, I suppose it's good enough.
Re: How fast is N tokens per second really?
#77The visualiser seems to be quite naive with what it defines as a token. I don't think a token is an entire word as often as the demo shows, and when it gets to the `def estimate_tokens` method, the entire `# Rough heuristic: ~1 token per 4 chars of English` comment is printed all at once as one token, which is certainly not accurate. This is not a realistic replay of what a common LLM might actually print out - it's…
I built something similar awhile back [1] and used OpenAI’s tokenizer playground [2] to recalculate tokens on a giant block of lorem ipsum text. I feel like this gives a much more accurate representation.
Re: How fast is N tokens per second really?
#78[flagged]
Re: How fast is N tokens per second really?
#79It's interesting how even 5 tok/s is still much faster than you'd typically type, but feels glacially slow for an agent. On the other hand, I've been using Mimo and Minimax a lot recently. They routinely reach 100-150 tokens per second and that feels too fast , to the point where it's hard to keep up with what it's actually doing. Great for subagents though.
isn't 5 tok/s like 100wpm? Pretty standard typing speed. You also would need to compare token generation not with the actual output, but with the thoughts and deleted and edited parts.
It's about 240 wpm on text.
Re: How fast is N tokens per second really?
#80It's interesting how even 5 tok/s is still much faster than you'd typically type, but feels glacially slow for an agent. On the other hand, I've been using Mimo and Minimax a lot recently. They routinely reach 100-150 tokens per second and that feels too fast , to the point where it's hard to keep up with what it's actually doing. Great for subagents though.
isn't 5 tok/s like 100wpm? Pretty standard typing speed. You also would need to compare token generation not with the actual output, but with the thoughts and deleted and edited parts.
100wpm is well above what the average person types at, which is estimated at about 40wpm.
100wpm might still bit a bit high even for your average programmer.