Live data from Hacker News

Show HN: Mcptoon – Token-efficient MCP CLI client

github.com

51–52 of 52 posts

Re: Show HN: Mcptoon – Token-efficient MCP CLI client

#52
post #13

Why is it replacing true/false with T/F? true/false is already 1 token in all tokenizer I've seen. Even worse is replacing null with ∅. ∅ is a special unicode symbol that takes up 2 tokens compared to the 1 token for null...

Agreed. This is what happens when you confuse token counts with byte counts.

A trivial test through tiktoken [1] (though technically you really have to match the tokenizer to the specific LLM) would have shown them that ∅ was a poor choice.

Even from the perspective of learned training data, you can probably just intuit that from a frequency standpoint alone the empty-set symbol ∅ can’t possibly have appeared that often outside of things like set theory and logic.

[1] - https://github.com/openai/tiktoken

Post reply on HN