Zed's Pricing Has Changed: LLM Usage Is Now Token-Based

Well ripgrep can be just a backend right. Just like your terminal is how you invoke it, you could have your editor do that.

In fact, I believe that is what vscode uses.

Apparently there’s been a neglected issue about bringing it to Zed. https://github.com/zed-industries/zed/issues/4560

Still very poor. Not sure why they have so much trouble catching up. I would have expected them to by now. Cursor is still the best by far, followed by Windsurf (free by the way).

I really do like the Claude code integration in zed. Conceptually it’s pretty well done. Provides some better visualization over running Claude code in the terminal and really aligns with the close supervisor style I like to work with. I really like the follow feature that brings the editor along with the agent.

My complaints:

No Claude code hooks support at the time of this writing. As someone who leverages this somewhat heavily this is why I don’t really use it all the time. I actually find it to be somewhat of a feature at times because I can simply run the thing through Zed if I want to temporarily run with no hooks.

Performance is noticeably degraded presumably because of the “ACP” protocol they invented. I usually work either directly in Claude terminal window using its edit tools or using repoprompt’s mcp editor tools and both are noticeably faster than running in zed.

What seems to be memory leaks in the agent window causes sluggish performance, especially when scrolling. It’s not bad enough to make it unusable, but for an editor whose main advertisement is speed it feels particularly painful.

Tokens correspond more to words in text land. The cat-feline etc connection happens when you train the model and not really by the tokenisation algorithm, which only sees text and not concepts. Byte pair encoding and SentencePiece (the two main tokenisation algorithms used by all LLMs) are mostly leetcode-medium-level algorithms. You can check it out and gain a permanent intuition, especially BPE.

For images, you take patches of the image (say 16x16 patches[1]), and then directly pass it into the FFN+transformer machinery[2]. As such, there is no vocabulary of tokens for images[3]. So, the billing happens per image patch. i.e, for large images, your cost will go up[2] Since it will have more px*py patches.

[1] x 3, due to RGB

[2] Upto a point, it gets downsamples to lower quality beyond a certain res. The downsampling happens in many ways... Qwen-VL uses a CNN, GPT iirc stuffs a downsampler after the embedding layer... As well as before. Anyways, they usually just take some average reduction by that downsampler and cut your billed tokens by that much in all these cases. OpenAIs bin-based billing is like this.

[3] Dall-E from way back when did have a discrete set of tokens and it mapped all patches of all images in the world to one from that, IIRC.

OAI/Anthropic/Big AI players today never ever need to become profitable... The same way roads never need to become profitable by themselves. This per token charge is like tolls, it has nothing to do with the costs.

AI, just like cloud was before this, is being treated as "infrastructure". The reason people invest in roads is not to make money from the road itself, but rather recuper all the losses from the new extremely profitable businesses that will come on top of this infrastructure. To the stock ticker, this looks like a boom+bust cycle. But really, it's a bust+boom cycle as far as the investors are concerned. Saudi Arabia don't give a hoot about transformers, but they know if they invest their large amounts of capital in the new infrastructure - huge models in expensive datacenters using cities worth of power, they can invest in the many new hyper profitable businesses on top. Also helps that it locks in demand for oil, I guess :P

A similar example for cloud would be the gajillion cloud SaaS, DevOps, cloud security whatever businesses that only exist today because the whole cloud infra segment ran unprofitable for a long time and created the infrastructure for all this.

These new businesses will not be multi hundred-billion dollar businesses, no. They will all be million dollar businesses. But you'll have a million such businesses. Everyone that's stuffing money into AI today is hoping that there will be a huge product layer [1] on top of this new infrastructure once it's all consolidated at a few major players allowing "infra" costs to drop considerably, which they can milk.

The stock of all these companies going high is just wall st making it easy for people to buy debt from this infrastructure. If they're indisciplined, they will co-mingle this debt with normal people's daily lives debt and sell it as one, and everything gets affected badly. If they're not, then the stock market will complain, but normal folks will be kind of OK.

[1] well, everyone has this same idea, meaning there also a lot of short term investors trying to make gains while openai/nvidia/etc are on their way up, sort of like greater fool investing, but let's ignore them for the purposes of this argument.

[Note] of course, whether cloud/ai/whatever are actually useful infra that deserves to be force-created is up for debate.. many disagree, not me though.