Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Which programming languages are most token-efficient?

martinalderson.com

Post reply on HN
Print view View original

81–90 of 96 posts

Re: Which programming languages are most token-efficient?

#81

Post by solomonb »

torginus wrote: post #67
tikhonj wrote: post #54

Earlier quoted context omitted.

From what I've heard—and in my own very limited experiments—LLMs are much better at less popular languages than I would have expected. I've had good results with OCaml, and I've talked to people who've had good results with Haskell and even Unison. I've also seen multiple startups that have had some pretty impressive performance with Lean and Rocq. My current theory is that as long as the LLM has sufficiently good ba…

You might be right, but I think you must take into account that (I think) you're not super familiar with these languages as well, so you might not notice all the warts a programmer with a lot of experience in these langs would, and overrate the skill of the LLM. Nowadays, I write C# and TS at work, and it's absolutely crazy how much better the LLM is at TS, with almost all code being decent the first try, but with C#…

I write Haskell professionally and I can tell you that Opus 4.5 can do a great writing industrial Haskell code consistent with an existing code base.

I don't think it is capable of writing galaxy brain Haskell libraries, it absolutely missed the forest for the trees, but if you have an existing code base with consistent patterns it can emulate then it can do a surprisingly good job.

Here is an example side project I have done extremely heavily with Claude: https://github.com/solomon-b/kpbj.fm

I built a library (without Claude) that wraps in an opinionated way Servant and a handful of other common libraries used to build Haskell web-apps and then have let Claude use that to build this site. There is absolutely some hairy code and I have done a ton of manual refactors on what Claude produces, but claude has been highly effective for me here.

Re: Which programming languages are most token-efficient?

#82

Post by HarHarVeryFunny »

CuriouslyC wrote: post #80
HarHarVeryFunny wrote: post #77

Earlier quoted context omitted.

This scribe tool seems to offer somewhat similar functionality to a Language Server and/or Cursor's chunked vector index. The idea would seem to be to give instructions to your agent (Claude Code, etc) to use this tool to discover the chunks of code (not entire source files) it needs to look at to modify a particular function. You could put these instructions on how/when to use scribe someplace like .claude/rules/scr…

Scribe grew out of fixing all the problems with code bundlers like Repomix. The covering set feature is the thing that clearly sets it apart, the performance difference is extreme; up to 98% token use reduction on SWE-bench tasks. I lead with it because it's the place where I'm far ahead of other tools, people won't adopt something because it's slightly better, scribe is a step change.

It would be useful if you had some documentation (or maybe you do?) as to how you are integrating scribe with Claude Code etc (same for Gemini CLI, or different?), and what your work flow looks like if necessary. Do you have something like scribe.md so that Claude Code is automatically invoking scribe when appropriate, or are you invoking scribe manually?

Has anyone tried scribe for larger scale projects, and green field development?

Re: Which programming languages are most token-efficient?

#84

Post by jwr »

kaliszad wrote: post #78
jwr wrote: post #63

I program mostly in Clojure and I expected it to be near the top, as it tends to be very concise and expressive (qualities I really admire). I am getting excellent results from Claude Code (Opus 4.5), and I think this might be one of the reasons. I'm using Claude with a large code base and the token-efficiency of Clojure might help with fitting more into the context window.

I also program a lot in Clojure/Script. Do you also consider thinking token and the number of iterations in the token efficiency?

I don't think thinking tokens are affected, as LLMs "think" mostly in plain language, with occasional code snippets.

Re: Which programming languages are most token-efficient?

#85

Post by zcw100 »

jaggederest wrote: post #42
make3 wrote: post #32

Earlier quoted context omitted.

LLMs in their current form rely heavily on the vast amount of human data that's available, to learn from it as a first step (the second step is RL). We're not building a language for LLMs just yet.

> We're not building a language for LLMs just yet. Working on it, actually! I think it's a really interesting problem space - being efficient on tokens, readable by humans for review, strongly typed and static for reasoning purposes, and having extremely regular syntax. One of the biggest issues with symbols is that, to a human, matching parentheses is relatively easy, but the models struggle with it. I expect a lang…

Working on it too. It's actually more like a meta language that is very token efficient.

Re: Which programming languages are most token-efficient?

#86

Post by janalsncm »

muyuu wrote: post #20
janalsncm wrote: post #19

This is kind of just a measurement of how representative a language is in the distribution of the tokenizer training. You could have a single token equal to “public static void main”.

You could, but you wouldn't when those keywords can all change in equivalent contexts.

The BPE or wordpiece tokenization algorithm will greedily take the longest valid token prefix. So if your text starts with “public static void main” it will try to find the longest token which matches that prefix. Even if “public” is a token, it will prefer to tokenize “public static” together.

Re: Which programming languages are most token-efficient?

#87

Post by make3 »

energy123 wrote: post #40
make3 wrote: post #32

Earlier quoted context omitted.

LLMs in their current form rely heavily on the vast amount of human data that's available, to learn from it as a first step (the second step is RL). We're not building a language for LLMs just yet.

It's worth asking why we haven't had the AlphaZero moment for general learning yet, where no human data is needed.

That's easy, AlphaZero had a perfect simulator of the world it existed in (chess, super easy), so it was insanely easy to run simulations of that world ad infinitum, and learn from it.

It's simply not the case for the real world, you can't simulate the world perfectly and see what happens when you do things.

Re: Which programming languages are most token-efficient?

#88

Post by make3 »

jaggederest wrote: post #42
make3 wrote: post #32

Earlier quoted context omitted.

LLMs in their current form rely heavily on the vast amount of human data that's available, to learn from it as a first step (the second step is RL). We're not building a language for LLMs just yet.

> We're not building a language for LLMs just yet. Working on it, actually! I think it's a really interesting problem space - being efficient on tokens, readable by humans for review, strongly typed and static for reasoning purposes, and having extremely regular syntax. One of the biggest issues with symbols is that, to a human, matching parentheses is relatively easy, but the models struggle with it. I expect a lang…

There's no way there's enough data for you to get a model that is anywhere as strong as mainstream languages.

If your model is struggling with parentheses, that means it's not even the level of GPT-3 for a mainstream language.

It's not completely impossible with in-context learning I guess, but it will still be much weaker than .. eg all of GitHub and more on Python

Re: Which programming languages are most token-efficient?

#89

Post by kaliszad »

jwr wrote: post #84
kaliszad wrote: post #78

Earlier quoted context omitted.

I also program a lot in Clojure/Script. Do you also consider thinking token and the number of iterations in the token efficiency?

I don't think thinking tokens are affected, as LLMs "think" mostly in plain language, with occasional code snippets.

I would assume for certain problems LLMs have a solution readily available for JavaScript/ TypeScript or similarly popular languages but not for Clojure/Script. Therefore my thinking was that the process of getting to a workable solution would be longer and more expensive in terms of tokens. I however don't have any relevant data on this so I may just be wrong.

Re: Which programming languages are most token-efficient?

#90

Post by eigenspace »

singularity2001 wrote: post #76

That's why I love Julia so much. Also it's semi-static?

Semantically, julia is a fully dynamic language. But the trick is that it does this by recognizing that being static is a constraint on a dynamic language, so it implements dynamic typing by stitching together islands of statically typed code.
Post reply on HN