Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

medium.com

Post reply on HN
Print view View original

41–50 of 56 posts

Re: Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

#41

Post by socketcluster »

I was kind of taken aback by the author's definition of 'terse'. I was expecting a discussion about architecture not about syntax aesthetics.

Personally I don't like short variable names, short function names or overly fancy syntactical shortcuts... But I'm obsessed with minimizing the amount of logic.

I want my codebases to be as minimalist as possible. When I'm coding, I'm discovering the correct lines, not inventing them.

This is why I like using Claude Code on my personal projects. When Claude sees my code, it unlocks a part of its exclusive, ultra-elite, zero-bs code training set. Few can tap into this elite set. Your codebase is the key which can unlock ASI-like performance from your LLMs.

My friend was telling me about all the prompt engineering tricks he knows... And in a typical midwit meme moment; I told him, dude, relax, my codebase basically writes itself now. The coding experience is almost bug free. It just works first time.

I told my friend I'd consider letting him code on my codebase if he uses an LLM... And he took me up on the offer... I merged his first major PR directly without comment. It seems even his mediocre Co-pilot was capable of getting his PR to the standard.

Re: Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

#42

Post by ahussain »

socketcluster wrote: post #41

I was kind of taken aback by the author's definition of 'terse'. I was expecting a discussion about architecture not about syntax aesthetics. Personally I don't like short variable names, short function names or overly fancy syntactical shortcuts... But I'm obsessed with minimizing the amount of logic. I want my codebases to be as minimalist as possible. When I'm coding, I'm discovering the correct lines, not inventi…

I'd bet a lot of people are trying to optimize their codebases for LLMs. I'd be interested to see some examples of your ASI-unlocking codebase in action!

Re: Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

#43

Post by Analemma_ »

quotemstr wrote: post #17
haolez wrote: post #9

Earlier quoted context omitted.

I don't think that's the biggest problem. I think it's the tokenizer: it probably does a poor job with array languages.

Perhaps for array languages LLMs would do a better job running on a q/APL parse tree (produced using tree-sitter?) with the output compressed into the traditional array-language line noise just before display, outside the agentic workflow.

This is the dream, but it keeps crashing and sinking against reality. It seems intuitive that running language models on the AST should work better than running them on the source code, but as far as I'm aware every attempt to do this has resulted in much worse performance. There's so much more training data available as source code, and working in source code form gives you access to so much more outside context (comments, documentation, Stack Overflow posts), that it more than cancels out the disadvantages.

Re: Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

#44

Post by gabiteodoru »

Majromax wrote: post #38

> When I(short proof)=I(long proof), per-token average surprisal must be lower for the long proof than for the short proof. But since surprisal for a single token is simply -log P, that would mean that, on average, the shorter proof is made out of less probable tokens. This assertion is intuitive, but it isn't true. Per-token entropy of the long proof can be larger if the long proof is not minimal. For example, consi…

Thanks so much for this challenge! I just ran the experiment with i = 1 + i and you're absolutely right - it breaks my theoretical framework (same semantic information, but much higher perplexity).

While setting this up, I realized I hadn't used chat templates in my original measurements (rookie mistake with an Instruct model!). Re-running with proper methodology completely flips the results - the terse version actually wins.

I'll add a correction note to the article once AWS/Medium comes back online, and will write a proper follow-up with all the corrected experiments. Your comment literally made the research better - thank you!

Re: Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

#45

Post by gabiteodoru »

Author here. Majromax challenged me to test `i = 1 + i`, which broke my theoretical framework. While setting up that experiment, I realized I hadn't used chat templates in my original measurements (rookie mistake with an Instruct model!).

Re-running with proper methodology completely flips the results - the terse version actually wins. I'll add a correction note to the article once AWS/Medium comes back online and will write a follow-up with the corrected experiments.

This is open science working as intended - community scrutiny improves the work. Thank you all for the engagement, and especially to Majromax for the challenge that led to discovering this!

Re: Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

#46

Post by gabiteodoru »

segmondy wrote: post #39

First pass on my local deepseekv3.1-Terminus at Q4 answered it correctly. if anything, i think LLMs should write terse code, Q/J/APL/Forth/Prolog/Lisp, tokens is precious. It's insane to waste precious tokens generating Java, javascript and other overly verbose code... https://pastebin.com/VVT74Rp9

It did go back on itself 3 times, no? "Actually, let’s trace for x=3:" (it had just computed for x=3 the first time); then "Better to check actual q output:" -- did it actually run it in a q session, or just pretended? And another one "That doesn’t seem to align. Let’s do it step by step:"

Re: Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

#47

Post by gabiteodoru »

Analemma_ wrote: post #43
quotemstr wrote: post #17

Earlier quoted context omitted.

Perhaps for array languages LLMs would do a better job running on a q/APL parse tree (produced using tree-sitter?) with the output compressed into the traditional array-language line noise just before display, outside the agentic workflow.

This is the dream, but it keeps crashing and sinking against reality. It seems intuitive that running language models on the AST should work better than running them on the source code, but as far as I'm aware every attempt to do this has resulted in much worse performance. There's so much more training data available as source code, and working in source code form gives you access to so much more outside context (co…

Perhaps if we also trained them on natural language ASTs at the same time when asking the questions? :)

Re: Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

#48

Post by gabiteodoru »

rak1507 wrote: post #16
gabiteodoru wrote: post #4

Earlier quoted context omitted.

I agree with you, though in the q world people tend to take it to the extreme, like packing a whole function into a single line rather than a single screen. Here's a ticker plant standard script from KX themselves; I personally find this density makes it harder to read, and when reading it I put it into my text editor and split semicolon-separated statements onto different lines: https://github.com/KxSystems/kdb-tick…

There's a difference between one line and short/terse/elegant. {m:(x,x)#til x*x; r:til[x]-x div 2; 2(flip r rotate')/m} generates magic squares of odd size, and the method is much clearer. This isn't even golfed as the variables have been left.

Oh nice solution, thanks for sharing!

Re: Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

#49

Post by daxfohl »

I was wondering recently, is fine tuning an effective way to make this the default? If so, does fine tuning this behavior on one language have a carry-over effect on other languages (maybe even non-programming language?), or is the effect localized to the language of the fine-tuning dataset?
Post reply on HN