Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

medium.com

Post reply on HN
Print view View original

31–40 of 56 posts

Re: Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

#31

Post by geocar »

afc wrote: post #21
chmod775 wrote: post #20

LLMs were created to use the same interface as humans (language/code). Asking humans to change for the sake of LLMs is an utterly indefensible position. If humans want terse code, your LLM better cope or go home.

Disagree. If some small adjustments to your workflow or expectations enable you to use LLMs to produce good, working, high-quality code much faster than you could otherwise, at some point you should absolutely welcome this, not stubbornly refuse change.

I think there's a mighty big assumption in there.

I see no reason to believe LLMs can write working let alone good or high-quality code, nor that the adjustments to my workflow or expectations will be small. But sure, if such a thing happened, I would probably welcome it.

Meanwhile, there are people who write good and high-quality working code faster than me, and they all write as much as possible on one line with the most bare-bones of text editors, so I will continue to learn from them, rather than the people who say LLMs are helping them. Maybe you should reconsider.

Re: Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

#32

Post by dzaima »

torginus wrote: post #30
gabiteodoru wrote: post #4

Earlier quoted context omitted.

I agree with you, though in the q world people tend to take it to the extreme, like packing a whole function into a single line rather than a single screen. Here's a ticker plant standard script from KX themselves; I personally find this density makes it harder to read, and when reading it I put it into my text editor and split semicolon-separated statements onto different lines: https://github.com/KxSystems/kdb-tick…

I've been dabbling in programming language design as of late, when trying to decide if including feature 'X' makes sense or not, with readability being the main focus I realized some old wisdom: 1 line should do 1 thing - that's something C has established, and I realized that putting conceptually different things on the same line destroys readability very quickly. For example if you write some code to check if the c…

I'd perhaps generalize that to "it's useful to have visual grouping correlate with semantic grouping"; applies to separating words with spaces, sentences with punctuation, paragraphs with newlines, lines of code, UIs, and much more.

An important question for this also is what qualifies for "single thing"; you can look at a "for (int i = 0; i < n; i++) sum += arr[i]" as like 5 or more things (separate parts), 2 things (loop, body), or just one thing ("sum"). What array languages enable is squeezing quite a bit into the space of "single thing" (though I personally don't go as far as the popular k examples of ultra-terseness).

Re: Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

#33

Post by cwmoore »

quotemstr wrote: post #14
Veedrac wrote: post #13

> Let’s start with an example: (2#x)#1,x#0 is code from the official q phrasebook for constructing an x-by-x identity matrix. Is this... just to be clever? Why not (!x)=/:!x aka. the identity matrix is defined as having ones on the diagonal? Bonus points AI will understand the code better.

The vibe I get from q/kdb in general is that its concision has passed the point of increasing clarity through brevity and is now about some kind of weird hazing or machismo thing. I've never seen even numpy's verbosity be an actual impediment to understanding an algorithm, so we're left speculation about social and psychological explanations for why someone would write (2#x)#1,x#0 and think it beautiful. Some brief n…

It’s…remarkable. Is this meant to be entered on a payphone? I will never be cool enough.

Re: Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

#34

Post by nlitened »

dapperdrake wrote: post #10
sanjayjc wrote: post #3

> I think the aesthetic preference for terseness should give way to the preference for LLM accuracy, which may mean more verbose code From what I understand, the terseness of array languages (Q builds on K) serves a practical purpose: all the code is visible at once, without the reader having to scroll or jump around. When reviewing an LLM's output, this is a quality I'd appreciate.

Perl and line noise also share these properties. Don’t particularly want to read straight binary zip files in a hex editor, though. Human language has roughly, say, 36% encoding redundancy on purpose . (Or by Darwinian selection so ruthless we might as well call it "purpose".)

> Human language has roughly, say, 36% encoding redundancy on purpose.

The purpose is being understandable by a person of average intellect and no specialized training. Compare with redundancy in math notation, for example.

Re: Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

#35

Post by orbifold »

neprotivo wrote: post #12

This approach of solving a problem by building a low-perplexity path towards the solution reminds me of Grothendieck's approach towards solving complex mathematical problems - you gradually build a theory which eventually makes the problem obvious. https://ncatlab.org/nlab/show/The+Rising+Sea

what is striking to me is how far reasoning by analogy and generalization can get you. some of the deepest theorems are about relating disparate things by analogy.

Re: Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

#36

Post by [deleted] »

mikkupikku wrote: post #27
chmod775 wrote: post #24

Earlier quoted context omitted.

Chances are hell is going to freeze over before people start writing verbose q code. Q being less verbose than alternatives is the whole point. Nobody is feeling any pressure to bend over backwards to accommodate the guy who struggles to get by when his LLM can't explain a piece of code to him. To use your nailgun analogy as an example: Waddling in with your LLM and demanding the q community change is like walking in…

I'm pretty sure the time will soon come when nobody is trying to accommodate the personal tastes and preferences of developers anymore; languages and tools will be chosen based on how well LLMs work with them, and the way the LLMs are used with those will be determined again by the traits of the tool, not the preferences of the user. Management won't be in the mood to humor devs who are stuck in their old mindset of…

[deleted]

Re: Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

#37

Post by Qem »

nlitened wrote: post #34
dapperdrake wrote: post #10

Earlier quoted context omitted.

Perl and line noise also share these properties. Don’t particularly want to read straight binary zip files in a hex editor, though. Human language has roughly, say, 36% encoding redundancy on purpose . (Or by Darwinian selection so ruthless we might as well call it "purpose".)

> Human language has roughly, say, 36% encoding redundancy on purpose. The purpose is being understandable by a person of average intellect and no specialized training. Compare with redundancy in math notation, for example.

> The purpose is being understandable by a person of average intellect and no specialized training.

The purpose probably is keeping human speech understandable through the often noise-filled channel of ambient sound. Human speech with no redundancy would have a hard time fighting the noise floor.

Re: Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

#38

Post by Majromax »

> When I(short proof)=I(long proof), per-token average surprisal must be lower for the long proof than for the short proof. But since surprisal for a single token is simply -log P, that would mean that, on average, the shorter proof is made out of less probable tokens.

This assertion is intuitive, but it isn't true. Per-token entropy of the long proof can be larger if the long proof is not minimal.

For example, consider the "proof" of "list the natural numbers from 1 to 3, newline-delimited." The 'short proof' is:

"1\n2\n3\n" (Newlines escaped because of HN formatting)

Now, consider the alternative instruction to give a "long proof", "list the natural numbers from 1 to 3, newline-delimited using # for comments. Think carefully, and be verbose." Trying this just now with Gemini 2.5-pro (Google AI Studio) gives me:

"# This is the first and smallest natural number in the requested sequence.\n 1

# Following the first, this is the second natural number, representing the concept of a pair.\n 2

# This is the third natural number, concluding the specified range from 1 to 3.\n 3"

We don't have access to Gemini's per-token logits, but repeating the prompt gives different comments so we can conclude that there is 'information' in the irrelevant commentary.

The author's point regains its truth, however, if we consider the space of all possible long proofs. The trivial 'long' proof has higher perplexity than the short proof, but that's because there are so many more possible long proofs of approximately equal value. The shortest possible proof is a sharp minimum, but and longer proofs are shallower and 'easier'.

The author also misses a trick with:

> Prompted with “Respond only with code: How do you increment i by 1 in Python?”, I compared the two valid outputs: i += 1 has a perplexity of approximately 38.68, while i = i + 1 has a perplexity of approximately 10.88.

… in that they ignore the equally-valid 'i = 1 + i'.

Re: Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

#39

Post by segmondy »

First pass on my local deepseekv3.1-Terminus at Q4 answered it correctly. if anything, i think LLMs should write terse code, Q/J/APL/Forth/Prolog/Lisp, tokens is precious. It's insane to waste precious tokens generating Java, javascript and other overly verbose code...

https://pastebin.com/VVT74Rp9

Re: Don't Force Your LLM to Write Terse [Q/Kdb] Code: An Information Theory Argument

#40

Post by torginus »

dzaima wrote: post #32
torginus wrote: post #30

Earlier quoted context omitted.

I've been dabbling in programming language design as of late, when trying to decide if including feature 'X' makes sense or not, with readability being the main focus I realized some old wisdom: 1 line should do 1 thing - that's something C has established, and I realized that putting conceptually different things on the same line destroys readability very quickly. For example if you write some code to check if the c…

I'd perhaps generalize that to "it's useful to have visual grouping correlate with semantic grouping"; applies to separating words with spaces, sentences with punctuation, paragraphs with newlines, lines of code, UIs, and much more. An important question for this also is what qualifies for "single thing"; you can look at a "for (int i = 0; i < n; i++) sum += arr[i]" as like 5 or more things (separate parts), 2 things…

I'd say I'd decompose this into 2 'units'

- iterate through the array arr,

- sum its elements

of course, since C is a low level language, implementation details seep in there but those are the 2 major logical concepts that mentally can be quickly parsed as a single unit each.

Some languages allow reductions on collections, so the actual iteration becomes logically part of that single unit, so it might count as one, but I'd say that's the absolute upper bound of complexity you should stuff into a singlwe line.

The goal here is to make every statement conform to a template so that your brain quickly recognizes it and you can move on, rather than you having to break it apart bit by bit to figure out the goal.

Post reply on HN