Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

medium.com

Post reply on HN
Print view View original

41–50 of 154 posts

Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

#41

Post by wild_egg »

cess11 wrote: post #23
strangescript wrote: post #10

Earlier quoted context omitted.

If your model is getting confused by python, its a bad model. Python is routinely the best language for all major models.

I don't know what counts as a major model. Relevant to this, I've dabbled with Gemma, Qwen, Mistral, Llama, Granite and Phi models, mostly 3-14b varieties but also some larger ones on CPU on a machine that has 64 GB RAM.

I think the issue there is those smaller versions of those models. I regularly use Gemma3 and Qwen3 for programming without issue but in the 27b-32b range. Going smaller than that generally yields garbage.

Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

#43

Post by Timwi »

gizmo686 wrote: post #11
trjordan wrote: post #5

Seems like it could easily be training data set size as well. I'd love to see some quantification of errors in q/kdb+ (or hebrew) vs. languages of similar size that are left-to-right.

Hebrew is still written sequentially in Unicode. The right-to-left aspect there is simply about how the characters get displayed. On mixed documents, there is U+200E and U+200F to change the text direction mid stream. From the perspective of a LLM learning from Unicode, this would appear as a delimeter that needs to be inserted on language direction boundaries; but everything else should work the same.

I know I'm being pedantic, but I just want to point out that even U+200E/U+200F are generally not needed. If you put a Hebrew word in the middle of an English sentence, it displays correctly all by itself. This is due to the Unicode bidirectional algorithm, which defines a super sensible default behavior. You only need the RTL control characters in weird circumstances, perhaps ones involving punctuation marks or unusual uses of special characters.

Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

#44

Post by Timwi »

IshKebab wrote: post #40
anonzzzies wrote: post #16

Earlier quoted context omitted.

Once you get used to it, traditional ways look tedious and annoying to me. I think the power is in 'once you get used to it'. That will keep out most people. See python llm implementations vs k ones as a novice and you will see verbose unreadable stuff vs line noise. When you learn the math you see verbose code where the verbose code adds nothing at all vs exactly what you would write if you could.

Tedious and annoying for one-off commands maybe. It's like regex. Pretty compelling if you're writing a one-off pattern, you get immediate feedback and then you throw it away. But it's not a good idea to use regexes in code that you're going to use long term. It's justifiable for simple regexes, and many people go against this advice, but really for anything remotely complex regexes become totally unreadable and extr…

What would you propose as an alternative to regexes that provides the same functionality without the unreadable syntax?

I wrote something like that in C# once [0] but I'm not getting the impression that there's a lot of demand for that kind of thing.

[0] https://github.com/Timwi/Generex

Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

#45

Post by fwip »

electroly wrote: post #27

I always thought APL was written in the wrong direction. It writes like a concatenative language that's backwards--you tack things onto the front. NumPy fixes it by making the verbs all dotted function calls, effectively mirroring the order. e.g. in APL you write "10 10 ⍴ ⍳100" but in NumPy you write "np.arange(1, 101).reshape(10, 10)". Even if you don't know either language, you can tell that the APL version is the…

With complicated formulas, it often makes more sense and can give more guidance by first talking about the last operations to be applied. This seems to match the LLM structure, by starting by describing what we want, and then filling in the more specialized holes as we get to them. "Top-down" design vs "bottom-up".

Your insight about APL being reverse-concatenative is very cool.

Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

#46

Post by fwip »

Timwi wrote: post #44
IshKebab wrote: post #40

Earlier quoted context omitted.

Tedious and annoying for one-off commands maybe. It's like regex. Pretty compelling if you're writing a one-off pattern, you get immediate feedback and then you throw it away. But it's not a good idea to use regexes in code that you're going to use long term. It's justifiable for simple regexes, and many people go against this advice, but really for anything remotely complex regexes become totally unreadable and extr…

What would you propose as an alternative to regexes that provides the same functionality without the unreadable syntax? I wrote something like that in C# once [0] but I'm not getting the impression that there's a lot of demand for that kind of thing. [0] https://github.com/Timwi/Generex

Chiming in - I've found the Pattern type in Unison[0] to be very nice to use. When you're just using the built-in patterns, it is similar to verbose regex. The real power of them is that it's easy to define, name, and re-use sub-patterns. I think it's similar to parser combinators in this way, like nom from rust[1].

[0] https://share.unison-lang.org/@unison/website/code/main/late...

[1] https://docs.rs/nom/latest/nom/

Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

#47

Post by altruios »

hnuser123456 wrote: post #37
clord wrote: post #31

There is something deep in this observation. When I reflect on how I write code, sometimes it’s backwards. Sometimes I start with the data and work back through to the outer functions, unnesting as I go. Sometimes I start with the final return and work back to the inputs. I notice sometimes LLMs should work this way, but can’t. So they end up rewriting from the start. Makes me wonder if future llms will be composing…

Yes, there are already diffusion language models, which start with paragraphs of gibberish and evolve them into a refined response as a whole unit.

Right, but that smoothly(ish) resolves all at the same time. That might be sufficient, but it isn't actually replicating the thought process described above. That non-linear thinking is different than diffuse thinking. Resolving in a web around a foundation seems like it would be useful for coding (and other structured thinking, in general).

Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

#48

Post by leprechaun1066 »

It's not because of the left of right evaluation. If the difference was that simple, most humans, let alone LLMs, wouldn't struggle with picking up q when they come from the common languages.

Usually when someone solves problems with q, they don't use the way one would for Python/Java/C/C++/C#/etc.

This is probably a poor example, if I asked someone to write a function to create an nxn identity matrix for a given number the non-q solution would probably involve some kind of nested loop that checks if i==j and assigns 1, otherwise assigns 0.

In q you'd still check equivalence, but instead of looping, you generate a list of numbers as long as the given dimension and then compare each item of the list to itself:

  {x=/:x:til x}3
An LLM that's been so heavily trained on an imperative style will likely struggle to solve similar (and often more complex) problems in a standard q manner.

Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

#49

Post by wat10000 »

leprechaun1066 wrote: post #48

It's not because of the left of right evaluation. If the difference was that simple, most humans, let alone LLMs, wouldn't struggle with picking up q when they come from the common languages. Usually when someone solves problems with q, they don't use the way one would for Python/Java/C/C++/C#/etc. This is probably a poor example, if I asked someone to write a function to create an nxn identity matrix for a given num…

A human can deal with right-to-left evaluation by moving the cursor around to write in that direction. An LLM can’t do that on its own. A human given an editor that can only append would struggle too.

Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

#50

Post by cenamus »

FeepingCreature wrote: post #42

Another example of this is Claude placing unnecessary imports when writing Python, because it's hedge-importing modules that it suspects it might need later.

Is it hedging or did the training data just have lots of unecessary imports?
Post reply on HN