Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

medium.com

Post reply on HN
Print view View original

11–20 of 154 posts

Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

#11

Post by gizmo686 »

trjordan wrote: post #5

Seems like it could easily be training data set size as well. I'd love to see some quantification of errors in q/kdb+ (or hebrew) vs. languages of similar size that are left-to-right.

Hebrew is still written sequentially in Unicode. The right-to-left aspect there is simply about how the characters get displayed. On mixed documents, there is U+200E and U+200F to change the text direction mid stream.

From the perspective of a LLM learning from Unicode, this would appear as a delimeter that needs to be inserted on language direction boundaries; but everything else should work the same.

Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

#12

Post by strangescript »

rob_c wrote: post #4

Same reason the same models don't fundamentally understand all languages. They're not trained to. Frankly the design changes to get this to work in training is minimal but this isn't the way English works so expect most of the corporate LLM to struggle because that's where the interest and money is. Give it time until we have true globally multi lingual models for superior context awareness.

A byte tokenized model is naturally 100% multi-lingual in all languages in its data set. There just isn't a lot of reason for teams to spend the extra training time to build that sort of model.

Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

#13

Post by fer »

trjordan wrote: post #5

Seems like it could easily be training data set size as well. I'd love to see some quantification of errors in q/kdb+ (or hebrew) vs. languages of similar size that are left-to-right.

>Seems like it could easily be training data set size as well.

I'm convinced that's the case. On any major LLM I can carpet bomb Java/Python boilerplate without issue. For Rust, at least last time I checked, it comes up with non-existing traits, more frequent hallucinations and general struggle to use the context effectively. In agent mode it turns into a first fight with the compiler, often ending in credit destroying loops.

And don't get me started when using it for Nix...

So not surprised about something with orders of magnitude smaller public corpus.

Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

#14

Post by cubefox »

gizmo686 wrote: post #11
trjordan wrote: post #5

Seems like it could easily be training data set size as well. I'd love to see some quantification of errors in q/kdb+ (or hebrew) vs. languages of similar size that are left-to-right.

Hebrew is still written sequentially in Unicode. The right-to-left aspect there is simply about how the characters get displayed. On mixed documents, there is U+200E and U+200F to change the text direction mid stream. From the perspective of a LLM learning from Unicode, this would appear as a delimeter that needs to be inserted on language direction boundaries; but everything else should work the same.

> Hebrew is still written sequentially

Everything is written sequentially in the sense that the character that is written first can only be followed by the character that is written next. In this sense writing non-sequentially is logically impossible.

Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

#15

Post by dlahoda »

fer wrote: post #13
trjordan wrote: post #5

Seems like it could easily be training data set size as well. I'd love to see some quantification of errors in q/kdb+ (or hebrew) vs. languages of similar size that are left-to-right.

>Seems like it could easily be training data set size as well. I'm convinced that's the case. On any major LLM I can carpet bomb Java/Python boilerplate without issue. For Rust, at least last time I checked, it comes up with non-existing traits, more frequent hallucinations and general struggle to use the context effectively. In agent mode it turns into a first fight with the compiler, often ending in credit destroyi…

i tried gemini, openai, copilot, claude on reasonably big rust project. claude worked well to fix use, clippy, renames, refactorings, ci. i used highest cost claude with custom context per crate. never was able to get it write new code well.

for nix, i is nice template engine to start or search. did not tried big nix changes.

Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

#16

Post by anonzzzies »

yujzgzc wrote: post #6

Humans can't either? I think if this convention had been more usable form of programming, we'd know by now

Once you get used to it, traditional ways look tedious and annoying to me. I think the power is in 'once you get used to it'. That will keep out most people. See python llm implementations vs k ones as a novice and you will see verbose unreadable stuff vs line noise. When you learn the math you see verbose code where the verbose code adds nothing at all vs exactly what you would write if you could.

Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

#17

Post by goatlover »

cubefox wrote: post #14
gizmo686 wrote: post #11

Earlier quoted context omitted.

Hebrew is still written sequentially in Unicode. The right-to-left aspect there is simply about how the characters get displayed. On mixed documents, there is U+200E and U+200F to change the text direction mid stream. From the perspective of a LLM learning from Unicode, this would appear as a delimeter that needs to be inserted on language direction boundaries; but everything else should work the same.

> Hebrew is still written sequentially Everything is written sequentially in the sense that the character that is written first can only be followed by the character that is written next. In this sense writing non-sequentially is logically impossible.

Multiple characters can be written at once, they can also be done in reverse or out of order.

Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

#18

Post by dotancohen »

cubefox wrote: post #14
gizmo686 wrote: post #11

Earlier quoted context omitted.

Hebrew is still written sequentially in Unicode. The right-to-left aspect there is simply about how the characters get displayed. On mixed documents, there is U+200E and U+200F to change the text direction mid stream. From the perspective of a LLM learning from Unicode, this would appear as a delimeter that needs to be inserted on language direction boundaries; but everything else should work the same.

> Hebrew is still written sequentially Everything is written sequentially in the sense that the character that is written first can only be followed by the character that is written next. In this sense writing non-sequentially is logically impossible.

An older Hebrew encoding actually encoded the last character first, then the penultimate character, then the character preceding that, etc.

Exercise to the reader to guess how line breaks, text wrapping, and search algorithms worked.

Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

#19

Post by cubefox »

goatlover wrote: post #17
cubefox wrote: post #14

Earlier quoted context omitted.

> Hebrew is still written sequentially Everything is written sequentially in the sense that the character that is written first can only be followed by the character that is written next. In this sense writing non-sequentially is logically impossible.

Multiple characters can be written at once, they can also be done in reverse or out of order.

No no, the second character you write must always be temporally preceded by the character you wrote first. Otherwise the second wouldn't have been the second, but the first, and moreover, the first would have been the second, which it wasn't.
Post reply on HN