Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left
1–10 of 154 posts
Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left
#2Interesting. Upshot - right to left eval means you generally must start at the end, or at least hold an expression in working memory - LLMs - not so good at this.
I wonder if diffusion models would be better at this; most start out as sequential token generators and then get finetuned.
Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left
#3Interesting. Upshot - right to left eval means you generally must start at the end, or at least hold an expression in working memory - LLMs - not so good at this. I wonder if diffusion models would be better at this; most start out as sequential token generators and then get finetuned.
Try it out?
https://deepmind.google/models/gemini-diffusion/
Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left
#4Same reason the same models don't fundamentally understand all languages. They're not trained to. Frankly the design changes to get this to work in training is minimal but this isn't the way English works so expect most of the corporate LLM to struggle because that's where the interest and money is.
Give it time until we have true globally multi lingual models for superior context awareness.
Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left
#5Seems like it could easily be training data set size as well.
I'd love to see some quantification of errors in q/kdb+ (or hebrew) vs. languages of similar size that are left-to-right.
Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left
#6Humans can't either? I think if this convention had been more usable form of programming, we'd know by now
Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left
#7"Claude is aware of that, but it struggled to write correct code based on those rules"
It's actually not, and unless they in some way run a rule engine on top of their LLM SaaS stuff it seems far fetched to believe it adheres to rule sets in any way.
Local models confuse Python, Elixir, PHP and Bash when I've tried to use them for coding. They seem more stable for JS, but sometimes they slip out of that too.
Seems pretty contrived and desperate to invent transpilers from quasi-Python to other languages to try and find a software development use for LLM SaaS. Warnings about Lisp macros and other code rewrite tools ought to apply here as well. Plus, of course, the loss of 'notation as a tool of thought'.
Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left
#8[deleted]
Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left
#9Humans can't either? I think if this convention had been more usable form of programming, we'd know by now
I think there is a reason for this, but maybe not a good one.
1. Function application should be left to right, e.g. `sqrt 4`
2. Precedence order should be very simple. In k, everything has the same precedence order (with the exceptions of brackets)
1 + 2 forces you to have this right to left convention, annoyingly.
Fwiw, I think 2 is great and I would rather give up 1 than 2. However, writing function application as `my_fun arg` is a very strong convention.
Re: Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left
#10"Claude is aware of that, but it struggled to write correct code based on those rules" It's actually not, and unless they in some way run a rule engine on top of their LLM SaaS stuff it seems far fetched to believe it adheres to rule sets in any way. Local models confuse Python, Elixir, PHP and Bash when I've tried to use them for coding. They seem more stable for JS, but sometimes they slip out of that too. Seems pr…
If your model is getting confused by python, its a bad model. Python is routinely the best language for all major models.