I’d ask for a refund on the tokens tbh
Vera: a programming language designed for machines to write
71–80 of 105 posts
Re: Vera: a programming language designed for machines to write
#72> The empirical literature shows that models are particularly vulnerable to naming-related errors like choosing misleading names, reusing names incorrectly, and losing track of which name refers to which value. I think Vera might be missing something here. In my experience, LLMs code better the less of a mental model you need, vs the more is in text on the page. Go – very little hidden, everything in text on the page…
This will serve as an interesting empirical test, then: will LLMs do better with Vera than with Go or other languages? The testing so far seems inconclusive ( https://github.com/aallan/vera-bench ), but the authors make this interesting observation: "No LLM has ever been trained on Vera. There are no Vera examples on GitHub, no Stack Overflow answers, no tutorials — the language was created after these models' traini…
[deleted]
Re: Vera: a programming language designed for machines to write
#73Earlier quoted context omitted.
> Of course there's no way to tell at compile time that a value will definitely be zero. Yes there is. Dependently typed languages like Idris can inspect terms at the value-level during compile time. Rather, instead of proving that the divisor will be zero, you must instead statically prove that the divisor cannot be zero; otherwise the code will not typecheck.
Okay, int integer_division(int a, int b) { if (b!=0) return a/b; raise(SIGFPE); } Great.
[deleted]
Re: Vera: a programming language designed for machines to write
#74The lack of naming seems to indicate a fundamental misunderstanding of how LLM coding agents are successful, and just makes me doubt anything about this project being useful and workable.
Yeah it seems based on 2023 research which is ancient, back when we didn't have coding agents at all, and on some 1980s sci fi concepts of "how machines think" (beedeeboop) rather than the all too human coding agents we have. If I had to design one of these, I'd go for: 1. Token minimization (which may be circular, I'm sure tokens are selected for these models at least in part based on syntax of popular languages) 2.…
Well, Rust does fulfill these to a reasonable degree. There is obvious room for improvement, but the vast majority of languages don't even bother being a Rust successor. Instead, they take a step back and decided that what Rust is doing is too much, e.g. Zig. It's kind of irritating that everyone and their dog is coming up with a new programming language that barely changes anything when there are so many low hanging fruit. The vast majority of programming languages that people are coming up with could have been language subsets, extensions or alternative runtimes for existing languages.
Re: Vera: a programming language designed for machines to write
#75Earlier quoted context omitted.
You don't appear to understand the difference between runtime and static analysis/compile time, or term-level and type-level.
Great! Explain it to us while I read to my kid!
This is a very antagonistic comment. Some people would call it "passive aggressive".
Dude, if you're reading to your kid you're clearly busy doing something else. No matter how simple the concept is, if you don't pay attention you're not going to get it so it's a failure on your part and not a failure on the part of the person patiently trying to explain something to you.
Re: Vera: a programming language designed for machines to write
#76Is there any evidence that using structural references rather than names allows large language models to generate better code? This bit just feels like obfuscation for obfustcation’s sake.
I've read the FAQ ( https://github.com/aallan/vera/blob/main/FAQ.md ) that provides the justification for this and it is, IMO, fairly weak. The main argument is that misleading names can confuse models. I have no problem believing this bit I'm not sure why we should assume code will have misleading names. In fact, the same document says that in tests they've had LLMs mix up the indices, which is exactly the problem I…
[dead]
Re: Vera: a programming language designed for machines to write
#77> The empirical literature shows that models are particularly vulnerable to naming-related errors like choosing misleading names, reusing names incorrectly, and losing track of which name refers to which value. I think Vera might be missing something here. In my experience, LLMs code better the less of a mental model you need, vs the more is in text on the page. Go – very little hidden, everything in text on the page…
Hmm, interesting. Are you speaking from experience for Haskell? I'm a Haskell developer since 2017, and have been using LLMs to write code (including Haskell) since 2024. In my experience, LLMs perform much better generating Haskell/Rust code over Python/Javascript.
Re: Vera: a programming language designed for machines to write
#78It feels wrong to dump identifiers to save tokens: now they're devoid of semantics, and can't be grep'ed or mapped to concepts. CPUs are good with numbers, but LLMs are good with words.
Re: Vera: a programming language designed for machines to write
#79There are many problems we will need to address in the future. A programming language that is easy for machines to write but hard for humans to read isn’t one of them.
This isn't that different from circuit languages.
Whittling everything down so the language is relatively 1-to-1 with the structure of the compute. With little or no extraneous decoration.
Re: Vera: a programming language designed for machines to write
#80> There are no variable names. @Int.0 is the most recent Int binding; @Int.1 is the one before. You already lost me here. There's a reason variable names are a thing in programming, and that's to semantically convey meaning. This matters no matter whether a human is writing the code or a LLM.
>The short answer is that variable names are one of the things that confuses LLMs rather than helps them. Unlike with humans, names undermine a model's efforts to keep track of state over larger scales. Models confuse similarly named variables in different parts of the codebase easily So I wonder, doesn't this apply to function names too, which the author keeps in? I've seen LLMs use wrong functions/classes as well.…
The only way the author could have come up with that rationale is that he doesn't understand what a token is, what attention is and how coding agents work.
Tokens combine multiple characters into a single vector. Attention computes similarity scores between vectors. This means you'd want each variable to be a single token so that the LLM can instantly know that two names refer to the same variable. If everything is numbered, the attention mechanism will attend every first parameter to every first parameter in every function. This means that the numbering scheme would have to be randomized instead of starting at zero.
Coding agents are now capable of using tools, including text search, which means that having the ability to look for specific variable names is extremely helpful. By using numbering, the author of the language has now given himself the burden of relying entirely on LSPs rather than innate model properties that operate on the text level.
So yeah, on a textual level, the language is designed for an era of LLMs that has been obsolete for a long time.