I think TypeScript is uniquely positioned to be the optimal language for LLMs. Tons of training data (benefiting from all the JS examples as well) plus the structure of types for LLMs to follow and tools to enforce.
God help us…
Type-constrained code generation with language models
21–30 of 134 posts
Re: Type-constrained code generation with language models
#22I think TypeScript is uniquely positioned to be the optimal language for LLMs. Tons of training data (benefiting from all the JS examples as well) plus the structure of types for LLMs to follow and tools to enforce.
There are languages that constrain types a lot more tightly than TypeScript, e.g. Kotlin, Rust, and Haskell. The more constrained the types, the more correct the program could be.
Re: Type-constrained code generation with language models
#23Earlier quoted context omitted.
This is about the speed with which the compiler can advise an LLM that a particular thing checks or doesn't check. Typescript is much slower than Go
okay so basically the faster compiling means a tigher feedback loop for the LLM to -know- if the code compiles or not etc? interesting is go faster than rust?
> is go faster than rust?
Depends on how you write the Go or Rust code. The most optimal Rust re-write of the TypeScript compiler would very likely be faster than the most optimal version in Go. However they didn't want to do a re-write, they wanted to port the existing compiler codebase written in TS. Go like TS (ultimately the JS runtime) also has GC which makes a 1-to-1 port much easier.
Re: Type-constrained code generation with language models
#24Re: Type-constrained code generation with language models
#25Earlier quoted context omitted.
okay so basically the faster compiling means a tigher feedback loop for the LLM to -know- if the code compiles or not etc? interesting is go faster than rust?
Go’s compiler is WAY faster than Rust’s. As far as speed of the actual program, Rust will generally be faster.
Go or Rust compiler speeds won't have any effect here. The program in this context is the TypeScript compiler.
Re: Type-constrained code generation with language models
#26we really need LLM trained on AST, instead of token, is there any research on this?
ASTrust: Towards More Trustworthy and Interpretable LLMs for Code through Syntax-Grounded Explanations
https://arxiv.org/abs/2407.08983
AST-T5: Structure-Aware Pretraining for Code Generation and Understanding
https://arxiv.org/abs/2401.03003
CodeGRAG: Bridging the Gap between Natural Language and Programming Language via Graphical Retrieval Augmented Generation
Re: Type-constrained code generation with language models
#27I think TypeScript is uniquely positioned to be the optimal language for LLMs. Tons of training data (benefiting from all the JS examples as well) plus the structure of types for LLMs to follow and tools to enforce.
LLMs work well with any static analysis tool. I frequently instruct Claude to use stuff like “go vet” and “deadcode” when it goes on a tear and writes a bunch of broken trash and declares mission accomplished.
Re: Type-constrained code generation with language models
#28Earlier quoted context omitted.
okay so basically the faster compiling means a tigher feedback loop for the LLM to -know- if the code compiles or not etc? interesting is go faster than rust?
Go’s compiler is WAY faster than Rust’s. As far as speed of the actual program, Rust will generally be faster.
cargo check is WAY faster than go build
Re: Type-constrained code generation with language models
#29The vibe code society would benefit way more if libraries hosted their docs in a way that's easy to copy and paste into an LLM.
Re: Type-constrained code generation with language models
#30I think TypeScript is uniquely positioned to be the optimal language for LLMs. Tons of training data (benefiting from all the JS examples as well) plus the structure of types for LLMs to follow and tools to enforce.
There are languages that constrain types a lot more tightly than TypeScript, e.g. Kotlin, Rust, and Haskell. The more constrained the types, the more correct the program could be.
Yep, and Rust famously goes beyond this by modelling memory ownership at compile time.
In fact, the more behaviour we can model at compile time the better when it comes to LLMs - there's some cool ideas here like transpiling Rust into languages for formal verification. See https://github.com/formal-land/coq-of-rust as an example.
Formal verification was one of those things that was previously so annoying to do that it rarely made it past academic use cases or extremely important libraries, but I think LLMs take the tedium out of it. Perhaps formal verification will have a "test driven development" type of moment in the sun thanks to this.