[flagged]
Use boring languages with LLMs
71–80 of 180 posts
Re: Use boring languages with LLMs
#72Instead of empty theorizing, we should have benchmarks for this. There is at least one benchmark which suggests that LLMs are better at writing Elixir than most other languages: see the AutoCodeBenchmark.
Send me an email I'd love to make one
Re: Use boring languages with LLMs
#73I can probably fix package manager issues by hand, and quickly with a little rubber ducking with the LLM itself. I'm not sure that's a huge problem in the grand scheme.
There's a lot of stuff in Python's favor in regard to coding with LLMs: its wildly popular so there's a lot of references for the right and wrong ways to use it, it can be typed using included libraries - its as simple as telling the LLM "use typing for this", and there are several great lint and unit testing tools to cover the hallucinations and poor decisions. The flexibility seems like an advantage to me personally, but I've always been a Python stan.
Re: Use boring languages with LLMs
#74I don't think the python package manager is the high level difficulty for LLMs doing python. I think the high level difficulty are nonlocal effects. At any given callsite, it might be difficult to know exactly what is going to happen to the data you pass into the call.
A sibling mentioned that LLMs benchmark better on elixir. Immutability and functional programming are likely the reason why it benches better.
Re: Use boring languages with LLMs
#75I program in two languages: Swift (my main language), for client work, and PHP, for backend work. It’s overwhelmingly Swift. In the last year or so, I have been using LLMs, to assist my work, with generally, excellent results. I have noticed that the LLM delivers much better PHP, than Swift. I seldom need to rewrite or correct, the PHP code I get from it, and am constantly correcting the Swift. Part of the reason, ma…
> I have noticed that the LLM delivers much better PHP, than Swift.
If you ever have the time and inclination to try Axiom (https://charleswiltgen.github.io/Axiom/), I'd really appreciate knowing if you feel it quantitatively changes the Swift experience with your LLM/coding harness of choice, especially in regards to Swift concurrency.
Re: Use boring languages with LLMs
#76"The concurrency model is the first of these. Goroutines are a far more tractable primitive for coding agents than threads, callbacks, async/await, or any of the colored-function regimes that dominate elsewhere. They are simple, type-safe, and ubiquitously used in the corpus the model was trained on. There is no question of what color your function is, because the question does not exist."
I don't really buy the intuition (aka Goroutines are more 'clear' than 'coloured' functions or threads), and there's no evidence presented for this either.
Although this could very well be true, I'm doubtful without seeing some real world data points.
The 'general premise' aka 'cosine similarity' may have been true before bit it may not be that anymore.
AI just pretty good at anything it's 'seen enough' and that's it, I think it's more likely a 'threshold' problem than an ability problem, at least for most things.
'Rust' may represent a different domain, given the very detailed nature of notation and the vast possibilities that arise from that.
Re: Use boring languages with LLMs
#77Large codebases are much easier to manage with type safety. Not a fan of Go but definitely much better than python in this regard.
Re: Use boring languages with LLMs
#78I disagree. "Boring" languages leave a lot of assumptions in code, which will start to compound the more changes model (and programmers) make to the code. The more assumptions I can move to compile time the better models are at dealing with emerging complexity. I would go the other way with LLMs and I wish for liquid types and effects in Rust to make type specifications even more strict. P.S. effects and liquid types…
The problem is that most of Rust annotations are related to memory management in special conditions aka solving problems that don't even exist in JS or Java. That's not going help the AI solve problem space issues, it just helps the AI (and us) do things in the solution space aka solve lower level things we consider important.
Re: Use boring languages with LLMs
#79I program in two languages: Swift (my main language), for client work, and PHP, for backend work. It’s overwhelmingly Swift. In the last year or so, I have been using LLMs, to assist my work, with generally, excellent results. I have noticed that the LLM delivers much better PHP, than Swift. I seldom need to rewrite or correct, the PHP code I get from it, and am constantly correcting the Swift. Part of the reason, ma…
> I have noticed that the LLM delivers much better PHP, than Swift. If you ever have the time and inclination to try Axiom ( https://charleswiltgen.github.io/Axiom/ ), I'd really appreciate knowing if you feel it quantitatively changes the Swift experience with your LLM/coding harness of choice, especially in regards to Swift concurrency.
Looks like a labor of love.
Honestly, not sure that I'll be able to use it, right now (no telling, in the future). Looks like you really did a good job, though!
Re: Use boring languages with LLMs
#80Author here, wasn't expecting this piece of writing to show up on HN.
The specifics of Python were chosen only due to the language ecosystem being fragmented and inconsistent while Python remains an essential learning, research, and now ML programming language (it was my first language and I still love it).
My thoughts on LLM generated code have changed immensely in the last 9 months as I've taken on teams and projects through my consulting work [1] as a fractional CTO. Python remains a difficult, flakey, and inconsistent programming language for complex production systems. Most other programming languages suffer from fragmented toolchains and ecosystems: JavaScript (famously), PHP, and even C/C++ to a degree.
Languages with a single way to do things benefit the most: Ruby, Rust, Swift (even). Low entropy is the way to go and convention > configuration seems to pay off with LLMs.
Mean cost of management is more important than specific edge examples "X company run on Y language". I think that 'boring' languages with rock-solid compilers, toolchains, testing frameworks, and package managers make for high return on engineering time and production maintenance.
[1]: sancho.studio