151–160 of 204 posts
This is what tells me that AI's are not 'self improving'. If they could read a manual and understand it, then they should come up with better solutions. Not just regurgitate bad slop they learned from bad examples.
Contrary to what most comments seem to indicate, my takeaway from this is that it doesn't really matter all that much for the agents what language you pick. If humans are still to be involved in the process at some point, then its imperative that the language can be read by them, so the preference or skills of the developer(s) are of primary concern, not the agent.
I've got some education in materials engineering; it'd be trivial to drop a curve (line) for optimizing the language selection, and Python obviously comes out on top.
The author doesn't even mention it. That discredits the whole article as far as I'm concerned.
i dont see enough love for Ruby. I've been using it since last year and it feels like php's more robust brother
> Most of the claims that get thrown around about how a particular language is good for LLM use seem to be wrong (e.g., the claim that Ruby, Clojure, and J, are particularly well suited to LLMs, which were mentioned in the evals linked above, as well as the somewhat common claim that Elixir is particularly suited to LLMs), but it's not clear what's right. I feel this is a missed opportunity to explore the architectur…
I'm not sure I trust a source that says "just 70 tokens average, nearly half of Clojure (109 tokens)". There's no reason to add the phrase "nearly half of", and there's especially no reason to add it when it's significantly far away from half. But on the main topic, I still feel that Go is an excellent choice for LLMs. There is pretty much just one way of doing most things, and the available training data is pretty c…
Go is absolutely one of the best programming languages for LLMs for the reason you say, and Python is just what LLMs like to use to write short throwaway scripts. Frontier LLMs are generally pretty good at most programming languages and can pick up new ones pretty quickly. Training data seems to mostly just increase the speed which they write code, for example, GPTs tend to write Rust and Python faster than other pro…
Add in golang doesn’t have proper types for Option to have a nilable type. Its pointers and null pointer bugs are plenty, no proper stack traces.
I find myself to be less productive.
Earlier quoted context omitted.
Correctness is binary, while quality is not. Correctness is a suitable property to act as a multiplier in your formula, where incorrect is 0 and correct is 1, but you also need other facets to find a quality gradient.
In this context it's not binary. Context is everything. If it was binary there would only be 0 and 1 on one of the axis in the graph. That's not the case.
Think of it as being like a road trip. Arrival is binary. You have either arrived or have not arrived, but distance can tell you how close you are to arrival. Being almost there does not imply that you have arrived, however. Same applies here. Some samples were closer to being correct than others, but none were correct. They were all incorrect.
But as you alluded to earlier, I don't suppose anyone wants code that fails to solve the stated problem. Given that we have empirical evidence that LLMs cannot produce correct code within a given set of problems (and I suspect that extends to most problems), correctness is a weak signal.
What is a useful is to know is how much additional effort is required to make the program correct. That is what quality has traditionally meant as it pertains to code. High quality codebases are considered high quality because the effort to reach and maintain correctness is considered to be low. We do not have enough context to know for certain if that is what was meant in the earlier comment, but it seems likely. What we do know is that the comment is about quality while the article is about correctness.
Earlier quoted context omitted.
Go is absolutely one of the best programming languages for LLMs for the reason you say, and Python is just what LLMs like to use to write short throwaway scripts. Frontier LLMs are generally pretty good at most programming languages and can pick up new ones pretty quickly. Training data seems to mostly just increase the speed which they write code, for example, GPTs tend to write Rust and Python faster than other pro…
If you want the opposite experience, try LLMs with an older Oracle Database. Even with the official Oracle SqlDeveloper MCP Server they struggle a lot. However, I struggle as well and the error messages of Oracle are some of the worst I've ever experienced.