21–30 of 97 posts
Its tooling like this that really makes golang an excellent language to work with. I had missed that rangeint addition to the language but with go fix I'll just get that improvement for free! Real kudos to the golang team.
This tool is way cooler, post-redesign.
I really liked this part: In December 2024, during the frenzied adoption of LLM coding assistants, we became aware that such tools tended—unsurprisingly—to produce Go code in a style similar to the mass of Go code used during training, even when there were newer, better ways to express the same idea. Less obviously, the same tools often refused to use the newer ways even when directed to do so in general terms such a…
They're particularly bad about concurrent go code, in my experience - it's almost always tutorial-like stuff, over-simplified and missing error and edge case handling to the point that it's downright dangerous to use... but it routinely slips past review because it seems simple and simple is correct, right? Go concurrency is so easy! And then you point out issues in a review, so the author feeds it back into an LLM,…
That's a langage problem that humans face as well, which golang could stop having (see C++'s Thread Safety annotations).
I think tooling that can modify your source code to make it more modern is really cool stuff. OpenRewrite comes to mind for Java, but nothing comes to the top of my mind for other languages. And heck, I into recently learned about OpenRewrite and I've been writing Java for a long time. Even though I don't like Go, I acknowledge that tooling like this built right into the language is a huge deal for language popularit…
I really liked this part: In December 2024, during the frenzied adoption of LLM coding assistants, we became aware that such tools tended—unsurprisingly—to produce Go code in a style similar to the mass of Go code used during training, even when there were newer, better ways to express the same idea. Less obviously, the same tools often refused to use the newer ways even when directed to do so in general terms such a…
PHP went through a similar effort a while back to just clear out places like Stackoverflow of terrible out of date advice (e.g. posts advocating magic_quotes). LLMs make this a slightly different problem because, for the most part, once the bad advice is in the model it's never going away. In theory there's an easier to test surface around how good the advice it's giving is but trying to figure out how it got to that…
The way you should think of RL (both RLVR and RLHF) is the "elicitation hypothesis[1]." In pretraining, models learn their capabilities by consuming large amounts of web text. Those capabilities include producing both low and high quality outputs (as both low and high quality outputs are present in their pretraining corpora). In post training, RL doesn't teach them new skills (see E.G. the "Limits of RLVR"[2] paper). Instead, it "teaches" the models to produce the more desirable, higher-quality outputs, while suppressing the undesirable, low-quality ones.
I'm pretty sure you could design an RL task that specifically teaches models to use modern idioms, either as an explicit dataset of chosen/rejected completions (where the chosen is the new way and the rejected is the old), or as a verifiable task where the reward goes down as the number of linter errors goes up.
I wouldn't be surprised if frontier labs have datasets for this for some of the major languages and packages.
[1] https://www.interconnects.ai/p/elicitation-theory-of-post-tr...
I really liked this part: In December 2024, during the frenzied adoption of LLM coding assistants, we became aware that such tools tended—unsurprisingly—to produce Go code in a style similar to the mass of Go code used during training, even when there were newer, better ways to express the same idea. Less obviously, the same tools often refused to use the newer ways even when directed to do so in general terms such a…
They're particularly bad about concurrent go code, in my experience - it's almost always tutorial-like stuff, over-simplified and missing error and edge case handling to the point that it's downright dangerous to use... but it routinely slips past review because it seems simple and simple is correct, right? Go concurrency is so easy! And then you point out issues in a review, so the author feeds it back into an LLM,…
Earlier quoted context omitted.
PHP went through a similar effort a while back to just clear out places like Stackoverflow of terrible out of date advice (e.g. posts advocating magic_quotes). LLMs make this a slightly different problem because, for the most part, once the bad advice is in the model it's never going away. In theory there's an easier to test surface around how good the advice it's giving is but trying to figure out how it got to that…
This is one area where reinforcement learning can help. The way you should think of RL (both RLVR and RLHF) is the "elicitation hypothesis[1]." In pretraining, models learn their capabilities by consuming large amounts of web text. Those capabilities include producing both low and high quality outputs (as both low and high quality outputs are present in their pretraining corpora). In post training, RL doesn't teach t…
In Stackoverflow data is trivial to edit and the org (previously, at least) was open to requests from maintainers to update accepted answers to provide more correct information. Editing is trivial and cheap to carry out for a database - for a model editing is possible (less easy but do-able), expensive and a potential risk to the model owner.
Earlier quoted context omitted.
The use of LLMs will lead to homogeneous, middling code.
Middling code should not exist. Boilerplate code should not exist. For some reason we're suddenly accepting code-gen as SOP instead of building a layer of abstraction on top of the too-onerous layer we're currently building at. Prior generations of software development would see a too-onerous layer and build tools to abstract to a higher level, this generation seems stuck in an idea that we just need tooling to gener…
Earlier quoted context omitted.
Middling code should not exist. Boilerplate code should not exist. For some reason we're suddenly accepting code-gen as SOP instead of building a layer of abstraction on top of the too-onerous layer we're currently building at. Prior generations of software development would see a too-onerous layer and build tools to abstract to a higher level, this generation seems stuck in an idea that we just need tooling to gener…
LLMs have always been great at generating code that doesn't really mean anything - no architectural decisions, the same for "any" program. But only rarely does one see questions why we're needing to generating "meaningless" code in the first place.
For a long time my motto around software development has been "optimize for maintainability" and I'm quite concerned that in a few years this habit is going to hit us like a truck in the same way the off-shoring craze did - a bunch of companies will start slowly dying off as their feature velocity slows to a crawl and a lot of products that were useful will be lost. It's not my problem, I know, but it's quite concerning.
Earlier quoted context omitted.
The use of LLMs will lead to homogeneous, middling code.
Middling code should not exist. Boilerplate code should not exist. For some reason we're suddenly accepting code-gen as SOP instead of building a layer of abstraction on top of the too-onerous layer we're currently building at. Prior generations of software development would see a too-onerous layer and build tools to abstract to a higher level, this generation seems stuck in an idea that we just need tooling to gener…