51–60 of 90 posts
My prophecy is that in 3 years we'll see a complete reversal of this. Using GenAI to code will be the default and we'll see policies that put limits on human/artisan development. Possibly even projects that outright ban non-LLM development.
Earlier quoted context omitted.
I don't think they ever will on the "maintainable" part until their absolutely atrocious memory gets improved at least tenfold. (and no, the tricks we deployed until now aren't enough) They already are better than most humans at writing code for sure but even the best SOTA model is worse than your average intern at memorization.
That's the entire point of the "maintainable" part: someone with no "memory" of what was written where and why should be able to walk into the codebase, understand how it works, and be able to make a sensible change to it without setting things on fire. Someone with "memory" should be able to do the same faster, but having that memory should not be a requirement. Whether that "someone" is a freshly onboarded mid-leve…
While I understand the caution, the current policy seems too draconian. It states in part: > Until that policy is in place, the Governing Board has approved this interim policy: > Contributions in the OpenJDK Community must not include content generated, in part or in full, by large language models... Note, this would exclude most spell checkers, as they often are LLM based. That said, they do soften this with the ad…
- spell checkers: ok
- '.getC' autocompleting to '.getColor()': ok
- 'getC' autocompleting to 'getColor()' + five more lines based on preceding code and documentation: NO.
This makes sense. AI contribution is basically just "prompt + AI work". Even if you are okay with AI work per se, you should accept prompts (after reviewing them) and let your own AI generate the code (and then also review the code)... rather then accept an output of someone else's AI with an unknown prompt, that may or may not include an instruction to create a vulnerability. In the age of AI, the prompt is becoming…
And the AI companies aren't focused on making this process any more deterministic, where repeated prompts are sampled using different seeds. I don't know if it's because the randomness makes the models explore a larger space and perform better, but it's probably intentionally done because it'd be quite easy to remove the stochasticity.
Earlier quoted context omitted.
That's the entire point of the "maintainable" part: someone with no "memory" of what was written where and why should be able to walk into the codebase, understand how it works, and be able to make a sensible change to it without setting things on fire. Someone with "memory" should be able to do the same faster, but having that memory should not be a requirement. Whether that "someone" is a freshly onboarded mid-leve…
Depends what you mean by your comment then, if you want to say that you can direct LLMs to write maintainable code yes sure you can through very strict guardrails, if they can do it themselves, no they can't and need help because of their bad memory.
Seems short sighted for an interim policy, to me. Why not just require clear and detailed disclosure of the use of said tools, to generate data for which to make a properly informed non interim decision?
I suspect one of the principle reasons for the blanket decline is specifically that AI generated PRs are already responsible for a substantial uptick in the amount of work required to be done by maintainers. Engaging in discussions about those PRs may be more fulfilling and useful for the developer but it also then increases that work yet again manifold, so that would likely be highly counter-productive. What I don't…
But even aside from the fact that writing the code is only a very small portion of the effort in this particular project (you can see that the volume of code making its way into the JDK is very small compared to the number of people involved), I'm often amazed by the gap between how well a frontier model (GPT 5.6 Sol in my case) can comprehend code and investigate a bug, and how badly it writes code and documentation, even when it understands things well. So not only is writing the code not a large portion of the effort in this project, it also happens to be one of the things current models don't do as well as other things.
How do you enforce this?
It's not impossible - right now there are all these low-effort PRs. If the PR is indistinguishable from a normal human-written PR, that addresses some of their concerns.
In any case, a rule or policy that can be worked around is not a reason to do away with that rule or policy.
I mean, if we only ever catch 10% of murderers, do you really think it is reasonable to remove that law from the books?
This makes sense. AI contribution is basically just "prompt + AI work". Even if you are okay with AI work per se, you should accept prompts (after reviewing them) and let your own AI generate the code (and then also review the code)... rather then accept an output of someone else's AI with an unknown prompt, that may or may not include an instruction to create a vulnerability. In the age of AI, the prompt is becoming…
Not really though, since the result of the prompt is not deterministic. It greatly depends on the model, the version, the harness, even time of day if the provider's infrastructure is currently overloaded and is silently degrading performance. Some things also require multi-turn interactions.
All of that are even more reasons to reject AI-generated code: if it cannot be trusted to produce same (or even similar) output just from the prompts, why accept it at all?
Earlier quoted context omitted.
Depends what you mean by your comment then, if you want to say that you can direct LLMs to write maintainable code yes sure you can through very strict guardrails, if they can do it themselves, no they can't and need help because of their bad memory.
Why would "memory" be relevant to that? I honestly don't see it be much more relevant to "write maintainable code" than it is to "play chess".
And that difference is in the memory. The LLM can work during 5 years on the codebase and still will be as good as a newcomer.
You can somewhat partially compensate for this bad memory by writing tons of guardrails and tons of extra LLM focused documentation, but it doesn't do everything.
Essentially, it's like talking to somebody who has a permanent memory damage.
Earlier quoted context omitted.
Not really though, since the result of the prompt is not deterministic. It greatly depends on the model, the version, the harness, even time of day if the provider's infrastructure is currently overloaded and is silently degrading performance. Some things also require multi-turn interactions.
> Not really though, since the result of the prompt is not deterministic. It greatly depends on the model, the version, the harness, even time of day if the provider's infrastructure is currently overloaded and is silently degrading performance. Some things also require multi-turn interactions. All of that are even more reasons to reject AI-generated code: if it cannot be trusted to produce same (or even similar) out…