Apple just released a weirdly interesting coding language model
1–10 of 73 posts
Re: Apple just released a weirdly interesting coding language model
#2[deleted]
Re: Apple just released a weirdly interesting coding language model
#3Articles such as this really make me wish that the current generation of LLMs were more often described as workable implementations of "the infinite monkey theorem" --- the descriptions/images in this article are esp. worth showing to aperson to whom one is trying to describe how an AI model "creates" an image.
Re: Apple just released a weirdly interesting coding language model
#4Short version: A Qwen-2.5 7b model that has been turned into a diffusion model.
A couple notable things: first is that you can do this at all, (left to right model -> out of order diffusion via finetuning) which is really interesting. Second, the final version beats original by a small margin on some benchmarks. Third is that it’s in the ballpark of Gemini diffusion, although not competitive — to be expected for any 7B parameter model.
A diffusion model comes with a lot of benefits in terms of parallelization and therefore speed; to my mind the architecture is a better fit for coding than strict left to right generation.
Overall, interesting. At some point these local models will get good enough for ‘real work’ and they will be slotted in at API providers rapidly. Apple’s game is on-device; I think we’ll see descendants of these start shipping with Xcode in the next year as just part of the coding experience.
Re: Apple just released a weirdly interesting coding language model
#5Does diffusion allow for 'size editing'? Unsure how to ask this, or if this (most likely) reveals a fundamental misunderstanding of my own, but: for an image, the size is set (say, 256x256.) For text, if each token were a pixel, it's very small. The article image showed text colour-coded by generation order. What if it would need to, say, insert another line for the rest of a comment sentence? How would it even know the size upfront, the way an image size is known?
Re: Apple just released a weirdly interesting coding language model
#6Short version: A Qwen-2.5 7b model that has been turned into a diffusion model. A couple notable things: first is that you can do this at all, (left to right model -> out of order diffusion via finetuning) which is really interesting. Second, the final version beats original by a small margin on some benchmarks. Third is that it’s in the ballpark of Gemini diffusion, although not competitive — to be expected for any…
I think Apple will ultimately destroy the data center, I hope they succeed.
Re: Apple just released a weirdly interesting coding language model
#7Short version: A Qwen-2.5 7b model that has been turned into a diffusion model. A couple notable things: first is that you can do this at all, (left to right model -> out of order diffusion via finetuning) which is really interesting. Second, the final version beats original by a small margin on some benchmarks. Third is that it’s in the ballpark of Gemini diffusion, although not competitive — to be expected for any…
I think Apple will ultimately destroy the data center, I hope they succeed.
Shades of 1980s Apple v. Big Blue. I can't wait for the rehash of the "1984" ad.
Re: Apple just released a weirdly interesting coding language model
#8Here's the paper that they wrote: https://arxiv.org/pdf/2506.20639
It's notable that this was an intern project.
Re: Apple just released a weirdly interesting coding language model
#9Jetbrains has 100MB models of languages for their IDEs that can auto complete single lines. It's good but I think we can do better for local code auto complete. I hope Apple succeeds in their on device AI attempts.
Re: Apple just released a weirdly interesting coding language model
#10Short version: A Qwen-2.5 7b model that has been turned into a diffusion model. A couple notable things: first is that you can do this at all, (left to right model -> out of order diffusion via finetuning) which is really interesting. Second, the final version beats original by a small margin on some benchmarks. Third is that it’s in the ballpark of Gemini diffusion, although not competitive — to be expected for any…
> to my mind the architecture is a better fit for coding
We have to see if it produces better results. Humans have a planning phase, followed be a part-by-part implementation phase. This is reasonably well emulated by plan/architect + codegen tools.