11–20 of 73 posts
Here's the paper that they wrote: https://arxiv.org/pdf/2506.20639 It's notable that this was an intern project.
> Apple’s model is built on top of Qwen2.5‑7B, an open-source foundation model from Alibaba. Alibaba first fine-tuned that model for better code generation (as Qwen2.5‑Coder‑7B), then Apple took it and made its own adjustments.
> it still doesn’t quite reach the level of GPT-4 or Gemini Diffusion.
Does diffusion allow for 'size editing'? Unsure how to ask this, or if this (most likely) reveals a fundamental misunderstanding of my own, but: for an image, the size is set (say, 256x256.) For text, if each token were a pixel, it's very small. The article image showed text colour-coded by generation order. What if it would need to, say, insert another line for the rest of a comment sentence? How would it even know…
The Zed team recently posted a pretty good intro to diffusion models for text: https://www.youtube.com/watch?v=oot4O9wMohw
Short version: A Qwen-2.5 7b model that has been turned into a diffusion model. A couple notable things: first is that you can do this at all, (left to right model -> out of order diffusion via finetuning) which is really interesting. Second, the final version beats original by a small margin on some benchmarks. Third is that it’s in the ballpark of Gemini diffusion, although not competitive — to be expected for any…
Jetbrains has 100MB models of languages for their IDEs that can auto complete single lines. It's good but I think we can do better for local code auto complete. I hope Apple succeeds in their on device AI attempts.
Short version: A Qwen-2.5 7b model that has been turned into a diffusion model. A couple notable things: first is that you can do this at all, (left to right model -> out of order diffusion via finetuning) which is really interesting. Second, the final version beats original by a small margin on some benchmarks. Third is that it’s in the ballpark of Gemini diffusion, although not competitive — to be expected for any…
Are these small models good enough for anything but autocomplete?
Short version: A Qwen-2.5 7b model that has been turned into a diffusion model. A couple notable things: first is that you can do this at all, (left to right model -> out of order diffusion via finetuning) which is really interesting. Second, the final version beats original by a small margin on some benchmarks. Third is that it’s in the ballpark of Gemini diffusion, although not competitive — to be expected for any…
> At some point these local models will get good enough for ‘real work’ Are these small models good enough for anything but autocomplete?
They predict more than just the second half of a word you are typing, but at the end of the day they're still just predicting what a human would have typed.
Earlier quoted context omitted.
> At some point these local models will get good enough for ‘real work’ Are these small models good enough for anything but autocomplete?
Isn't that all they're designed for? They predict more than just the second half of a word you are typing, but at the end of the day they're still just predicting what a human would have typed.
Short version: A Qwen-2.5 7b model that has been turned into a diffusion model. A couple notable things: first is that you can do this at all, (left to right model -> out of order diffusion via finetuning) which is really interesting. Second, the final version beats original by a small margin on some benchmarks. Third is that it’s in the ballpark of Gemini diffusion, although not competitive — to be expected for any…
> At some point these local models will get good enough for ‘real work’ Are these small models good enough for anything but autocomplete?