Would love to try this in ollama/llama.cpp. Using llama.cpp for VsCode is painful since (realistically) I can only generate on the order of <100 tokens at a time
Apple just released a weirdly interesting coding language model
51–60 of 73 posts
Re: Apple just released a weirdly interesting coding language model
#52Earlier quoted context omitted.
How many people own a NAS, but not a PC or Mac? Apple already provides first party software to handle iDevice backups on Windows or Mac. Backing up an Android device to a PC using adb is significantly more difficult, especially for the less technically minded.
> How many people own a NAS, but not a PC or Mac? That’s arguably the wrong question: I bet a lot more would own one if they could easily backup their iOS devices to it.
The number of people that would but a NAS over just spending the 5$/month for storage is well below a percent and if you combine that with the requirement of not having a PC/Mac you may well end up in the hundreds…
There aren’t that many people that are willing to own a device from a company but not trusting that company with their data
Re: Apple just released a weirdly interesting coding language model
#53Short version: A Qwen-2.5 7b model that has been turned into a diffusion model. A couple notable things: first is that you can do this at all, (left to right model -> out of order diffusion via finetuning) which is really interesting. Second, the final version beats original by a small margin on some benchmarks. Third is that it’s in the ballpark of Gemini diffusion, although not competitive — to be expected for any…
When we look at the small models suitable for running locally, by far the best programming model is DeepSeek-R1-0528-Qwen3-8B. It is quite comparable in real world usage even to much bigger models.
Comparable to which bigger models?
Re: Apple just released a weirdly interesting coding language model
#54Earlier quoted context omitted.
How many people own a NAS, but not a PC or Mac? Apple already provides first party software to handle iDevice backups on Windows or Mac. Backing up an Android device to a PC using adb is significantly more difficult, especially for the less technically minded.
> How many people own a NAS, but not a PC or Mac? That’s arguably the wrong question: I bet a lot more would own one if they could easily backup their iOS devices to it.
I'm willing to bet that more people would backup their Android device if Google provided a first party tool for user friendly backups of Android devices to local computers.
Re: Apple just released a weirdly interesting coding language model
#55Earlier quoted context omitted.
> to my mind the architecture is a better fit for coding We have to see if it produces better results. Humans have a planning phase, followed be a part-by-part implementation phase. This is reasonably well emulated by plan/architect + codegen tools.
It's delusional to think that most software projects can be planned in advance beyond "there will be a beginning, a middle, and an end". People do it, but their efforts are in my experience generally ignored once implementation get underway.
laughs nervously under a waterfall
Re: Apple just released a weirdly interesting coding language model
#56Earlier quoted context omitted.
I think Apple will ultimately destroy the data center, I hope they succeed.
Not without fundamentally changing the way they think about computing and there seems to be zero willingness among their leadership to do that. In fact they seem to want to move things into the data center. That's why I'm shorting them.
I think it’s just a convenient stepping stone more than a long term strategy.
Re: Apple just released a weirdly interesting coding language model
#57Earlier quoted context omitted.
Without having tried it, what I keep getting surprised with is how apparently widely different architectures (and in other cases training data) lead to very similar outcomes. I'd expect results to vary a lot more.
But if the limiting factor is the data on which the models are trained and not the actual “computation” than this would be exactly expected right?
The data might be the limiting factor of current transformer architectures, but there's no reason to believe it's a general limiting factor of any language model (e.g. humans brains are "trained" on orders of magnitude less data and still generally perform better than any model available today)
Re: Apple just released a weirdly interesting coding language model
#58Short version: A Qwen-2.5 7b model that has been turned into a diffusion model. A couple notable things: first is that you can do this at all, (left to right model -> out of order diffusion via finetuning) which is really interesting. Second, the final version beats original by a small margin on some benchmarks. Third is that it’s in the ballpark of Gemini diffusion, although not competitive — to be expected for any…
When we look at the small models suitable for running locally, by far the best programming model is DeepSeek-R1-0528-Qwen3-8B. It is quite comparable in real world usage even to much bigger models.
Would you mind sharing how you arrived at this conclusion? Was there some benchmark that it really shined at? Personal use?
Re: Apple just released a weirdly interesting coding language model
#59Short version: A Qwen-2.5 7b model that has been turned into a diffusion model. A couple notable things: first is that you can do this at all, (left to right model -> out of order diffusion via finetuning) which is really interesting. Second, the final version beats original by a small margin on some benchmarks. Third is that it’s in the ballpark of Gemini diffusion, although not competitive — to be expected for any…
> A diffusion model comes with a lot of benefits in terms of parallelization and therefore speed; to my mind the architecture is a better fit for coding than strict left to right generation.
I had a similar notion and am excited to see this research being done. My experience of writing code is that the structure of the whole system influences each individual part, which has always felt like a better match for a diffusion type model.
I’m suspecting this is a 7B model because it’s an experiment, but I do like seeing Apple playing with smaller models - I think Google’s “no moat” memo is still fundamentally correct, either via better architectures or Moore’s law, and it seems like Apple thinks the same.
Re: Apple just released a weirdly interesting coding language model
#60Earlier quoted context omitted.
> to my mind the architecture is a better fit for coding We have to see if it produces better results. Humans have a planning phase, followed be a part-by-part implementation phase. This is reasonably well emulated by plan/architect + codegen tools.
It's delusional to think that most software projects can be planned in advance beyond "there will be a beginning, a middle, and an end". People do it, but their efforts are in my experience generally ignored once implementation get underway.
That’s true at the project level. But surely when you sit down to actually work for a couple hours you think about what you are going to do, and then mostly do that.