Would be awesome if it ran Qwen (the MoE probably won't squeeze that low, but...). This because I have hardly been able to use Gemma for any sort of useful coding.
You’re right, Gemma isn’t the best model for coding (afaik more "everyday tasks" related). My first idea was to use Qwen, but its architecture was much more complex to implement in this stack. I chose Gemma so I wouldn’t spend all my time debugging custom kernels and could actually move the project forward with simpler approach
In defense of this model, Gemma is actually a very good general-purpose model that can work with multiple languages. I use it for spam classification and for processing dictation, which means that I hold the entire model in memory all of the time, which is somewhat problematic (64GB RAM total, but heavy usage by docker, databases, etc)