301–310 of 382 posts
Can the same be done with qwen3.6-35b-a3b?
Owen 3.6-35b-a3b was my initial idea, but I switched to Gemma because of its simpler architecture and kernels
Nice. Gemma feels nice to write with but every time I use it for coding it struggles with tool calling significantly.
Earlier quoted context omitted.
In defense of this model, Gemma is actually a very good general-purpose model that can work with multiple languages. I use it for spam classification and for processing dictation, which means that I hold the entire model in memory all of the time, which is somewhat problematic (64GB RAM total, but heavy usage by docker, databases, etc)
Gemma is a great reference model and it’s easy to work with. Once you have Gemma working well, then do the extra work to use Qwen as well. I am using Gemma for a few tasks simply because it’s “good enough”.
Nice. Gemma feels nice to write with but every time I use it for coding it struggles with tool calling significantly.
Wow this is really cool! What are your thoughts on doing this with larger models?
Earlier quoted context omitted.
Let my karma burn for saying this: Maybe it is time to let this go man. These comments are really the new incarnation of "grammar policing". (1) They don't add anything of value, did the author use an LLM to fix his prose but no useless slop was added in the process: who cares ? Is the article useless slop: fine, downvote it to oblivion. (1) For those not old enough to remember that wonderful practice please use your…
The writing is made worse by a specific choice which the commentator identified. That’s actionable feedback.
I can tell you first hand it’s sometimes hard to figure out what is LLM writing and what isn’t, English isn’t my first language either.
Why it is only for Mac M-series? What's not compatible in a PC (with Linux) to run it?
Nice, I think this is the second time I see this here on HN, I always wondered why we need to shove the entire model into memory, I don't care who King Charles is every single time. It always felt as though we already figured out how to break up large files and parse them efficiently with very little memory. Frontier AI feels like its full of people who are brilliant at making models, but when it comes to scale and p…
> I don't care who King Charles is every single time Always curious when someone will figure out how we can elide most of the data from an LLM (but retain the logical ability). I don't actually need an LLM to have a very big internal knowledge base to be useful, so long as it can invoke a search tool...
to what extent is "retain the logical ability" meaningful without attaching it to some knowledge
Earlier quoted context omitted.
hm. just open repo, copy commands into your terminal and you will get app installed (if you have swift toolchain installed) after that download 14gb of weights and enjoy offline inference (and a bit of Gemma4 intelligence) for your everyday tasks multi turn chat is coming!
what is a swift tool chain?
Ran this on a 64 GB M4 Max MacBook. I figured having Gemma available with a small footprint would be a nice setup. No more unloading models when I need more RAM for work? Hell yea. Got 48 tok/s decode at 1.9 GB RSS (2.4 GB peak), faster than the 24 GB M5 Pro mentioned in the benchmarks. The ~2.0 GB/s SSD number quoted for M4 is the base chip. This M4 Max does ~7 GB/s. Page cache seems to be why it beats the M5 Pro. W…