Gemma 4 QAT models: Optimizing compression for mobile and laptop efficiency

Post by steno132 » Fri, Jun 05, 2026, 8:48 PM UTC

Zambyte wrote: post #42

steno132 wrote: post #40

I don't get this obsession with smaller models. I've been using Claude and GPT models for years and have had zero issues with them. I see absolutely no benefit to me as a end user for a local model which is going to take up more of my CPU and memory and slow down my machine. I almost always have Internet and if I don't then not having access to a AI model is the least of my concerns.

I like using my computer.

Post by steno132 » Fri, Jun 05, 2026, 8:52 PM UTC

mannanj wrote: post #51

steno132 wrote: post #40

I don't get this obsession with smaller models. I've been using Claude and GPT models for years and have had zero issues with them. I see absolutely no benefit to me as a end user for a local model which is going to take up more of my CPU and memory and slow down my machine. I almost always have Internet and if I don't then not having access to a AI model is the least of my concerns.

I don't like the gaslighting of paying Anthropic or Open(Closed)AI and it being said its unsustainable for them to take my payment while simultaneously they take my data (edit: which is incredibly valuable) and I cannot opt out of that. The obsession is for leaving hostile and abusive entities, the corporations or the people who fund them that have a horrible track record in regards to ethicality, rights and respect…

Post by steno132 » Fri, Jun 05, 2026, 9:09 PM UTC

satvikpendem wrote: post #55

steno132 wrote: post #52

Earlier quoted context omitted.

Exactly, thank you, we are on the same page! It's great to be able to use our own devices and not have their compute coopted by a third party. I'd rather not have intensive compute needed shifted onto my personal machine which I want to use for something else.

By that logic, any software you run that isn't fully built by yourself is "third party" therefore you shouldn't run anything at all on your machine, thus obviating the need for it entirely.

Post by klardotsh » Fri, Jun 05, 2026, 9:11 PM UTC

steno132 wrote: post #54

mannanj wrote: post #51

Earlier quoted context omitted.

I don't like the gaslighting of paying Anthropic or Open(Closed)AI and it being said its unsustainable for them to take my payment while simultaneously they take my data (edit: which is incredibly valuable) and I cannot opt out of that. The obsession is for leaving hostile and abusive entities, the corporations or the people who fund them that have a horrible track record in regards to ethicality, rights and respect…

My view is, if you're going to use the service - you should give the data. It's like using Gmail and expecting them not to train their AI models on your data - how can you expect that when they're giving you a secure, reliable, highly functional email client completely for free? The digital economy only works if everyone pays their fair share. If you don't want to give your data then you are really harming everyone b…

Post by Zambyte » Fri, Jun 05, 2026, 9:19 PM UTC

steno132 wrote: post #56

satvikpendem wrote: post #55

Earlier quoted context omitted.

By that logic, any software you run that isn't fully built by yourself is "third party" therefore you shouldn't run anything at all on your machine, thus obviating the need for it entirely.

But practically AI inference requires substantial local computing resources. It's not some web app, it's a order of magnitude more compute needed