This is fun. I just wish I could add more skills, the UX is too dumbed down but knowing there is a run_js tool there is a lot that can be done here.
Gemma 4 on iPhone
171–180 of 267 posts
Re: Gemma 4 on iPhone
#172it's Google, so is it really private? remember, megacorps are dying for infinite amounts of analytics data
When I saw it wants me to "agree" to Google's "privacy policy", I deleted the app on the spot.
Re: Gemma 4 on iPhone
#173Earlier quoted context omitted.
> or in the cloud but way more expensive then it is today. Why? It's widely understood that the big players are making profit on inference. The only reason they still have losses is because training is so expensive, but you need to do that no matter whether the models are running in the cloud or on your device. If you think about it, it's always going to be cheaper and more energy-efficient to have dedicated cloud ha…
> It's widely understood that the big players are making profit on inference. This is most definitely not widely understood. We still don't know yet. There's tons of discussions about people disagreeing on whether it really is profitable. Unless you have proof, don't say "this is widely understood".
I don’t have “proof” but the existence of so many providers of free models on OpenRouter strongly suggests inference is running at a profit. There’s no winner-takes-all angle to being a faceless provider there (often the consumer doesn’t know who fulfilled the request), so there’s just no incentive at all for these small provider companies to exist unless inference is profitable under the right conditions.
Re: Gemma 4 on iPhone
#174Earlier quoted context omitted.
From what I've seen gemma 4 doesn't refuse a lot regarding sex, it only needs little nudging in the right direction sometimes. But it does refuse being critical of the usual topics: israel, islam, trans, or race. So wanting to discuss one of those is the real reason people would use an uncensored model.
It’s so dispiriting to me that we’ve achieved those closest thing yet to an “objective truth” machine (with the caveat of garbage in, garbage out, etc.) and these big companies are either afraid to actually let it exist, want to push their own politics, or a combination of the two.
"closest thing yet" is still a long way from close; as you say, gin=gout, and the internet without an attempt to be our best selves is instead our loudest propagandists and all our cultural stereotypes.
Of course, humans are also impacted by these things, at best we can be a little deliberate about rejecting a few of the more on-the-nose examples.
Re: Gemma 4 on iPhone
#175Earlier quoted context omitted.
The manufacturing of biologics can be heavily censored to an absurd degree. I don’t know about Gemma 4 in particular.
Really? That's fascinating. Why is that?
Do you want every malicious idiot in the world to have a competent helper for bioweapons?
Or indeed an incompetent but enthusiastic helper accidentally getting them to posion themselves and friends with botox:? https://news.ycombinator.com/item?id=40724283
That is why they were pushed away from this. At least with vibe coded software, errors may prevent compilation, then when we're past that simply bad experiences, before they become human catastrophes.
Re: Gemma 4 on iPhone
#176Earlier quoted context omitted.
I tried it on my mac, for coding, and I wasn't really impressed compared to Qwen. I guess there are things it's better at?
Assuming you’re not copy/pasting for these tasks. What’s the stack required to use local models for coding? I’ve got a capable enough machine to produce tokens slowly, but don’t understand how to connect that to the likes of VSCode or a JetBrains ide.
You need some way to give it tools - the essential ones for coding are running bash commands, reading files and editing files.
You need the LLM to be able to respond with tool use requests, and then your local harness to process them and respond to it. You can read how tool calling works with eg Claude API to get the idea: https://platform.claude.com/docs/en/agents-and-tools/tool-us...
Under the hood something like Claude Code is calling the API with tools registered, and then when it gets a tool use request it runs that locally, and then responds to the API with the result. That’s the loop that enables coding.
Integrating with an IDE specifically is really just a UI feature, rather than the core functionality.
Re: Gemma 4 on iPhone
#177English version of the page: https://apps.apple.com/us/app/google-ai-edge-gallery/id67496... Also on Android: https://play.google.com/store/apps/details?id=com.google.ai.... It's a demo app for Google's Edge project: https://ai.google.dev/edge
The S25 (edge) runs this very well. 29 tok/s for E2B.
Re: Gemma 4 on iPhone
#178Earlier quoted context omitted.
> It's widely understood that the big players are making profit on inference. I love the whole “they are making money if you ignore training costs” bit. It is always great to see somebody say something like “if you look at the amount of money that they’re spending it looks bad, but if you look away it looks pretty good” like it’s the money version of a solar eclipse
The reason it matters is that if they are making a profit on inference, then when people use their services more, it cuts their losses. They might even break even eventually and start making a profit without raising the price. But if they're losing money on inference, they will lose more money when people use their services more. There's no way to turn that around at that price.
We don't even have any evidence inference excluding training is actually profitable.
Re: Gemma 4 on iPhone
#179Impressive model, for sure. I've been running it on my Mac, now I get to have it locally in my iPhone? I need to test this. Wait, it does agent skills and mobile actions, all local to the phone? Whaaaat? (Have to check out later! Anyone have any tips yet?) I don't normally do the whole "abliterated" thing (dealignment) but after discovering https://github.com/p-e-w/heretic , I was too tempted to try it with this mode…
> It's also possible to make an MLX version of it, which runs a little faster on Macs
FWIW, I found MLX variants to perform consistently worse (in terms of expected output, not speed) than GGUF in my measurements on my benchmark that matters to me (spam filtering). I used MLX models in LM Studio. GGUF was always slightly better.
Perhaps someone who knows more can pitch in and explain this.
Re: Gemma 4 on iPhone
#180Earlier quoted context omitted.
Realistically, a lot of people do this for porn. In my experience, though, it's necessary to do anything security related. Interestingly, the big models have fewer refusals for me when I ask e.g. "in situation, how do you exploit ?", but local models will frequently flat out refuse, unless the model has been abliterated.
From what I've seen gemma 4 doesn't refuse a lot regarding sex, it only needs little nudging in the right direction sometimes. But it does refuse being critical of the usual topics: israel, islam, trans, or race. So wanting to discuss one of those is the real reason people would use an uncensored model.
So–called uncensored versions simply do not refuse addressing a topic. They do not guarantee an alignment with reality.
A truly uncensored model is impossible as human societies exist under various censorship regimes, anyways.