Live data from Hacker News

Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference

gizmoweek.com

181–190 of 196 posts

Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference

#181

Earlier quoted context omitted.

I think Apple will become increasingly draconian about LLMs. Very soon people won't need to buy many of their apps. They can just make them. This threatens Apple's entire business model.

It came out in the Epic trial that 90% of App Store revenue comes from in app purchases of loot boxes and other pay to win mechanics. Apple doesn’t care about revenue from a random TODO app.

From games? Why would people pay to play those games when they can make their own?

Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference

#182
post #44

Earlier quoted context omitted.

I think Apple will become increasingly draconian about LLMs. Very soon people won't need to buy many of their apps. They can just make them. This threatens Apple's entire business model.

But… why would I put the effort into getting an llm to make me an app when a there’s an existing app that I don’t have to maintain? I don’t want to have to make every app I use?

It depends on the value of your time relative to the price of the application. Many of us here are well paid and time poor. It makes sense to pay for well built apps. For the world, we are truly the top 1%. 99% of people would be happy to spend a few hours building an app they need to occasionally maintain. Especially if the app is subscription based.

We should also remember that the effort of building and maintaining apps is dropping precipitously as LLMs get smarter, faster, and cheaper. OpenClaw signalled the direction in which we're heading, and within a year, Anthropic will no doubt have cheap and competent agents which can handle the maintenance autonomously in the background.

This is why SaaS valuations are getting hammered.

Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference

#183

Earlier quoted context omitted.

It came out in the Epic trial that 90% of App Store revenue comes from in app purchases of loot boxes and other pay to win mechanics. Apple doesn’t care about revenue from a random TODO app.

From games? Why would people pay to play those games when they can make their own?

You’re not going to make a game using AI.

Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference

#184
post #179

Earlier quoted context omitted.

An LLM is just computer function that predicts next word based on the input you give it. It doesn't make any difference what the input is (e.g. please respond in style X) - the function doesn't change, and the statistical signature of how it works will still be there. If you don't believe me, try it for yourself. Ask an AI to generate some text and give it to the AI detector below (paste your text, then click on scan…

I can't use that linked app, paywall immediately. Unlike the person you were replying to here[0], I do not claim that this is impossible: LLM is indeed just computer function that does stats. And our brains are just electro-chemistry that does stats. This is why stylometric analysis of human writing is a thing. My previous experience with things such as you have linked to, is they used to be quite poor. I assume they…

> I assume they're better since then, but then again so are the models.

Yes, but "better" means different things for each of these.

Detectors are trying to get better at distinguishing human from LLM-generated text.

LLMs are being improved to generate more useful (and benchmark maxxing) outputs, not to attempt to avoid detection.

LLMs are in fact explicitly trained to be as predictable as possible. The training goal is to minimize continuation prediction errors, which means they are in effect being trained to generate output where each word can be predicted by what came before it (which we can contrast to a human who tries to spice it up and keep it interesting by not being too predictable!).

RL post-training, which is especially used for computer code and math, is going to change this word-by-word predictability (detectability) a bit since the focus is now on a longer term goal rather than next word, but to some extent you could also view it as just steering/narrowing the output of the model towards that goal, not totally overriding the next-word statistics.

I don't know if there are AI detectors specifically trained to detect AI code rather than prose, but I'd expect that is more difficult to do, both because of the RL factor, and because computer code is so predictable in the first place - adhering to rigid syntax etc.

Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference

#185

For those who would like an example of its output, I'm currently working through creating a small, free (cc0, public domain) encyclopedia (just a couple of thousand entries) of core concepts in Biology and Health Sciences, Physical Sciences, and Technology. Each entry is being entirely written by Gemma 4:e4b (the 10 GB model.) I believe that this may be slightly larger than the size of the model that runs locally on…

What's your goal? Do you have a project you want the encyclopedia for?

I've now published the encyclopedia.[1] I wrote the goals on that page, which are educational. Or you can use it to judge the state of Gemma 4 and its knowledge of these areas.

[1] https://stateofutopia.com/encyclopedia/

Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference

#186
post #143

Earlier quoted context omitted.

Are the Apple neural engines even a practical target of LLMs? Maybe not strictly impossible, but ANE was designed with an earlier, pre-LLM style of ML. Running LLMs on ANE (e.g. via Core ML) possible in theory, but the substantial model conversion and custom hardware tuning required makes for a high hurdle IRL. The LLM ecosystem standardized around CPU/GPU execution, and to date at least seems unwilling to devote res…

It is possible but requires a very specific model design to utilize. As this reverse engineering effort has shown [0] "The ANE is not a GPU. It’s not a CPU. It’s a graph execution engine." To build one requires using a specific pipeline specifically for CoreML [1]. [0] https://maderix.substack.com/p/inside-the-m4-apple-neural-en... [1] https://developer.apple.com/documentation/coreml

That's the best "what is ANE, really?" investigation / explanation I've seen. Directly lays out why LLMs aren't an ideal fit, its "convolution engine" architecture, the need for feeding ANE deep operation sequence plans / graphs (and the right data sizes) to get full performance, the fanciful nature of Apple's performance claims (~2x actually achievable, natch), and the (superior!) hard power gating... just _oodles_ of insight.

Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference

#187

Earlier quoted context omitted.

From games? Why would people pay to play those games when they can make their own?

You’re not going to make a game using AI.

Not yet, which is the point of this discussion.

Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference

#188

Earlier quoted context omitted.

Huh I didn't see those instructions when I tried it last week. Must not have looked closely enough. I do remember it not having NPU support (confirmed by other people) back at the Gemma 3 launch a while ago.

It won't even let you try Gemma4 until you install a beta update to AICore as of today.

Where do you see that? In my app I can see both AICore and non-AICore versions.

Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference

#189
post #98

Is it me, or does the article sound like LLM output? The pattern "It's not mere X — it's Y", occurs like 4 times in the text :v

I don't care if it's written by an LLM. The problem with the article is the complete lack of details. No benchmarks on the iPhone capable models. No details, whatsoever. Human or LLM - the article is a whole lot of nothing.

The problem is not authorship. It's the lack of substance

Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference

#190
post #98

Is it me, or does the article sound like LLM output? The pattern "It's not mere X — it's Y", occurs like 4 times in the text :v

I don't care if it's written by an LLM. The problem with the article is the complete lack of details. No benchmarks on the iPhone capable models. No details, whatsoever. Human or LLM - the article is a whole lot of nothing.

This is just prompting an LLM and just dumping it on the site (which is clearly what is happening here, all the articles show the same signs of AI output, no human writing, no style, as far as I can tell).

If this is the level of care that goes into news articles, then we're doomed. What will ultimately happen is that AI summarizes AI articles, which got summarized from another AI article, which got summarized from another AI article, .. and after enough rewriting all facts will be gone from articles. I don't care to read this slop, and I'm shocked people are so readily accepting this new state of affairs.

Post reply on HN