Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 4 on iPhone

apps.apple.com

Post reply on HN
Print view View original

91–100 of 267 posts

Re: Gemma 4 on iPhone

#91

Post by janalsncm »

crazygringo wrote: post #44
janandonly wrote: post #28

OP Here. It is my firm belief that the only realistic use of AI in the future is either locally on-device for almost free, or in the cloud but way more expensive then it is today. The latter option will only bemusedly for tasks that humans are more expensive or much slower in. This Gemma 4 model gives me hope for a future Siri or other with iPhone and macOS integration, “Her” (as in the movie) style.

> or in the cloud but way more expensive then it is today. Why? It's widely understood that the big players are making profit on inference. The only reason they still have losses is because training is so expensive, but you need to do that no matter whether the models are running in the cloud or on your device. If you think about it, it's always going to be cheaper and more energy-efficient to have dedicated cloud ha…

> It's widely understood that the big players are making profit on inference.

If you add in the cost of training, it’s not profitable.

Not including the cost of training is a bit like saying the only cost of a cup of coffee is the paper cup it’s in. The only way OpenAI gets to charge for inference is by selling a product people can’t get elsewhere for much cheaper, which means billions in R&D costs. But because of competition, each model effectively has a “shelf life”.

Re: Gemma 4 on iPhone

#92

Post by skybrian »

jrflowers wrote: post #67
crazygringo wrote: post #44

Earlier quoted context omitted.

> or in the cloud but way more expensive then it is today. Why? It's widely understood that the big players are making profit on inference. The only reason they still have losses is because training is so expensive, but you need to do that no matter whether the models are running in the cloud or on your device. If you think about it, it's always going to be cheaper and more energy-efficient to have dedicated cloud ha…

> It's widely understood that the big players are making profit on inference. I love the whole “they are making money if you ignore training costs” bit. It is always great to see somebody say something like “if you look at the amount of money that they’re spending it looks bad, but if you look away it looks pretty good” like it’s the money version of a solar eclipse

The reason it matters is that if they are making a profit on inference, then when people use their services more, it cuts their losses. They might even break even eventually and start making a profit without raising the price.

But if they're losing money on inference, they will lose more money when people use their services more. There's no way to turn that around at that price.

Re: Gemma 4 on iPhone

#93

Post by lzzqrd »

beeflet wrote: post #41

Isn't this already possible in a much more open-ended way with PocketPal? https://github.com/a-ghorbani/pocketpal-ai https://apps.apple.com/us/app/pocketpal-ai/id6502579498 https://play.google.com/store/apps/details?id=com.pocketpala...

Could you clarify what you mean by 'open-ended' in this context, since both initiatives are essentially open-source?

Re: Gemma 4 on iPhone

#95

Post by nl »

nothinkjustai wrote: post #46
crazygringo wrote: post #44

Earlier quoted context omitted.

> or in the cloud but way more expensive then it is today. Why? It's widely understood that the big players are making profit on inference. The only reason they still have losses is because training is so expensive, but you need to do that no matter whether the models are running in the cloud or on your device. If you think about it, it's always going to be cheaper and more energy-efficient to have dedicated cloud ha…

> It's widely understood that the big players are making profit on inference. Are they? Or are they just saying that to make their offerings more attractive to investors? Plus I think most people using agents for coding are using subscriptions which they are definitely not profitable in. Locally running models that are snappy and mostly as capable as current sota models would be a dream. No internet connection requir…

> Plus I think most people using agents for coding are using subscriptions which they are definitely not profitable in.

Where on earth do people get this idea? Subscriptions that are based around obscure, vendor defined "credits" are the perfect business model for vendors. They can change the amount you can use whenever they want.

It's likely they occasionally make a loss on some users but in general they are highly profitable for AI companies:

> Anthropic last month projected it would generate a 40% gross profit margin from selling AI to businesses and application developers in 2025

and

> OpenAI projected a gross margin of around 46% in 2025, including inference costs of both paying and nonpaying ChatGPT users.

https://archive.is/aKFYZ#selection-1075.0-1083.119

Re: Gemma 4 on iPhone

#96

Post by 1f60c »

buzzerbetrayed wrote: post #65
PullJosh wrote: post #10

This is awesome! 1) I am able to run the model on my iPhone and get good results. Not as good as Gemini in the cloud, but good. 2) I love the “mobile actions” tool calls that allow the LLM to turn on the flashlight, open maps, etc. It would be fun if they added Siri Shortcuts support. I want the personal automation that Apple promised but never delivered. 3) I am so excited for local models to be normalized. I build…

For me the hallucination and gaslighting is like taking a step back in time a couple of years. It even fails the “r’s in strawberry” question. How nostalgic. It’s very impressive that this can run locally. And I hope we will continue to be able to run couple-year-old-equivalent models locally going forward.

Strangely, reasoning is not on by default. If you enable it, it answers as you'd expect.

Re: Gemma 4 on iPhone

#97

Post by dimmke »

buzzerbetrayed wrote: post #65
PullJosh wrote: post #10

This is awesome! 1) I am able to run the model on my iPhone and get good results. Not as good as Gemini in the cloud, but good. 2) I love the “mobile actions” tool calls that allow the LLM to turn on the flashlight, open maps, etc. It would be fun if they added Siri Shortcuts support. I want the personal automation that Apple promised but never delivered. 3) I am so excited for local models to be normalized. I build…

For me the hallucination and gaslighting is like taking a step back in time a couple of years. It even fails the “r’s in strawberry” question. How nostalgic. It’s very impressive that this can run locally. And I hope we will continue to be able to run couple-year-old-equivalent models locally going forward.

I haven't seen anybody else post it in this thread, but this is running on 8GB of RAM. It's not the full Gemma 4 32B model. It's a completely different thing from the full Gemma 4 experience if you were running the flagship model, almost to the point of being misleading.

It's their E2B and E4B variants (so 2B and 4B but also quantized)

https://ai.google.dev/gemma/docs/core/model_card_4#dense_mod...

Re: Gemma 4 on iPhone

#98

Post by devmor »

dolebirchwood wrote: post #86
SL61 wrote: post #81

Earlier quoted context omitted.

LLMs are very helpful for transcribing handwritten historical documents, but sometimes those documents contain language/ideas that a perfectly aligned LLM will refuse to output. Sometimes as a hard refusal, sometimes (even worse) by subtly cleaning up the language. In my experience the latest batch of models are a lot better at transcribing the text verbatim without moralizing about it (i.e. at "understanding" that t…

I have a project where I'm using LLMs to parse data from PDFs with a very complicated tabular layout. I've been using the latest Gemini models (flash and pro) for their strong visual reasoning, and they've generally been doing a really good job at it. My prompt states that their job is to extract the text exactly as it appears in the PDF. One data point to be extracted is the race of each person listed. In one case,…

I was attempting to help someone who runs a small shop selling restored clothing set up a gemini pipeline that would restage images she took of clothing items with bad lighting, backgrounds, etc.

Basically anything that showed any “skin” on a mannequin it would refuse to interact with. Even just a top, unless she put pants on the mannequin.

It was infuriating.

Re: Gemma 4 on iPhone

#99

Post by rotexo »

E4B is pretty good for extracting tables of items from receipt scans and inferring categories, wish this could be called from within a shortcut to just select a photo and add the extracted table to the clipboard

Re: Gemma 4 on iPhone

#100

Post by pmarreck »

peyton wrote: post #79
pmarreck wrote: post #61

Earlier quoted context omitted.

1) Coming up with any valid criticism of Islam at all (for some reason, criticisms of Christianity or Judaism are perfectly allowed even with public models!). 2) Asking questions about sketchy things. Simply asking should not be censored. 3) I don't use it for this, but porn or foul language. 4) Imitating or representing a public figure is often blocked. 5) Asking security-related questions when you are trying to do…

The manufacturing of biologics can be heavily censored to an absurd degree. I don’t know about Gemma 4 in particular.

Really? That's fascinating. Why is that?
Post reply on HN