Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Gemma Gem – AI model embedded in a browser – no API keys, no cloud

github.com

Post reply on HN
Print view View original

11–20 of 33 posts

Re: Show HN: Gemma Gem – AI model embedded in a browser – no API keys, no cloud

#11

Post by veunes »

avaer wrote: post #2

There's also the Prompt API, currently in Origin Trial, which supports this api surface for sites: https://developer.chrome.com/docs/ai/prompt-api I just checked the stats: Model Name: v3Nano Version: 2025.06.30.1229 Backend Type: GPU (highest quality) Folder size: 4,072.13 MiB Different use case but a similar approach. I expect that at some point this will become a native web feature, but not anytime soon, since the…

That’s exactly where we’re headed. Architecturally it makes zero sense to spin up an LLM in every app's userspace. Since we have dedicated NPUs and GPUs now, we need a unified system-level orchestrator to balance inference queues across different programs - exactly how the OS handles access to the NIC or the audio stack. The browser should just be making an IPC call to the system instead of hauling its own heavy inference engine along for the ride

Re: Show HN: Gemma Gem – AI model embedded in a browser – no API keys, no cloud

#12

Post by jillesvangurp »

veunes wrote: post #10

It’s a neat idea, but giving a 2B model full JS execution privileges on a live page is a bit sketchy from a security standpoint. Plus, why tie inference to the browser lifecycle at all? If Chrome crashes or the tab gets discarded, your agent's state is just gone. A local background daemon with a "dumb" extension client seems way more predictable and robust fwiw

There's indexed db, opfs, etc. Plenty of ways to store stuff in a browser that will survive your browser restarting. Background daemons don't work unless you install and start them yourself. That's a lot of installation friction. The whole point of a browser app is that you don't have to install stuff.

And what you call sketchy is what billions of people default to every day when they use web applications.

Re: Show HN: Gemma Gem – AI model embedded in a browser – no API keys, no cloud

#13

Post by jillesvangurp »

emregucerr wrote: post #4

I would love to see someone build it as some kind of an SDK. App builders could use it as a local LLM plugin when dealing with data involving sensitive information. It's usually too much when an app asks someone to setup a local LLM but this I believe could solve that problem?

It's not too hard to code together with an LLM. I've been playing with small embeddings models in browsers in the last weeks. You don't really need that much. The limitation is that these things are fairly limited and slow to begin with and they run slower in a browser even with webgpu. But you can do some cool stuff. Adding an LLM is just more of the same.

If you want to see an example of this, https://querylight.tryformation.com/ is where I put my search library and demo. It does vector search in the browser.

Re: Show HN: Gemma Gem – AI model embedded in a browser – no API keys, no cloud

#14

Post by fastball »

eric_khun wrote: post #9

it would be awesome if a local model would be directly embeded to chrome and developer could query them. Anyone know if this is somehow possible without going through an extension?

https://developer.chrome.com/docs/ai/get-started

Re: Show HN: Gemma Gem – AI model embedded in a browser – no API keys, no cloud

#15

Post by shawabawa3 »

veunes wrote: post #10

It’s a neat idea, but giving a 2B model full JS execution privileges on a live page is a bit sketchy from a security standpoint. Plus, why tie inference to the browser lifecycle at all? If Chrome crashes or the tab gets discarded, your agent's state is just gone. A local background daemon with a "dumb" extension client seems way more predictable and robust fwiw

> but giving a 2B model full JS execution privileges on a live page is a bit sketchy from a security standpoint.

Every webpage I've ever visited has full JS execution privileges and I trust half of them less than an LLM

Re: Show HN: Gemma Gem – AI model embedded in a browser – no API keys, no cloud

#17

Post by saagarjha »

shawabawa3 wrote: post #15
veunes wrote: post #10

It’s a neat idea, but giving a 2B model full JS execution privileges on a live page is a bit sketchy from a security standpoint. Plus, why tie inference to the browser lifecycle at all? If Chrome crashes or the tab gets discarded, your agent's state is just gone. A local background daemon with a "dumb" extension client seems way more predictable and robust fwiw

> but giving a 2B model full JS execution privileges on a live page is a bit sketchy from a security standpoint. Every webpage I've ever visited has full JS execution privileges and I trust half of them less than an LLM

Note that every webpage does not have full JS execution privileges on other parts of the web.

Re: Show HN: Gemma Gem – AI model embedded in a browser – no API keys, no cloud

#18

Post by [deleted] »

saagarjha wrote: post #17
shawabawa3 wrote: post #15

Earlier quoted context omitted.

> but giving a 2B model full JS execution privileges on a live page is a bit sketchy from a security standpoint. Every webpage I've ever visited has full JS execution privileges and I trust half of them less than an LLM

Note that every webpage does not have full JS execution privileges on other parts of the web.

[deleted]

Re: Show HN: Gemma Gem – AI model embedded in a browser – no API keys, no cloud

#19

Post by michaelbuckbee »

avaer wrote: post #2

There's also the Prompt API, currently in Origin Trial, which supports this api surface for sites: https://developer.chrome.com/docs/ai/prompt-api I just checked the stats: Model Name: v3Nano Version: 2025.06.30.1229 Backend Type: GPU (highest quality) Folder size: 4,072.13 MiB Different use case but a similar approach. I expect that at some point this will become a native web feature, but not anytime soon, since the…

FWIW - I did a real world experiment pitting the built in Gemini Nano vs a free equivalent from OpenRouter (server call) and the free+server side was better in literally every performance metric.

That's not to say that the in browser isn't valuable for privacy+offline, just that the standard case currently is pretty rough.

https://sendcheckit.com/blog/ai-powered-subject-line-alterna...

Re: Show HN: Gemma Gem – AI model embedded in a browser – no API keys, no cloud

#20

Post by xkbear89 »

emregucerr wrote: post #4

I would love to see someone build it as some kind of an SDK. App builders could use it as a local LLM plugin when dealing with data involving sensitive information. It's usually too much when an app asks someone to setup a local LLM but this I believe could solve that problem?

[flagged]
Post reply on HN