Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

twitter.com

Post reply on HN
Print view View original

101–110 of 194 posts

Re: Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

#102

Post by rfoo »

richardw wrote: post #100
jchw wrote: post #89

Earlier quoted context omitted.

Apple may have a bit of a lead in getting it actually deployed end-to-end but given the number of times I've heard "AI accelerator" in reference to mobile processors I'm pretty sure that silicon with 'NPUs' are probably all over the place already, and if they're not, they certainly will be, for better or worse. I've got a laptop with a Ryzen 7040, which apparently has XDNA processors in it. I haven't a damn clue how…

For sure many devices will have them, but the trick will be to build this local web model in a way that leverages all of the local chips. Apple’s advantage is in not having to worry about all that. It has a simpler problem and better access to real local data. Give my personal local data to a model running in the browser? Just feels a bit more risky.

> in a way that leverages all of the local chips

Which, in a way, is similar to building a browser leveraging all of the local GPUs to do render and HW-accelerated video decoding.

Is Safari on Apple Silicon better than Chrome on random Windows laptop for playing YouTube in the last 5 years? Hardly.

Re: Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

#103

Post by visarga »

makeitdouble wrote: post #54
sheepscreek wrote: post #44

Earlier quoted context omitted.

Why is it brain rot? It still blows my mind that it’s even possible for a low-power device to talk and behave like us. For all practical purposes, LLMs pass the Turing test. This a major leap forward in human innovation and engineering. IMO, this could be as influential as the adoption of electricity/setting up of the power grid.

I share a bit of parent's skepticism: the possibilities are infinite （as many things really), but do we need to dive in head first and sprinkle "AI" dust everywhere just in case some of it could be useful ? For instance I don't need my browser to pass the Turing test. I might need better filtering and better search, but it also doesn't need to be baked in the browser. Your analogy to electricity is interesting: do yo…

Filtering, search, summarization, reranking, and security protection (pishing, data leaks) - the necessary functionality adds up

Re: Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

#104

Post by kortilla »

cqqxo4zV46cp wrote: post #38
riiii wrote: post #29

How the great have fallen. Google announces browser embedded AI and receives nothing but rightful hate and resentment.

Google quite rightfully cares very little about what Hacker News has to say anything.

That’s not the point. It’s an observation of how Google’s lost its status amongst hackers. It’s well into its transition into becoming the next IBM.

Re: Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

#105

Post by cuu508 »

rfoo wrote: post #102
richardw wrote: post #100

Earlier quoted context omitted.

For sure many devices will have them, but the trick will be to build this local web model in a way that leverages all of the local chips. Apple’s advantage is in not having to worry about all that. It has a simpler problem and better access to real local data. Give my personal local data to a model running in the browser? Just feels a bit more risky.

> in a way that leverages all of the local chips Which, in a way, is similar to building a browser leveraging all of the local GPUs to do render and HW-accelerated video decoding. Is Safari on Apple Silicon better than Chrome on random Windows laptop for playing YouTube in the last 5 years? Hardly.

The video plays fine on both, yes, but the Windows laptop, generally speaking, gets hotter, and runs out of battery sooner.

Re: Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

#106

Post by rfoo »

cuu508 wrote: post #105
rfoo wrote: post #102

Earlier quoted context omitted.

> in a way that leverages all of the local chips Which, in a way, is similar to building a browser leveraging all of the local GPUs to do render and HW-accelerated video decoding. Is Safari on Apple Silicon better than Chrome on random Windows laptop for playing YouTube in the last 5 years? Hardly.

The video plays fine on both, yes, but the Windows laptop, generally speaking, gets hotter, and runs out of battery sooner.

Exactly. So feature-wise it can work, and out of our "I proudly spent $2500 on my fancy Apple laptop"-tech bubble, people already learned to settle on something hotter.

Re: Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

#107

Post by simonw »

If this is the API that Google are going with here:

    const model = await window.ai.createTextSession();
    const result = await model.prompt("3 names for a pet pelican");
There's a VERY obvious flaw: is there really no way to specify the model to use?

Are we expecting that Gemini Nano will be the one true model, forever supported by this API baked into the world's most popular browser?

Given the rate at which models are improving that would be ludicrous. But... if the browser model is being invisibly upgraded, how are we supposed to test out prompts and expect them to continue working without modifications against whatever future versions of the bundled model show up?

Something like this would at least give us a fighting chance:

    const supportedModels = await window.ai.getSupportedModels();
    if (supportedModels.includes("gemini-nano:0.4")) {
        const model = await window.ai.createTextSession("gemini-nano:0.4");
        // ...

Re: Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

#108

Post by SquareWheel »

So it's loading an instruct model for inference? That seems a fair bit less useful than a base model, at least for more advanced use cases.

What about running LoRAs, adjusting temperature, configuring prompt templates, etc? It seems pretty early to build something like this into the browser. The technology is still changing so rapidly, it might look completely different in 5 years.

I'm a huge fan of local AI, and of empowering web browsers as a platform, but I'm feeling pretty stumped by this one. Is this a good inclusion at this time? Or is the Chrome team following the Google-wide directive to integrate AI _everywhere_, and we're getting a weird JS API as a result?

At the very least, I hope to see the model decoupled from the interface. In the same way that font-family loads locally installed fonts, it should be pluggable for other local models.

Re: Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

#109

Post by kinlan »

lolinder wrote: post #65

> The code below is all you need to stream text with Chrome AI and the Vercel AI SDK. ... `chromeai` implements a Provider that uses `window.ai` under the hood Leave it to Vercel to announce `window.ai` on Google's behalf by showing off their own abstraction but not the actual Chrome API. Here's a blog post from a few days ago that shows how the actual `window.ai` API works [0]. The code is extremely simple and reall…

Fwiw, this is a very experimental API and we're not really promoting this as we know the API shape will have to change. It's only available by enabling a flag so we're working with a lot of developers via the preview program (link below)

Overview: https://developer.chrome.com/docs/ai/built-in

Sign-up: https://docs.google.com/forms/d/e/1FAIpQLSfZXeiwj9KO9jMctffH...

Post reply on HN