Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

twitter.com

Post reply on HN
Print view View original

71–80 of 194 posts

Re: Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

#71

Post by Rauchg »

lolinder wrote: post #65

> The code below is all you need to stream text with Chrome AI and the Vercel AI SDK. ... `chromeai` implements a Provider that uses `window.ai` under the hood Leave it to Vercel to announce `window.ai` on Google's behalf by showing off their own abstraction but not the actual Chrome API. Here's a blog post from a few days ago that shows how the actual `window.ai` API works [0]. The code is extremely simple and reall…

Someone in our community created a provider and I wanted to showcase it.

It’s nice insofar with very little abstraction, runtime, and bundle size overhead, you can easily switch between models without having to learn a new API.

Re: Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

#72

Post by Fuzzwah »

imacomputertoo wrote: post #12
echelon wrote: post #8

I hope we can use LLMs in the browser to censor all ads, clickbait, and annoyances forever. An in-browser LLM will be the ultimate attention preserver if we task it to identify content we don't like and to remove it.

Google will certainly try to prevent that!

It depends if google can transition to making more revenue from harvesting user data to train AI than they do from ads.

Re: Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

#73

Post by flakiness »

So they don't standardize things anymore?

Look at WebNN [1]. It's from Microsoft and is basically DirecttML but they at least pretend to make it a Web thing.

The posture matters. Apple tried to expose Metal through WebGPU [2] then silent-abandoned it. But they had the posture, and other vendors picked it up and made it real.

That won't happen to window.ai until they stop sleepwalking.

[1] https://www.w3.org/TR/webnn/

[2] https://www.w3.org/TR/webgpu/

Re: Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

#74

Post by dchest »

mikeqq2024 wrote: post #69
oefrha wrote: post #35

See https://developer.chrome.com/docs/ai/built-in https://github.com/jeasonstudio/chrome-ai I can’t seem to find public documentation for the API with a cursory search, so https://github.com/jeasonstudio/chrome-ai/blob/ec9e334253713... might be the best documentation (other than directly inspecting the window.ai object in console) at the moment. It’s not really clear if the Gemini Nano here is Nano-1 (1.8B) or Nano-2…

1.8B/3.25B is still too much for edge devices. Ideally tens or hundreds mega would be ok. Is there an option to change the builtin Gemini Nano to other smaller models? By the way, haven't touch the lastest JS code for a while, what does this new syntax mean: "import { chromeai } " Also not get the textStream code: for await (const textPart of textStream) { result = textPart; } does result get override for each loop s…

That's just some supplemental code. The actual API is:

    const session = await window.ai.createTextSession()
    const outputText = await session.prompt(inputText)
That's all there is for now (createGenericSession does the same at this time, and there are canCreateTextSession/canCreateGenericSession).

Re: Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

#75

Post by oefrha »

mikeqq2024 wrote: post #69
oefrha wrote: post #35

See https://developer.chrome.com/docs/ai/built-in https://github.com/jeasonstudio/chrome-ai I can’t seem to find public documentation for the API with a cursory search, so https://github.com/jeasonstudio/chrome-ai/blob/ec9e334253713... might be the best documentation (other than directly inspecting the window.ai object in console) at the moment. It’s not really clear if the Gemini Nano here is Nano-1 (1.8B) or Nano-2…

1.8B/3.25B is still too much for edge devices. Ideally tens or hundreds mega would be ok. Is there an option to change the builtin Gemini Nano to other smaller models? By the way, haven't touch the lastest JS code for a while, what does this new syntax mean: "import { chromeai } " Also not get the textStream code: for await (const textPart of textStream) { result = textPart; } does result get override for each loop s…

These are 4-bit models.

Re: Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

#76

Post by ori_b »

Fuzzwah wrote: post #72
imacomputertoo wrote: post #12

Earlier quoted context omitted.

Google will certainly try to prevent that!

It depends if google can transition to making more revenue from harvesting user data to train AI than they do from ads.

The AI will contain ads. Here's the prototype:

https://www.anthropic.com/news/golden-gate-claude

Re: Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

#77

Post by richardw »

Going a touch further: make it a pluggable local model. Browser fetches the first 10 links from google in the background, watches the YouTube video, hides the Google ads, presents you with the results.

Now not only can Google front the web pages who feed them content they make summaries from, but the browser can front Google.

“Your honour, this is just what Google has been saying is a good thing. We just moved it to the edge. The users win, no?”

Re: Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

#78

Post by morkalork »

swatcoder wrote: post #27
foxfired wrote: post #15

Today I finally clicked on that "Create theme with AI" on chrome's default page. I'm really having a hard time trying to differentiate it with selecting any random theme. At this point I'm going to create an image generator that's just an api to return random images from pixabay. pix.ai (opensource of course)

So many AI applications right now are just buzzword plays like that, burning hundreds of watts and seconds of latency on features that are already solved better and cheaper with traditional programs. But lots of both stakeholders and users currently value the "magic" itself over anything practical.

It's going to be like the tide of "we saved $XYZ by moving to the cloud" articles followed by the inevitable "we saved 10x $XYZ by moving back to on prem". We just aren't at that 2nd half of the cycle yet, but it's coming. All that LLM processing isn't free.

Re: Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

#79

Post by richardw »

onion2k wrote: post #36

My first impression is that this should enable approximately what Apple is doing with their AI strategy (local on-device first, then filling back to a first party API, and finally something like ChatGPT), but for web users. Having it native in the browser could be really positive for a lot of use cases depending on whether the local version can do things like RAG using locally stored data, and generate structured inf…

Apple might have an advantage given they’ll have custom hardware to drive it, and the ability to combine data from outside the browser with data inside it. But it’s an interesting idea.

Re: Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

#80

Post by fyrn_ »

flakiness wrote: post #73

So they don't standardize things anymore? Look at WebNN [1]. It's from Microsoft and is basically DirecttML but they at least pretend to make it a Web thing. The posture matters. Apple tried to expose Metal through WebGPU [2] then silent-abandoned it. But they had the posture, and other vendors picked it up and made it real. That won't happen to window.ai until they stop sleepwalking. [1] https://www.w3.org/TR/webnn/…

Don't have to when you they have achived a functionally total monopoloy
Post reply on HN