Show HN: Code to run Gemini (Nano) locally on desktop/Chrome

#1

Post by notnotrishi » Wed, Jun 26, 2024, 7:01 PM UTC

Chrome Canary (nightly build for devs) now has Gemini LLM inbuilt. This is just some simple code/demo to enable and use that feature.

Show HN: Code to run Gemini (Nano) locally on desktop/Chrome
github.com

Re: Show HN: Code to run Gemini (Nano) locally on desktop/Chrome

#2

Post by DownrightNifty » Thu, Jun 27, 2024, 7:27 AM UTC

Instructions didn't work for me. Tried on Windows 10 and macOS 14.3 (x86) with Chrome Canary. The "On Device Model" component doesn't show up.

Re: Show HN: Code to run Gemini (Nano) locally on desktop/Chrome

#3

Post by notnotrishi » Thu, Jun 27, 2024, 6:46 PM UTC

DownrightNifty wrote: post #2

Instructions didn't work for me. Tried on Windows 10 and macOS 14.3 (x86) with Chrome Canary. The "On Device Model" component doesn't show up.

yea, it's a bit tricky. this came in from google team today:

> To get back on track, make sure you're using Chrome version 128.0.6545.0 or later. If you want the most up-to-date version, try using Chrome Canary or Chrome dev channel.

this document should have the latest changes from Google: https://docs.google.com/document/d/1VG8HIyz361zGduWgNG7R_R8X...

Re: Show HN: Code to run Gemini (Nano) locally on desktop/Chrome

#4

Post by samspenc » Thu, Jun 27, 2024, 8:10 PM UTC

Huh, looking at the code, it's simpler than I thought, basically these are the 2 lines that enable the request-response functionality:

  const session = await window.ai.createTextSession();
  const response = await session.prompt(prompt);

Re: Show HN: Code to run Gemini (Nano) locally on desktop/Chrome

#5

Post by notnotrishi » Thu, Jun 27, 2024, 10:22 PM UTC

samspenc wrote: post #4

Huh, looking at the code, it's simpler than I thought, basically these are the 2 lines that enable the request-response functionality: const session = await window.ai.createTextSession(); const response = await session.prompt(prompt);

exactly! also, one can set parameters etc. if they'd like.

also, after this post, i was able to do some tinkering and run it outside chrome, which i think would be more useful. hope they release the weights and make it open

Re: Show HN: Code to run Gemini (Nano) locally on desktop/Chrome

#6

Post by DownrightNifty » Sat, Jun 29, 2024, 6:45 PM UTC

notnotrishi wrote: post #5

samspenc wrote: post #4

Huh, looking at the code, it's simpler than I thought, basically these are the 2 lines that enable the request-response functionality: const session = await window.ai.createTextSession(); const response = await session.prompt(prompt);

exactly! also, one can set parameters etc. if they'd like. also, after this post, i was able to do some tinkering and run it outside chrome, which i think would be more useful. hope they release the weights and make it open

You should share this with us! If you have a blog, that'd be a really interesting post.