Installing and running LLM on 8GB RAM PC was one of fun activities last year*1. Now, LLM is integrated with a web browser and user can call it locally through browser`s console.
User can call Gemini Nano model with API in Google Chrome 128 or later. Even it is a small size LLM, essentially same environment can be brought "out-of-the-box" without effort experienced last year.
Blog articles about this topic are already posted here and there,
though they lacks one importance. API is changed bewtween Chrome 128 and 129. Script there are slightly different accordance with API.
This post introduce how to call Chrome integrated Gemini Nano including this difference.
- Chrome at the moment of this post
- Chrome configuration
- Downloading component
- Calling Prompt API
- Reference
Chrome at the moment of this post
In Chrome, release channel*2 is defined. Generally distributed one is called stable version called Stable. There is the other version as Dev for developer and Canary as experimental version. As mentioned, API is different between version 128 and 129. Major version in each channel is
| Stable | 128 |
| Dev | 129 |
| Canary | 130 |
Script to call Gemini Nano should be written accordance with API.
Canary is troublesome. Although It should supports API for 129 or later, calling Gemini Nano on Canary 130 is failed. This is due to unstability of Canary channel, not extra update of API. It is usual that trouble-free program yesterday suddenly crashes today, or tomorrow as reported below.
Actually, calling Gemini Nano on Chrome Canary 130 is failed at this moment.
Chrome configuration
Refer URL below and change values as specified. Chrome should eb rebooted after this.
| Enables optimization guide on device | Enabled BypassPerfRequirement |
|---|---|
chrome://flags/#optimization-guide-on-device-model | |
| Prompt API for Gemini Nano | Enabled |
|---|---|
chrome://flags/#prompt-api-for-gemini-nano | |
 |
 |
Downloading component
From now, operate through Console in Developer Tools of Chrome. Run script and call API there accordance with version 128 or 129 so that API is different accordance with the version of Chrome.
Although console indicates error, it is no problem.
| 128 |
await ai.createTextSession(); |
|---|---|
 | |
| 129 later |
await ai.assistant.create(); |
|---|---|
 | |
After this, access specified URL and push the button "Check for Update". Chrome should be rebooted adter downloading component.
| Optimization Gruide On Device Model |
chrome://components |
|---|---|
 | |
Calling Prompt API
Operate from Console in Developer Tools again. Run script and call API with prompt, Gemini Nano should give back response. Script to start session is different between version 128 and 129. Followings are typical sample code.
| 128 |
const session = await ai.createTextSession();
const result = await session.prompt("hello gemini");
console.log(result); |
|---|---|
| 129 later |
const session = await ai.assistant.create();
const result = await session.prompt("hello gemini");
console.log(result); |
Screenshot indicates Gemini Nano supports not only English, but also Japanese.
In next code, Gemini Nano responds per token.
const stream = session.promptStreaming("2024年9月4日、シアトルの気候はどのような感じですか？");
for await (const chunk of stream){console.log(chunk)};
When testing on your own, please also refer to the documents shown at the end of this document, including information on how to write the Prompt API, as well as information on future API specification changes.
