window.ai - Everything about the new Chrome AI feature

The new window.ai Chrome API (currently only available in Chrome Canary) introduces native AI models that are ran locally and completely offline.

Overview of the API

The only model available for use at the moment is Gemini Nano. The new API brings a total of 6 new functions:

• canCreateGenericSession and canCreateTextSession are just checks if you have everything needed to use these features. If the result is 'readily', then you're good to go.
• createGenericSession and createTextSession are used to create sessions, from which you can use the model.
• defaultGenericSessionOptions and defaultTextSessionOptions simply return the default options argument the last 2 functions use.

How do I get it?

It's currently only available on the latest build of Chrome Canary, hidden behind a few internal Chrome flags.

Enable the "Prompt API for Gemini Nano" flag

You can find this flag right here: chrome://flags/#prompt-api-for-gemini-nano

Turn on the "Enable optimization guide on device" flag

This flag is located at chrome://flags/#optimization-guide-on-device-model

Make sure you choose Enabled BypassPerfRequirement.

Install the Chrome component

Head to chrome://components and install the Optimization Guide On Device Model Chrome component, by clicking the Check for update button.

This may take a while because it's downloading the AI model in order to run it completely locally.

If everything is working properly, the page below should work.

Try it online

Thanks to https://github.com/lightning-joyce/chromeai, you can try this feature online in a simple chat environment.

Try turning your internet off, it runs fully offline!

Examples

Here's a simple example of how you can use these new features:

const start = new Date().getTime();
const model = await window.ai.createTextSession();
const result = await model.prompt("What do you think is the meaning of life?");
const end = new Date().getTime();

console.log(result);
console.log(`%cTook ${end - start}ms to generate.`, 'color: lime');

And here are the results I got:

Simpler prompts like "Hello!" can take as little as 50ms to complete. It also depends on your hardware and session properties.

Conclusion

The new window.ai API is a game changer because it runs locally and completely offline. This feature is set to become a web browser standard, which, in the future, will enable developers to use AI models that don't rely on a third party API.