window.ai - Everything about the new Chrome AI feature

About 1 week ago, Google quietly dropped that they will be releasing an AI feature in Chrome and that's finally here.

Overview of the API

The new API brings a total of 6 new functions:

• canCreateGenericSession and canCreateTextSession are just checks if you have everything needed to use these features. If the result is 'readily', then you're good to go.
• createGenericSession and createTextSession are used to create sessions, from which you can use the model.
• defaultGenericSessionOptions and defaultTextSessionOptions simply return the default options argument the last 2 functions use.

How do I get it?

It's currently only available on the latest build of Chrome Canary, hidden behind a few internal Chrome flags.

Enable the "Prompt API for Gemini Nano" flag

You can find this flag right here: chrome://flags/#prompt-api-for-gemini-nano

Turn on the "Enable optimization guide on device" flag

This flag is located at chrome://flags/#optimization-guide-on-device-model

Make sure you choose Enabled BypassPerfRequirement.

Install the Chrome component

Head to chrome://components/ and install the Optimization Guide On Device Model Chrome component, by clicking the Check for update button.

This may take a while because it's downloading the AI model in order to run it completely locally.

If everything is working properly, the page below should work.

Try it online

Thanks to https://github.com/lightning-joyce/chromeai, you can try this feature online in a simple chat environment.

Try turning your internet off, it runs fully offline!

Examples

Here's a simple example of how you can use these new features:

const start = new Date().getTime();
const model = await window.ai.createTextSession();
const result = await model.prompt("What do you think is the meaning of life?");
const end = new Date().getTime();

console.log(result);
console.log(`%cTook ${end - start}ms to generate.`, 'color: lime');

And here are the results I got:

Simpler prompts like "Hello!" can take as little as 50ms to complete. It also depends on your hardware and session properties.

Conclusion

The new chrome.ai API is a game changer because it runs locally and completely offline. This feature is set to become a web browser standard, which, in the future, will enable developers to use AI models that don't rely on a third party API.