Skip to main content

ASR Module

Add speech-to-text to SPECS Lenses with the ASR Module: 40+ languages, mixed-language input, translation, content types, and accuracy modes.

  • Snapchat: not supported
  • SPECS: fully supported
  • Camera Kit Web: not supported
  • Camera Kit Mobile: not supported
  • SnapchatNot supported
  • SPECSFully supported
  • Camera Kit WebNot supported
  • Camera Kit MobileNot supported

Platform details

  • Snapchat — Not supported.

  • SPECS — Fully supported.

  • Camera Kit Web — Not supported.

  • Camera Kit Mobile — Not supported.

The AsrModule (automatic speech recognition) adds speech-to-text to SPECS Lenses. It supports 40+ languages, including mixed language input, and delivers high accuracy across various accents and non-native speakers. This page is for developers who want a SPECS Lens to turn what the wearer says into text.

When to use the ASR Module

Use the ASR Module for dictation, transcription, and chat input, such as sending what the wearer says to an AI model as a question. When the wearer speaks more than one language, the ASR Module handles the mix automatically.

The ASR Module isn't reliable for voice commands or keyword spotting. Partial transcripts keep changing, final transcripts arrive late, short command words are often transcribed wrongly, and homophones ("two" and "to") trigger false matches. For commands, use a pinch, a button, or gaze instead. The ASR Module replaces the VoiceML Module for transcription on SPECS; voice-command features of VoiceML have no ASR equivalent.

Requirements for the ASR Module

  • Lens Studio v5.9.0 or later.
  • Snap OS v5.61 or later.
  • An internet connection: transcription streams over the internet. Without a connection, the session reports AsrModule.AsrStatusCode.NoInternet.

The ASR Module works in the Lens Studio Preview and on SPECS. The language, secondaryLanguages, and contentType options need Lens Scripting Version 384 or later.

The ASR Module and the microphone

Using the ASR Module turns on the Voice audio input profile, which applies Echo Cancellation and Bystander Speech Rejection to the microphone. See Audio on SPECS.

For permissions, the ASR Module counts as a Microphone sensitive sensor. A Lens that combines the ASR Module with internet access, for example through a Web Socket, falls under Transparent Permission.

Add the ASR Module to a script

  1. Create a new TypeScript or JavaScript file in the Asset Browser.

  2. Drag the file into the scene.

  3. Open the script for editing.

  4. Add the following line to include the AsrModule:

    const asrModule: AsrModule = require('LensStudio:AsrModule');

Handle ASR transcription updates and errors

The ASR Module reports transcription updates and errors through callbacks. A transcription update carries a TranscriptionUpdateEvent with the text and whether it isFinal. An error carries an AsrStatusCode: InternalError, Unauthenticated, or NoInternet (the enum also has Success).

const onTranscriptionUpdate = (
eventArgs: AsrModule.TranscriptionUpdateEvent
) => {
print(
`onTranscriptionUpdateCallback text=${eventArgs.text}, isFinal=${eventArgs.isFinal}`
);
};

const onTranscriptionError = (errorCode: AsrModule.AsrStatusCode) => {
print(`onTranscriptionErrorCallback errorCode: ${errorCode}`);
switch (errorCode) {
case AsrModule.AsrStatusCode.InternalError:
print('stopTranscribing: Internal Error');
break;
case AsrModule.AsrStatusCode.Unauthenticated:
print('stopTranscribing: Unauthenticated');
break;
case AsrModule.AsrStatusCode.NoInternet:
print('stopTranscribing: No Internet');
break;
}
};

Set the ASR transcription options

The AsrTranscriptionOptions object lets you configure the ASR session. To create options:

const options = AsrModule.AsrTranscriptionOptions.create();
options.silenceUntilTerminationMs = 1000;
options.mode = AsrModule.AsrMode.HighAccuracy;
options.onTranscriptionUpdateEvent.add((eventArgs) =>
onTranscriptionUpdate(eventArgs)
);
options.onTranscriptionErrorEvent.add((errorCode) =>
onTranscriptionError(errorCode)
);
OptionDefaultWhat it does
silenceUntilTerminationMs750Milliseconds of detected silence after which the current sentence or fragment is marked as final.
modeAsrModule.AsrMode.HighAccuracyThe operation mode of the session: HighAccuracy, Balanced, or HighSpeed.
language"auto"The language of the transcription in ISO 639 format. "auto" detects the spoken language.
secondaryLanguages[]Other languages, in ISO 639 format, that the speaker is expected to switch to.
contentTypeAsrModule.AsrContentType.TextThe kind of content the speech carries, which decides how the transcription is formatted.
onTranscriptionUpdateEventThe event triggered by transcription updates.
onTranscriptionErrorEventThe event triggered by transcription errors.

Once a transcription is finalized (isFinal is true), a new transcription begins within the same session. This process continues until you stop the session.

The mode option trades speed for accuracy:

  • HighAccuracy: higher accuracy at the cost of speed.
  • Balanced: a balance between accuracy and speed.
  • HighSpeed: faster transcription at the cost of accuracy.

Set the ASR language and secondary languages

By default, the ASR Module detects the spoken language. To name the language instead, set language to an ISO 639 code. If the speaker is expected to switch to other languages, list them in secondaryLanguages: this nudges the model toward those languages and can improve accuracy when several are spoken.

// The speaker mostly talks in English, but may switch to Italian or Spanish.
options.language = 'en';
options.secondaryLanguages = ['it', 'es'];

Translate speech with the ASR Module

To translate speech, set contentType to AsrModule.AsrContentType.Translation and language to the language to translate into. With Translation, language is the target language and "auto" isn't a valid value.

// Translate whatever is spoken into Spanish.
const translationOptions = AsrModule.AsrTranscriptionOptions.create();
translationOptions.contentType = AsrModule.AsrContentType.Translation;
translationOptions.language = 'es';

Format ASR transcriptions with a content type

The contentType option tells the ASR Module what the speech is expected to carry, so the transcription is processed and formatted to match, for example to fill in a text field of that type.

Content typeTranscription
Text (default)General speech, transcribed as text.
TranslationSpeech translated into language.
SearchA short query, such as a search term: lowercase, with no punctuation.
UrlA URL.
EmailAn email address.
NumericA number.
PhoneA phone number.
// Format the transcription as a URL.
const urlOptions = AsrModule.AsrTranscriptionOptions.create();
urlOptions.contentType = AsrModule.AsrContentType.Url;

Start and stop ASR transcribing

Start the session from an OnStartEvent handler, not from onAwake. To start transcribing, pass your options object to startTranscribing:

asrModule.startTranscribing(options);

If you call startTranscribing again while a session is active, the ASR Module cancels the current session and starts a new one.

To stop, call stopTranscribing. It discards the current session and returns a Promise that resolves once transcribing has stopped:

asrModule.stopTranscribing().then(() => {
print('Transcribing stopped');
});

Full ASR Module script

This script starts an ASR session when the scene starts and stops it when its Scene Object is destroyed.

@component
export class AsrExample extends BaseScriptComponent {
private asrModule: AsrModule = require('LensStudio:AsrModule');

onAwake(): void {
// Start the session once the scene has started, not in onAwake.
this.createEvent('OnStartEvent').bind(() => this.startSession());
// Stop the session when the script's object is destroyed.
this.createEvent('OnDestroyEvent').bind(() => this.stopSession());
}

private onTranscriptionUpdate(eventArgs: AsrModule.TranscriptionUpdateEvent) {
print(
`onTranscriptionUpdateCallback text=${eventArgs.text}, isFinal=${eventArgs.isFinal}`
);
}

private onTranscriptionError(errorCode: AsrModule.AsrStatusCode) {
print(`onTranscriptionErrorCallback errorCode: ${errorCode}`);
switch (errorCode) {
case AsrModule.AsrStatusCode.InternalError:
print('stopTranscribing: Internal Error');
break;
case AsrModule.AsrStatusCode.Unauthenticated:
print('stopTranscribing: Unauthenticated');
break;
case AsrModule.AsrStatusCode.NoInternet:
print('stopTranscribing: No Internet');
break;
}
}

private startSession(): void {
const options = AsrModule.AsrTranscriptionOptions.create();
options.silenceUntilTerminationMs = 1000;
options.mode = AsrModule.AsrMode.HighAccuracy;
options.onTranscriptionUpdateEvent.add((eventArgs) =>
this.onTranscriptionUpdate(eventArgs)
);
options.onTranscriptionErrorEvent.add((errorCode) =>
this.onTranscriptionError(errorCode)
);

this.asrModule.startTranscribing(options);
}

private stopSession(): void {
this.asrModule.stopTranscribing().then(() => {
print('Transcribing stopped');
});
}
}