Audio on SPECS
Configure audio for SPECS Lenses: playback modes (Low Power vs Low Latency), Mix to Snap, and the audio input profiles for the microphone and ASR.
- Snapchat: not supported
- SPECS: fully supported
- Camera Kit Web: not supported
- Camera Kit Mobile: not supported
- SnapchatNot supported
- SPECSFully supported
- Camera Kit WebNot supported
- Camera Kit MobileNot supported
When you develop for SPECS, the Audio Component behaves differently from the standard platform in a few ways: Mix to Snap is always on, playback has a power-saving mode, and the microphone gets audio input profiles. This page is for developers who play or record sound in SPECS Lenses.
Mix to Snap is always on in SPECS
On SPECS, the Mix to Snap feature, which allows recording sound directly into a Snap, is applied to every Audio component automatically. The Mix to Snap setting on the Audio component is ignored.
Set the SPECS playback mode from a script
The playbackMode property of the
AudioComponent can only be set from a script: the Audio component has no
Inspector field for it. SPECS sets every Audio component to
Audio.PlaybackMode LowPower by default. Set the property in an
OnStartEvent handler, as in the examples in this section.
Low Power playback mode
Low Power reduces power usage on the SPECS device, but introduces latency in audio playback. It suits ambient sounds or background music, where a slight delay is acceptable.
- TypeScript
- JavaScript
@component
export class LowPowerAudio extends BaseScriptComponent {
@input
audio: AudioComponent;
onAwake() {
// Set Audio component properties once the scene has started.
this.createEvent('OnStartEvent').bind(() => {
this.audio.playbackMode = Audio.PlaybackMode.LowPower;
});
}
}
// @input Component.AudioComponent audio
// Set Audio component properties once the scene has started.
script.createEvent('OnStartEvent').bind(function () {
script.audio.playbackMode = Audio.PlaybackMode.LowPower;
});
Low Latency playback mode
Low Latency minimizes audio playback latency, but increases power usage on the SPECS device. Use it for audio that needs an immediate reaction, such as feedback for a button press.
- TypeScript
- JavaScript
@component
export class LowLatencyAudio extends BaseScriptComponent {
@input
audio: AudioComponent;
onAwake() {
// Set Audio component properties once the scene has started.
this.createEvent('OnStartEvent').bind(() => {
this.audio.playbackMode = Audio.PlaybackMode.LowLatency;
});
}
}
// @input Component.AudioComponent audio
// Set Audio component properties once the scene has started.
script.createEvent('OnStartEvent').bind(function () {
script.audio.playbackMode = Audio.PlaybackMode.LowLatency;
});
How many audio tracks a SPECS Lens can play
SPECS has no fixed limit on how many audio tracks a Lens can play at once. Each track that plays costs CPU time and memory, though, so a Lens that plays many tracks at the same time can run out of resources. Play only the tracks you need at any moment, and stop tracks you no longer need.
Audio input profiles on SPECS
SPECS applies audio input profiles to the microphone automatically, based on the features your Lens uses. By default, no audio input profile is applied. A profile combines these filters:
- Bystander Speech Rejection: ignores speech that doesn't come from the SPECS wearer, so nearby conversations aren't transcribed while the wearer isn't talking.
- Echo Cancellation: removes the sound from the speakers that the microphone would otherwise pick up, which prevents a feedback loop.
| Profile | Applied when the Lens uses | Filters |
|---|---|---|
| Analysis | Microphone Audio | Echo Cancellation |
| Voice | The ASR Module | Echo Cancellation, Bystander Speech Rejection |
Analysis profile
SPECS applies the Analysis profile when your Lens project uses Microphone Audio. While the Analysis profile is active, the microphone uses Echo Cancellation.
Voice profile
SPECS applies the Voice profile when your Lens project uses the ASR Module (AsrModule). While the Voice profile is active, the microphone uses Echo Cancellation and Bystander Speech Rejection.
If a Lens uses both Microphone Audio and the ASR Module, the Voice profile takes precedence over the Analysis profile.
Related
- Audio Component: add sound to a Lens and set up the Audio component.
- ASR Module: speech-to-text, which turns on the Voice profile.
- Audio Tracks: Microphone Audio, which turns on the Analysis profile.
- Audio Listener: the listener for Spatial Audio.