SnapML
Running ML models, tensor math, and audio feature extraction for models.
Commonly used
MLComponentUsed to integrate machine learning models into a Lens.MLAssetRepresents a machine learning model that can be integrated with an MLComponent.MachineLearningInputPlaceholderControls input data for a neural network used by an MLComponent.OutputPlaceholderProvides output data from the neural network used by an MLComponent.TensorMath
Guides
Scripting ML ComponentAlthough you can configure and run your ML models using the ML Component found in the Lens Studio UI, you can have additional flexibility to fine-tune the way your model is run…SnapML on SPECSTrain a custom object detection model, quantize it to W8A8 and convert it to Qualcomm DLC, then run it with SnapML on SPECS.ML Component OverviewAlthough neural networks have been already widely presented in Lens Studio - in 3D face tracking, segmentation, object tracking, SnapML allows users to create their own Lens fea…ClassificationThis guide teaches you how to utilize Machine Learning model to determine whether a thing of a certain class is detected or not and apply an effect based on this information.
All APIs in this topic
Add components to a scene object and reference assets and other modules with a script input.
Components, assets, and modules
| API | Description |
|---|---|
| MLAsset | Represents a machine learning model that can be integrated with an MLComponent. |
| MLComponent | Used to integrate machine learning models into a Lens. |
Objects and data
| API | Description |
|---|---|
| BasePlaceholder | Base class for Input and Output Placeholders used by MLComponent. |
| Delay | Creates a buffer for the audio data. |
| DelayBuilder | |
| Delta | Local estimate of the derivative of the input data along the selected axis. |
| DeltaBuilder | |
| InputBuilder | Builds InputPlaceHolders for MLComponent. |
| InputPlaceholder | Controls input data for a neural network used by an MLComponent. |
| MachineLearning | |
| MelSpectrogram | Computes a mel scale spectrogram - a spectrogram where the frequencies are converted to the mel scale. |
| MelSpectrogramBuilder | |
| MFCC | Mel-frequency cepstral coefficients. |
| MFCCBuilder | Builder class for the MFCC (Mel Frequency Cepstral Co-efficients). |
| NoiseReduction | Base class for objects representing Script data. |
| NoiseReductionBuilder | Base class for objects representing Script data. |
| OutputBuilder | Builds OutputPlaceholders for MLComponent. |
| OutputPlaceholder | Provides output data from the neural network used by an MLComponent. |
| PitchShifter | Base class for objects representing Script data. |
| PitchShifterBuilder | Base class for objects representing Script data. |
| RotatedRect | |
| Sampler | |
| SamplerBuilder | Class for building Sampler. |
| Spectrogram | Representation the signal strength over time at various frequencies present in a particular waveform. |
| SpectrogramBuilder | |
| TensorMath | |
| Transformer | Applies additional transform processing on data for InputPlaceholders and OutputPlaceholders used with MLComponent. |
| TransformerBuilder | Builds Transformer objects for MLComponent. |
Enums
| API | Description |
|---|---|
| SortOrder | Order that inTensor will be sorted when applying TensorMath.argSortMasked(). |
| TransformerRotation | Rotation types used by TransformBuilder. |