Skip to main content

MLComponent

Class

Used to integrate machine learning models into a Lens.

  • Available on all platforms
Platform details: Snapchat, SPECS, Camera Kit Web, Camera Kit Mobile fully supported.

Platform details

  • Snapchat — Fully supported.

  • SPECS — Fully supported.

  • Camera Kit Web — Fully supported.

  • Camera Kit Mobile — Fully supported.

Index

Remarks

This component allows developers to use neural networks for processing inputs such as textures or data arrays to produce specific outputs, which could be in the form of processed textures or data alterations. The MLComponent relies on MLAsset that defines the neural network model used. It supports tasks like image classification, object detection, etc.

Example

//@input Asset.MLAsset model
var mlComponent = script.sceneObject.createComponent('MLComponent');
mlComponent.model = script.model;

Extends

Properties

autoRun

autoRun: boolean

Runs the MLComponent automatically when the object or component it's on is enabled.

Since

Lens Scripting Version 161

Default Value

false

enabled

enabled: boolean

If disabled, the Component will stop enacting its behavior.

Default Value

true

Inherited from

Component.enabled


inferenceMode

inferenceMode: InferenceMode

Controls the inference mode that MLComponent will run in. For example, GPU, CPU, etc.

Default Value

MachineLearning.InferenceMode.Auto

isBeingDestroyed

readonly isBeingDestroyed: boolean

Returns true if this Component is currently being destroyed. Becomes true as soon as destruction begins via Component#destroy or destruction of its owning SceneObject, before any OnDestroyEvent callbacks fire.

Since

Lens Scripting Version 364

Example

if (!script.isBeingDestroyed) {
// Component is still fully alive
}

Inherited from

Component.isBeingDestroyed


isEnabledInHierarchy

readonly isEnabledInHierarchy: boolean

Returns true if this Component, its SceneObject, and all of that SceneObjects parents are enabled.

Inherited from

Component.isEnabledInHierarchy


model

model: MLAsset

Binary ML model supplied by the user.

Default Value

null

onLoadingFailed

onLoadingFailed: (error) => void

Bind a function that is triggered when the component fails to load a model.

Parameters

ParameterType
errorstring

Returns

void

Default Value

null

onLoadingFinished

onLoadingFinished: () => void

Function that gets called when model loading is finished.

Returns

void

Default Value

null

onRunningFailed

onRunningFailed: (error) => void

Bind a function that is triggered when the component fails to run a model.

Parameters

ParameterType
errorstring

Returns

void

Default Value

null

onRunningFinished

onRunningFinished: () => void

Function that gets called when the model stops running.

Returns

void

Default Value

null

renderOrder

renderOrder: number

Render order of the MLComponent.

Default Value

0

sceneObject

readonly sceneObject: SceneObject

The SceneObject this component is on.

Inherited from

Component.sceneObject


state

readonly state: ModelState

Returns the current status of the neural network model.


uniqueIdentifier

readonly uniqueIdentifier: string

Since

Lens Scripting Version 176

Inherited from

Component.uniqueIdentifier

Methods

build()

build(placeholders): void

Builds the MLComponent model when all placeholders are determined. Config is an array of Input and Output placeholders.

Parameters

ParameterType
placeholdersBasePlaceholder[]

Returns

void


buildAsync()

buildAsync(placeholders): Promise<void>

Build the MLComponent asynchronously.

Parameters

ParameterType
placeholdersBasePlaceholder[]

Returns

Promise<void>

Since

Lens Scripting Version 323


cancel()

cancel(): void

Stops running the MLComponent. The onRunningFinished callback will not be executed.

Returns

void


destroy()

destroy(): void

Destroys the component.

Returns

void

Inherited from

Component.destroy


getInput()

getInput(name): InputPlaceholder

Returns the InputPlaceholder with the matching name.

Parameters

ParameterType
namestring

Returns

InputPlaceholder


getInputs()

getInputs(): InputPlaceholder[]

Returns an array of InputPlaceholders of the MLComponent's model with default settings. Returns empty array if the model asset is not set.

Returns

InputPlaceholder[]

Since

Lens Scripting Version 143


getOutput()

getOutput(name): OutputPlaceholder

Returns the OutputPlaceholder with the matching name.

Parameters

ParameterType
namestring

Returns

OutputPlaceholder


getOutputs()

getOutputs(): OutputPlaceholder[]

Returns an array of OutputPlaceholders of MLComponent's model with default settings. Returns empty array if the model is not set.

Returns

OutputPlaceholder[]

Since

Lens Scripting Version 143


getSceneObject()

getSceneObject(): SceneObject

Returns the SceneObject the component is attached to.

Returns

SceneObject

Inherited from

Component.getSceneObject


getScheduledEnd()

getScheduledEnd(): FrameTiming

Returns the end time of the scheduled MLComponent run.

Returns

FrameTiming


getScheduledStart()

getScheduledStart(): FrameTiming

Returns the start time of the scheduled MLComponent run.

Returns

FrameTiming


getTransform()

getTransform(): Transform

Returns the Transform this component is attached to.

Returns

Transform

Inherited from

Component.getTransform


getTypeName()

static getTypeName(): TypeName<MLComponent>

Returns the fully-qualified JavaScript type name for this class (for example, "Component.Camera"). This is a static accessor (no instance required) and returns the same value as the instance getTypeName().

Returns

TypeName<MLComponent>

Overrides

Component.getTypeName


getTypeName()

getTypeName(): string

Returns the name of this object's type.

Returns

string

Inherited from

Component.getTypeName


isOfType()

isOfType(type): boolean

Returns true if the object matches or derives from the passed in type.

Parameters

ParameterType
typestring

Returns

boolean

Inherited from

Component.isOfType


isRecurring()

isRecurring(): boolean

Returns true if running is requested on each frame.

Returns

boolean


isSame()

isSame(other): boolean

Returns true if this object is the same as other. Useful for checking if two references point to the same thing.

Parameters

ParameterType
otherScriptObject

Returns

boolean

Inherited from

Component.isSame


runImmediate()

runImmediate(sync): void

Runs the MLComponent once.

Parameters

ParameterType
syncboolean

Returns

void


runScheduled()

runScheduled(recurring, startTiming, endTiming): void

Schedules the MLComponent to run at the start timing and terminate at the end timing. The scheduled running will recur if recurring is true.

Parameters

ParameterType
recurringboolean
startTimingFrameTiming
endTimingFrameTiming

Returns

void


stop()

stop(): void

Stops running the MLComponent.

Returns

void


waitOnLoading()

waitOnLoading(): void

If loading asynchronously, makes the entire system wait until loading is finished.

Returns

void


waitOnRunning()

waitOnRunning(): void

If running asynchronously, makes the entire system wait until the last run is finished.

Returns

void

See also

MLComponent Overview.