Skip to main content

CameraModule.Frame

Class

A camera image, delivered by CameraModule.StreamingSession#onNewFrame or returned by CameraModule.CaptureSession#capture.

  • Snapchat: not supported
  • SPECS: fully supported
  • Camera Kit Web: not supported
  • Camera Kit Mobile: not supported
1 platform note
  • SnapchatNot supported — Available on SPECS only
  • SPECSFully supported
    • Restricted when the Lens uses internet-connected APIs (see Privacy Restrictions).
  • Camera Kit WebNot supported — Available on SPECS only
  • Camera Kit MobileNot supported — Available on SPECS only

Platform details

  • Snapchat — Not supported — Available on SPECS only.

  • SPECS — Fully supported. Restricted when the Lens uses internet-connected APIs (see Privacy Restrictions)

  • Camera Kit Web — Not supported — Available on SPECS only.

  • Camera Kit Mobile — Not supported — Available on SPECS only.

Since Lens Scripting v368Exposes user data

A camera image, delivered by CameraModule.StreamingSession#onNewFrame or returned by CameraModule.CaptureSession#capture.

Index

Remarks

The image is available as raw bytes (CameraModule.Frame#imageData) and as a GPU texture (CameraModule.Frame#texture); each is created on first access. Use raw bytes for ML or upload pipelines that do not need the GPU, and the texture for display, ML components or RemoteMediaModule.

Streaming frames are short-lived: a StreamingSession keeps the pixels of its two most recent frames only. Read imageData or texture inside onNewFrame. On an older frame texture returns null, and so does imageData unless you still hold the array you read from it; that array stays valid. Frames returned by capture() keep their pixels for as long as you reference them.

Example

//@input Component.MeshVisual mesh

let cameraModule = require('LensStudio:CameraModule');
let session = cameraModule.createStreamingSession();

let frameCount = 0;
session.onNewFrame.add(function (frame) {
// Display:
if (frame.texture) script.mesh.mainPass.baseTex = frame.texture;

// Raw bytes, without a GPU round-trip: read them only on the frames you
// process (here one per second), and read them now; older frames are released.
if (frameCount++ % 30 == 0 && frame.imageData) {
let bytes = frame.imageData; // NV21 for Yuv420 streams, Y8 for grayscale
}

// Metadata is not guaranteed on every frame:
let iso = frame.getMetadata(CameraModule.MetadataKey.Iso);
if (iso !== null) print('ISO ' + iso);
});

// Select the largest streaming capability on the RGB camera.
let rgb = cameraModule.getCameras().find(function (c) { return c.type == CameraModule.CameraType.Rgb; });
let stream = rgb && rgb.getStreams().find(function (s) {
return s.operation == CameraModule.Operation.Streaming;
});
if (stream) {
let config = new CameraModule.StreamConfig();
config.stream = stream;
config.minFrameRate = 15; // the camera may run faster, up to maxFrameRate
if (session.start(config) != CameraModule.Result.Success) print('Failed to start');
}

Extends

Properties

height

readonly height: number

Exposes user dataSPECS

Image height in pixels.

Since

Lens Scripting Version 368


imageData

readonly imageData: Uint8Array<ArrayBufferLike>

Exposes user dataSPECS

Raw image bytes, tightly packed with no row padding:

  • Jpeg: the JPEG file.
  • Yuv420: the Y plane (width * height bytes) followed by half-resolution chroma interleaved V then U (NV21, width * height / 2 bytes).
  • Y8: one luma byte per pixel (width * height bytes).

Null on a streaming frame older than the session's two most recent frames, unless you still hold the array you read from it, and on a capture whose image could not be read.

Remarks

The array is created on first access, so frames whose bytes you never read cost no script allocation. Reading imageData again returns the same array while you hold it. Each array you read keeps that frame's bytes until the script engine collects it, so read imageData only on the frames you process. The array shares the frame's buffer; treat it as read-only.

Since

Lens Scripting Version 368


texture

readonly texture: Texture

Exposes user dataSPECS

RGBA texture of the image; grayscale images have R = G = B. Created on first access, so pipelines that only read imageData pay no GPU upload. Null on a streaming frame older than the session's two most recent frames, and when the frame has no image data.

Remarks

Each session reuses one Texture: reading texture on a newer frame, including the next capture, replaces the image in the Texture returned for an earlier frame, and a later session of the same size can reuse it after stop(). To keep an image, keep its imageData or copy the texture. After its session stops, reading texture on a frame you kept decodes the image again into a new Texture, so read it while the session runs.

To read its pixels, read texture inside onNewFrame, keep that Texture, then copy it with ProceduralTextureProvider.createFromTexture from an UpdateEvent and call getPixels on the copy: createFromTexture throws inside onNewFrame and right after capture() resolves. Assigning the texture to a material works inside onNewFrame, and Base64.encodeTextureAsync works on a captured frame right after capture() resolves.

Since

Lens Scripting Version 368


width

readonly width: number

Exposes user dataSPECS

Image width in pixels.

Since

Lens Scripting Version 368

Methods

getMetadata()

getMetadata(key): number

Exposes user dataSPECS

Get a frame metadata value by key.

Parameters

ParameterTypeDescription
keyMetadataKeyMetadata key (e.g. CameraModule.MetadataKey.Iso).

Returns

number

The metadata value, or null if the key is not available for this frame.

Since

Lens Scripting Version 368


getTypeName()

static getTypeName(): TypeName<Frame>

Exposes user dataSPECS

Returns the fully-qualified JavaScript type name for this class (for example, "Component.Camera"). This is a static accessor (no instance required) and returns the same value as the instance getTypeName().

Returns

TypeName<Frame>

Since

Lens Scripting Version 368

Overrides

ScriptObject.getTypeName


getTypeName()

getTypeName(): string

Returns the name of this object's type.

Returns

string

Inherited from

ScriptObject.getTypeName


isOfType()

isOfType(type): boolean

Returns true if the object matches or derives from the passed in type.

Parameters

ParameterType
typestring

Returns

boolean

Inherited from

ScriptObject.isOfType


isSame()

isSame(other): boolean

Returns true if this object is the same as other. Useful for checking if two references point to the same thing.

Parameters

ParameterType
otherScriptObject

Returns

boolean

Inherited from

ScriptObject.isSame