World Query Module
Place virtual objects on real-world surfaces on SPECS with depth-based hit-test raycasting and optional surface classification.
- Snapchat: not supported
- SPECS: fully supported
- Camera Kit Web: not supported
- Camera Kit Mobile: not supported
- SnapchatNot supported
- SPECSFully supported
- Camera Kit WebNot supported
- Camera Kit MobileNot supported
Overview
The World Query Module lets a SPECS Lens place objects on real-world surfaces quickly and accurately. It is a lightweight solution designed for SPECS as a wearable device.
Why use the World Query Module for surface placement
Other ways to find surfaces, such as hitTestWorldMesh or reading depth frames with the Depth Module, can be computationally expensive on a wearable device. The World Query Module provides:
- Instant placement: Fast hit testing for real surfaces
- Lightweight performance: Optimized for wearable devices
- Surface analysis: Depth and normal sampling at specific locations
- Smoothing options: Results can be smoothed across multiple hits
How a World Query hit test works
A World Query Hit Test casts a ray against real surfaces:
- It computes a depth map for the current view.
- It intersects the ray with the depth data to find the hit point.
- It determines the surface position and normal at that point.
- It returns
nullwhen the ray hits nothing, including when the ray falls outside the camera's field of view.
The depth data behind World Query hit tests updates at about 5 Hz, so hit tests work only for static or slowly moving objects. Choose use cases accordingly.
Place an object on a surface with the World Query Module
Before you start
To place objects with the World Query Module, you need:
- A project set up for SPECS, with Interactive Preview enabled.
- Interaction Kit (formerly Spectacles Interaction Kit, SIK) imported and initialized: the example aims the hit test with the user's hand ray.
- A target object to place on surfaces.
How the surface placement example works
The complete surface placement example uses the WorldQueryHit API in four parts:
- It imports Interaction Kit and the World Query Module.
- It creates a hit test session with filtering options and starts it.
- It handles each hit test result by moving the target object to the hit position and orienting it to the surface normal.
- It spawns a copy of the object when the user releases a pinch.
Complete surface placement example
- TypeScript
- JavaScript
// import required modules
import {
InteractorTriggerType,
InteractorInputType,
} from 'InteractionKit.lspkg/Core/Interactor/Interactor';
import { IK } from 'InteractionKit.lspkg/IK';
const WorldQueryModule = require('LensStudio:WorldQueryModule');
const EPSILON = 0.01;
@component
export class NewScript extends BaseScriptComponent {
private primaryInteractor;
private hitTestSession: HitTestSession;
private transform: Transform;
private lastHitResult: any; // Store last hit result for trigger end callback
@input
indexToSpawn: number;
@input
targetObject: SceneObject;
@input
objectsToSpawn: SceneObject[];
@input
filterEnabled: boolean;
onAwake() {
// check the input before starting a hit test session
if (!this.targetObject) {
print('Error: Target Object input is not set');
return;
}
// create new hit session
this.hitTestSession = this.createHitTestSession(this.filterEnabled);
this.transform = this.targetObject.getTransform();
// disable target object when surface is not detected
this.targetObject.enabled = false;
this.setObjectEnabled(this.indexToSpawn);
// Interaction Kit interactors are ready in OnStartEvent, not onAwake
this.createEvent('OnStartEvent').bind(() => {
// Set up trigger end callback
this.setupTriggerEndCallback();
// create update event
this.createEvent('UpdateEvent').bind(this.onUpdate.bind(this));
});
// stop the session (and its depth computation) when the script is destroyed
this.createEvent('OnDestroyEvent').bind(() => {
this.hitTestSession.stop();
});
}
setupTriggerEndCallback() {
// Get all interactors and set up trigger end callbacks
const allInteractors = IK.InteractionManager.getInteractorsByType(
InteractorInputType.All
);
for (const interactor of allInteractors) {
interactor.onTriggerEnd.add(() => {
// Only place object if we have a valid hit result and this is the primary interactor
if (this.lastHitResult && this.primaryInteractor === interactor) {
this.placeObject();
}
});
}
}
placeObject() {
if (!this.lastHitResult) return;
// Copy the plane/axis object
const parent = this.objectsToSpawn[this.indexToSpawn].getParent();
const newObject = parent.copyWholeHierarchy(
this.objectsToSpawn[this.indexToSpawn]
);
newObject.setParentPreserveWorldTransform(null);
// Set position and rotation from last hit
const hitPosition = this.lastHitResult.position;
const hitNormal = this.lastHitResult.normal;
let lookDirection;
if (1 - Math.abs(hitNormal.normalize().dot(vec3.up())) < EPSILON) {
lookDirection = vec3.forward();
} else {
lookDirection = hitNormal.cross(vec3.up());
}
const toRotation = quat.lookAt(lookDirection, hitNormal);
newObject.getTransform().setWorldPosition(hitPosition);
newObject.getTransform().setWorldRotation(toRotation);
}
createHitTestSession(filterEnabled) {
// create hit test session with options
const options = HitTestSessionOptions.create();
options.filter = filterEnabled;
const session = WorldQueryModule.createHitTestSessionWithOptions(options);
// depth computation starts once the session is started
session.start();
return session;
}
onHitTestResult(results) {
if (results === null) {
this.targetObject.enabled = false;
this.lastHitResult = null;
} else {
this.targetObject.enabled = true;
// Store hit result for potential trigger end callback
this.lastHitResult = results;
// get hit information
const hitPosition = results.position;
const hitNormal = results.normal;
//identifying the direction the object should look at based on the normal of the hit location.
let lookDirection;
if (1 - Math.abs(hitNormal.normalize().dot(vec3.up())) < EPSILON) {
lookDirection = vec3.forward();
} else {
lookDirection = hitNormal.cross(vec3.up());
}
const toRotation = quat.lookAt(lookDirection, hitNormal);
//set position and rotation
this.targetObject.getTransform().setWorldPosition(hitPosition);
this.targetObject.getTransform().setWorldRotation(toRotation);
}
}
onUpdate() {
this.primaryInteractor =
IK.InteractionManager.getTargetingInteractors().shift();
if (
this.primaryInteractor &&
this.primaryInteractor.isActive() &&
this.primaryInteractor.isTargeting()
) {
const rayStartOffset = new vec3(
this.primaryInteractor.startPoint.x,
this.primaryInteractor.startPoint.y,
this.primaryInteractor.startPoint.z + 30
);
const rayStart = rayStartOffset;
const rayEnd = this.primaryInteractor.endPoint;
this.hitTestSession.hitTest(
rayStart,
rayEnd,
this.onHitTestResult.bind(this)
);
} else {
this.targetObject.enabled = false;
}
}
setObjectIndex(i) {
this.indexToSpawn = i;
}
setObjectEnabled(i) {
for (let i = 0; i < this.objectsToSpawn.length; i++)
this.objectsToSpawn[i].enabled = i == this.indexToSpawn;
}
}
// import required modules
const Interactor = require('InteractionKit.lspkg/Core/Interactor/Interactor');
const InteractorTriggerType = Interactor.InteractorTriggerType;
const InteractorInputType = Interactor.InteractorInputType;
const IK = require('InteractionKit.lspkg/IK').IK;
const WorldQueryModule = require('LensStudio:WorldQueryModule');
const EPSILON = 0.01;
//@input int indexToSpawn
//@input SceneObject targetObject
//@input SceneObject[] objectsToSpawn
//@input bool filterEnabled
function WorldQueryHitExample() {
// Private variables
var primaryInteractor;
var hitTestSession;
var transform;
var lastHitResult; // Store last hit result for trigger end callback
function onAwake() {
// check the input before starting a hit test session
if (!script.targetObject) {
print('Error: Target Object input is not set');
return;
}
// create new hit session
hitTestSession = createHitTestSession(script.filterEnabled);
transform = script.targetObject.getTransform();
// disable target object when surface is not detected
script.targetObject.enabled = false;
setObjectEnabled(script.indexToSpawn);
// Interaction Kit interactors are ready in OnStartEvent, not onAwake
script.createEvent('OnStartEvent').bind(function () {
// Set up trigger end callback
setupTriggerEndCallback();
// create update event
script.createEvent('UpdateEvent').bind(onUpdate);
});
// stop the session (and its depth computation) when the script is destroyed
script.createEvent('OnDestroyEvent').bind(function () {
hitTestSession.stop();
});
}
function setupTriggerEndCallback() {
// Get all interactors and set up trigger end callbacks
var allInteractors = IK.InteractionManager.getInteractorsByType(
InteractorInputType.All
);
for (var i = 0; i < allInteractors.length; i++) {
var interactor = allInteractors[i];
interactor.onTriggerEnd.add(
(function (currentInteractor) {
return function () {
// Only place object if we have a valid hit result and this is the primary interactor
if (lastHitResult && primaryInteractor === currentInteractor) {
placeObject();
}
};
})(interactor)
);
}
}
function placeObject() {
if (!lastHitResult) return;
// Copy the plane/axis object
var parent = script.objectsToSpawn[script.indexToSpawn].getParent();
var newObject = parent.copyWholeHierarchy(
script.objectsToSpawn[script.indexToSpawn]
);
newObject.setParentPreserveWorldTransform(null);
// Set position and rotation from last hit
var hitPosition = lastHitResult.position;
var hitNormal = lastHitResult.normal;
var lookDirection;
if (1 - Math.abs(hitNormal.normalize().dot(vec3.up())) < EPSILON) {
lookDirection = vec3.forward();
} else {
lookDirection = hitNormal.cross(vec3.up());
}
var toRotation = quat.lookAt(lookDirection, hitNormal);
newObject.getTransform().setWorldPosition(hitPosition);
newObject.getTransform().setWorldRotation(toRotation);
}
function createHitTestSession(filterEnabled) {
// create hit test session with options
var options = HitTestSessionOptions.create();
options.filter = filterEnabled;
var session = WorldQueryModule.createHitTestSessionWithOptions(options);
// depth computation starts once the session is started
session.start();
return session;
}
function onHitTestResult(results) {
if (results === null) {
script.targetObject.enabled = false;
lastHitResult = null;
} else {
script.targetObject.enabled = true;
// Store hit result for potential trigger end callback
lastHitResult = results;
// get hit information
var hitPosition = results.position;
var hitNormal = results.normal;
//identifying the direction the object should look at based on the normal of the hit location.
var lookDirection;
if (1 - Math.abs(hitNormal.normalize().dot(vec3.up())) < EPSILON) {
lookDirection = vec3.forward();
} else {
lookDirection = hitNormal.cross(vec3.up());
}
var toRotation = quat.lookAt(lookDirection, hitNormal);
//set position and rotation
script.targetObject.getTransform().setWorldPosition(hitPosition);
script.targetObject.getTransform().setWorldRotation(toRotation);
}
}
function onUpdate() {
primaryInteractor =
IK.InteractionManager.getTargetingInteractors().shift();
if (
primaryInteractor &&
primaryInteractor.isActive() &&
primaryInteractor.isTargeting()
) {
var rayStartOffset = new vec3(
primaryInteractor.startPoint.x,
primaryInteractor.startPoint.y,
primaryInteractor.startPoint.z + 30
);
var rayStart = rayStartOffset;
var rayEnd = primaryInteractor.endPoint;
hitTestSession.hitTest(rayStart, rayEnd, onHitTestResult);
} else {
script.targetObject.enabled = false;
}
}
function setObjectIndex(i) {
script.indexToSpawn = i;
}
function setObjectEnabled(indexToEnable) {
for (var i = 0; i < script.objectsToSpawn.length; i++) {
script.objectsToSpawn[i].enabled = i == indexToEnable;
}
}
// Initialize
onAwake();
// Expose public methods
script.setObjectIndex = setObjectIndex;
script.setObjectEnabled = setObjectEnabled;
}
// Register and initialize the script
script.WorldQueryHitExample = WorldQueryHitExample;
WorldQueryHitExample();
Add the surface placement script to your scene
- Save the script and add it to an object in your scene.
- Create a target object to place on surfaces.
- In the script component, set the Target Object input to that object.
- Test in Preview by moving the mouse and clicking, or send the Lens to SPECS (Send a Lens to SPECS).
The example is the simplest way to spawn objects on surfaces with hand gestures: point to move the target object, pinch to spawn a copy.
For ready-to-use implementations, import the World Query Module and Surface Placement assets from the Asset Library, or download them as unpacked packages (WorldQueryHit and SurfacePlacement) from the SPECS packages repository on GitHub.
Trade speed for accuracy, or reset a hit test session
Two more controls shape a World Query hit test session:
- Set
highAccuracyon the HitTestSessionOptions totruefor more accurate hit tests. This mode favors accuracy over response time and uses more power while the session is active, until you callstop()on the session (with several high-accuracy sessions active, stop all of them). WhenhighAccuracyis on,filteris ignored. - Call
reset()on a HitTestSession to reset it.
const options = HitTestSessionOptions.create();
options.highAccuracy = true;
const session = WorldQueryModule.createHitTestSessionWithOptions(options);
session.start();
Detect the type of surface a hit test lands on
Surface classes the World Query Module reports
With classification turned on, each World Query hit test result reports what kind of surface it hit, as a SurfaceClassification: Ground, Wall, Ceiling, Table, or None when the surface can't be identified. Use it to place objects automatically by surface type, such as only on the ground.
Semantic hit testing is an Experimental feature: enable Experimental APIs in your project settings to use it.
Turn on surface classification in a hit test session
Set classification to true on the HitTestSessionOptions, then read classification from each result:
- TypeScript
- JavaScript
import {
InteractorTriggerType,
InteractorInputType,
} from 'InteractionKit.lspkg/Core/Interactor/Interactor';
import { IK } from 'InteractionKit.lspkg/IK';
const WorldQueryModule =
require('LensStudio:WorldQueryModule') as WorldQueryModule;
@component
export class HitTestClassification extends BaseScriptComponent {
private hitTestSession: HitTestSession;
private primaryInteractor;
onAwake() {
this.hitTestSession = this.createHitTestSession();
this.createEvent('UpdateEvent').bind(this.onUpdate);
// stop the session (and its depth computation) when the script is destroyed
this.createEvent('OnDestroyEvent').bind(() => {
this.hitTestSession.stop();
});
}
createHitTestSession() {
const options = HitTestSessionOptions.create();
options.classification = true;
const session = WorldQueryModule.createHitTestSessionWithOptions(options);
// depth computation starts once the session is started
session.start();
return session;
}
onHitTestResult = (result: WorldQueryHitTestResult) => {
if (result === null) {
// Hit test failed
return;
}
const hitPosition = result.position;
const hitNormal = result.normal;
const hitClassification = result.classification;
switch (hitClassification) {
case SurfaceClassification.Ground:
print('Hit ground!');
break;
case SurfaceClassification.Wall:
print('Hit wall!');
break;
case SurfaceClassification.Ceiling:
print('Hit ceiling!');
break;
case SurfaceClassification.Table:
print('Hit table!');
break;
case SurfaceClassification.None:
print('Hit unknown surface!');
break;
}
};
onUpdate = () => {
this.primaryInteractor =
IK.InteractionManager.getTargetingInteractors().shift();
if (
this.primaryInteractor &&
this.primaryInteractor.isActive() &&
this.primaryInteractor.isTargeting()
) {
const rayStartOffset = new vec3(
this.primaryInteractor.startPoint.x,
this.primaryInteractor.startPoint.y,
this.primaryInteractor.startPoint.z + 30
);
const rayStart = rayStartOffset;
const rayEnd = this.primaryInteractor.endPoint;
this.hitTestSession.hitTest(rayStart, rayEnd, this.onHitTestResult);
}
};
}
// import required modules
const Interactor = require('InteractionKit.lspkg/Core/Interactor/Interactor');
const InteractorTriggerType = Interactor.InteractorTriggerType;
const InteractorInputType = Interactor.InteractorInputType;
const IK = require('InteractionKit.lspkg/IK').IK;
const WorldQueryModule = require('LensStudio:WorldQueryModule');
function HitTestClassification() {
// Private variables
var hitTestSession;
var primaryInteractor;
function onAwake() {
hitTestSession = createHitTestSession();
script.createEvent('UpdateEvent').bind(onUpdate);
// stop the session (and its depth computation) when the script is destroyed
script.createEvent('OnDestroyEvent').bind(function () {
hitTestSession.stop();
});
}
function createHitTestSession() {
var options = HitTestSessionOptions.create();
options.classification = true;
var session = WorldQueryModule.createHitTestSessionWithOptions(options);
// depth computation starts once the session is started
session.start();
return session;
}
function onHitTestResult(result) {
if (result === null) {
// Hit test failed
return;
}
var hitPosition = result.position;
var hitNormal = result.normal;
var hitClassification = result.classification;
switch (hitClassification) {
case SurfaceClassification.Ground:
print('Hit ground!');
break;
case SurfaceClassification.Wall:
print('Hit wall!');
break;
case SurfaceClassification.Ceiling:
print('Hit ceiling!');
break;
case SurfaceClassification.Table:
print('Hit table!');
break;
case SurfaceClassification.None:
print('Hit unknown surface!');
break;
}
}
function onUpdate() {
primaryInteractor =
IK.InteractionManager.getTargetingInteractors().shift();
if (
primaryInteractor &&
primaryInteractor.isActive() &&
primaryInteractor.isTargeting()
) {
var rayStartOffset = new vec3(
primaryInteractor.startPoint.x,
primaryInteractor.startPoint.y,
primaryInteractor.startPoint.z + 30
);
var rayStart = rayStartOffset;
var rayEnd = primaryInteractor.endPoint;
hitTestSession.hitTest(rayStart, rayEnd, onHitTestResult);
}
}
// Initialize
onAwake();
}
// Register and initialize the script
script.HitTestClassification = HitTestClassification;
HitTestClassification();
World Query Module API reference and related APIs
API Reference
- WorldQueryModule - Main module for world querying functionality
- HitTestSession - Session management for hit testing operations
- HitTestSessionOptions - Configuration options for hit test sessions
- WorldQueryHitTestResult - Results returned from hit test operations
Related APIs
hitTestWorldMesh- Alternative world mesh hit testingPhysics.Probe.raycast- Physics-based ray casting- DepthModule - Per-pixel depth frames, as described in Depth Module