Robotics training data glossary

Last updated: 2026-11-09. These are the terms that come up when you compare robotics data companies.

Collection types

Teleoperation (teleop) data

Demonstrations recorded while a person remotely drives a real robot. They include video plus the robot's joint states and actions, which makes them direct material for imitation learning.

Egocentric video

First-person video from a head-mounted camera on a person doing a task. It is used for pre-training and world models. On its own it does not contain robot actions; hand pose, depth and camera trajectory supply much of the action signal.

Ego-exo / multi-view video

Egocentric video recorded alongside synchronized fixed external cameras that show the same activity.

UMI (Universal Manipulation Interface)

Demonstrations collected with handheld, instrumented grippers. The data can transfer to robots without a robot on site during collection.

Motion capture

High-fidelity recording of human body movement, used in humanoid control and robot learning.

Signals and labels

Sensor fusion

Combining and labeling camera, LiDAR, radar and other streams together so that labels stay consistent across sensors.

LiDAR point cloud

A 3D set of points measured by laser ranging. It is labeled with cuboids, segmentation or tracks.

6-DoF pose

An object's or camera's position and orientation in 3D: three translations and three rotations.

IMU

Inertial measurement unit: accelerometer and gyroscope data, often used to check camera motion.

Action segmentation

Splitting an episode into time-stamped steps, often with a natural-language description of each.

Success / failure / recovery labels

Labels for whether a demonstration achieved the task, and how the operator recovered from mistakes.

Models

VLA (vision-language-action) model

A robot model that takes images and language instructions as input and outputs actions.

World model

A model that predicts how an environment will change, trained largely on continuous video and action data.

Delivery formats

LeRobot

Hugging Face's robotics dataset format, built on Parquet files and video.

RLDS

Reinforcement Learning Datasets, an episodic format built on TensorFlow Datasets. It is used by RT-X-style pipelines.

HDF5

A general-purpose hierarchical data file format. robomimic datasets use it.

MCAP

A log container format used across the ROS ecosystem for multi-sensor recordings.

WebDataset

A format that shards data into tar files for streaming during large-scale training.

Commercial terms

Validated / finished hour

A billing unit used by some collection vendors: footage that passed sync and QA checks and was delivered. Rejected hours are excluded.

Provenance

A record tracing each clip to its capture session, device and consent record.

Related guides: Best AI training data