PHYSICAL WORLD
Multimodal understanding
One event. More than one signal.
Words, sound and visible action may describe different parts of the same event.
The human contribution
Synchronized speech, video and task demonstrations, captured with an explicit recording protocol.
The structured dataset
Aligned streams, event boundaries, cross-modal labels and synchronization metadata.
Where it helps
Evaluate systems that combine spoken instructions with visual context and operator interfaces.
Collection scope, availability, participant permissions and permitted use are defined for each engagement.
Discuss a data program ↗