INTRODUCING

Project Aria Pilot Dataset

The Aria Pilot Dataset is a collection of 159 sequences captured using Project Aria, to accelerate the state of machine perception and AI.

WHAT IS IT?

A multi-purpose egocentric dataset created using Project Aria

The Aria Pilot Dataset and accompanying tools provides researchers in computer vision access to anonymized Aria sequences, captured in a variety of scenarios, such as cooking, playing games, or exercising. In addition to ‘Everyday Activities’, the dataset also includes ‘Desktop Activities’ captured with a multi-view motion capture system, helping to accelerate research into human-object interactions.

We believe this dataset will provide a baseline for external researchers to build and foster reproducible research on egocentric computer vision and AI/ML algorithms for scene perception, reconstruction and understanding.

LEARN MORE ABOUT PROJECT ARIA

Sensor Data

1 x 110 degree FOV Rolling Shutter RGB camera
2 x 150 degree FOV Global Shutter mono cameras for SLAM and hand tracking
2 x 80 degree FOV Global Shutter mono cameras for eye-tracking with IR illumination
2 x 1KHz IMU + barometer & magnetometer environmental sensors
7 x 48 KHz spatial microphones
GPS 1 Hz

HOW IS IT ANNOTATED?

Automatic and manual annotations

In addition to providing sensor data from Project Aria, the Pilot Dataset also contains derived results from machine perception services which provide additional context to the spatial-temporal reference frames.

Multiple point clouds being aligned to the same frame of reference.

Multi-user poses in shared reference frame

In addition to providing per-frame trajectory for every recording, sequences captured within the same environment have been aligned to the same reference-frame, allowing those sequences to be understood within the same context.

A visualization of the sensor calibration from Project Aria Glasses.

Camera calibration

For a high-quality egocentric dataset, it is essential to understand how cameras perceive the world. The Project Aria Pilot Dataset provides full camera calibration parameters, including both intrinsics and extrinsics of every sensor.

A visualization of a multi-camera capture rig, used within the Project Aria Pilot Dataset.

Multi-view motion capture

To facilitate research into human-object interactions, the Aria Pilot Dataset includes a subset of “Desktop Activities” captured using a multi-view motion capture system.

Multiple camera trajectories, being visualized within the same frame of reference.

Multi-device time sync

In addition to aligning the trajectories of sequences captured within the same environment, the Project Aria Pilot Dataset also provides precise time-alignment between sequences captured simultaneously.

An image from Project Aria glasses, showing a woman in a house. The woman's face has been blurred to preserve privacy.

Speech-to-text

For sequences where actors speak, we provide speech-to-text annotation. This supports egocentric communications research, such as predicting turn-taking in conversations and multi-speaker transcription.

A visualization of the eye gaze annotation, contained within the Project Aria Pilot Dataset.

Calibrated eye-gaze

Using data from Project Aria’s eye-tracking cameras, the Pilot dataset includes an estimate of the wearer’s eye-gaze. This can be used to accelerate research into user-object interactions.

HOW WILL THE DATASET BE USED?

Accelerating the state of machine perception and artificial intelligence

The Project Aria Pilot dataset consists of 159 sequences, which can be used to unlock several areas of research for progressing the state of machine perception and AI, including camera relocalization, and scene reconstruction.

Studying these research areas is crucial for researchers to engage with the challenges associated with AR devices.

A visualization of a point cloud and trajectory from two Project Aria devices.

PROJECT ARIA TOOLS

Powerful tools to harness Aria datasets

Tools for working with Project Aria Pilot Dataset allow researchers to access, interact with, and visualize all raw data and annotations available in the dataset.

Both C++ and python interfaces are provided to load data, so that researchers can access data in the way best suited to their needs.

VIEW ARIA DATA TOOLS ON GITHUB

A screenshot of the visualization tools, provided with the Project Aria Pilot Dataset.

Locations

To demonstrate a few representative scenarios in all-day-long activities with always-on sensing, multiple sequences were recorded using actors in five locations across USA.

RGB footage from Project Aria, showing a North American home.

Enabling innovation, responsibly

All sequences within the Aria Pilot Dataset have been captured using fully consented actors in controlled environments.

Additionally, faces and license plates have been manually blurred using human annotation prior to public release.

RESPONSIBLE INNOVATION PRINCIPLES

Sensor data from the Project Aria Pilot Dataset. Faces have been blurred to preserve privacy.

Access Project Aria Pilot Dataset and accompanying Tools

If you are a researcher in AI or ML research, access the Project Aria Pilot Dataset and accompanying tools from Meta.com.

By submitting your email and accessing the Aria Pilot Dataset, you agree to abide by the dataset license agreement and to receive emails in relation to the dataset.

Subscribe to Project Aria Updates

Stay in the loop with the latest news from Project Aria.

By providing your email, you agree to receive marketing related electronic communications from Meta, including news, events, updates, and promotional emails related to Project Aria. You may withdraw your consent and unsubscribe from these at any time, for example, by clicking the unsubscribe link included on our emails. For more information about how Meta handles your data please read our Data Policy.