Public Spatial AI Benchmarks
Major public benchmarks and datasets for evaluating 3D vision, spatial understanding, and physical-world AI. These resources have driven advances in scene reconstruction, object detection, and robotics. Sky3 will provide a community-governed benchmark focused on real-world edge cases contributed and verified by independent participants.
Indoor Scene Understanding
- ScanNet
- RGB-D video dataset of indoor scenes with 3D camera poses, surface reconstructions, and semantic segmentations.
- ARKitScenes
- High-resolution depth scans of indoor environments captured with LiDAR on mobile devices.
- Matterport3D
- Large-scale RGB-D dataset of building-scale indoor spaces with semantic annotations.
- Replica
- High-quality 3D reconstructions of indoor spaces designed for evaluating visual SLAM and embodied AI.
- SUN RGB-D
- Large-scale RGB-D scene understanding benchmark with 10,335 images and dense annotations.
Autonomous Driving and Outdoor Scenes
- KITTI Vision Benchmark Suite
- Autonomous driving dataset with stereo video, LiDAR point clouds, GPS/IMU data, and 3D object annotations.
- nuScenes
- Large-scale autonomous driving dataset with full sensor suite (cameras, LiDAR, radar) and 3D bounding boxes.
- Waymo Open Dataset
- High-quality sensor data from autonomous vehicles, including LiDAR, cameras, and 3D labels for perception tasks.
3D Objects and Shapes
Robotics and Embodied AI
- Open X-Embodiment
- Large-scale dataset of robotic manipulation demonstrations across multiple robot platforms and tasks.
Join the data-ingestion beta
Help build the benchmark for spatial AI. Contribute data, review cases, or test your models on verified real-world edge cases.
Join the waitlist →