Frequently Asked Questions

What is Sky3?

Sky3 is an open, community-governed network for testing spatial AI and physical world models against real-world data. Contributors will submit difficult real-world spatial cases—LiDAR scans, CAD files, robot trajectories and geometry imagery. Independent reviewers will verify those submissions. Models will then be scored on geometric accuracy, structural understanding and physical reasoning, and the results will be published on a public leaderboard.

The goal is to provide neutral, reproducible benchmarks that reflect how models perform on real-world edge cases, not just on data their developers selected.

Is the leaderboard on the homepage real?

No. The leaderboard currently displays made-up models and made-up numbers to demonstrate the format. All model names are clearly marked as samples, and a warning label states "SAMPLE DATA · FICTIONAL MODELS" at the top of the table.

Once the beta is complete and verified test cases are available, real models will be evaluated and the leaderboard will show actual results. Until then, the sample leaderboard shows what the public scorecard will look like.

How does the beta work?

The data-ingestion beta is the first phase of Sky3. A small group of contributors, AI developers and researchers will be invited to:

• Contributors: submit real-world spatial data that models find challenging.
• Reviewers: verify submitted cases for quality, correctness and labeling accuracy.
• AI developers: provide feedback on metrics, evaluation methods and the sandbox API (planned).

Beta participants earn points that record their contribution to the archive. The beta helps refine the protocol, metrics and verification process before opening to a wider community.

How is data privacy and sanitization handled?

The protocol is designed with a sanitization pipeline for contributed data:

• Metadata stripping: GPS coordinates, timestamps, device identifiers and file metadata will be removed.
• Face and personal information removal: images and scans will be processed to detect and blur faces, license plates and other personally identifiable information.

The goal is to enable spatial edge cases to be shared for benchmarking without exposing private or proprietary information. Details of the sanitization process will be published as part of the protocol documentation.

How is Sky3 governed?

Sky3 is being built as a community-governed protocol, with plans for a non-profit foundation (currently in formation). Governance decisions—such as which metrics to use, how verification works, and what changes to make to the protocol—will be made through an open process with input from contributors, reviewers and model developers.

The planned foundation's role will be to maintain the archive, run the infrastructure, publish results and facilitate the governance process. It will not control which models are tested or which cases are accepted; those decisions will follow the community-defined protocol.

Points earned during the beta are a record of contribution. They have no monetary value, and any future use of points (for example, in governance voting) will be decided through the governance process itself.

Who can join the beta?

The beta is open to:

• Contributors: individuals or organisations with access to real-world spatial data—factory scans, construction site LiDAR, CAD assemblies, robotics lab recordings—who want to help build a better benchmark.
• Reviewers: people with expertise in 3D geometry, CAD, structural analysis or robotics who can verify that submitted cases are correctly labeled and meet quality standards.
• AI developers and researchers: teams building spatial AI or world models who want to test their systems on independent real-world data and provide feedback on the protocol.

The beta is invite-only initially to keep the group small enough to iterate quickly. Once the protocol is stable, the network will open more widely.

What data formats are accepted?

The beta is planned to accept:

• Point clouds: .las, .laz, .ply, .xyz
• CAD files: .step, .stp, .stl, .obj, .iges
• Trajectories: .csv, .json (with position, orientation and timestamp fields)
• Images: .jpg, .png (especially depth images, multi-view geometry sets or images with known camera poses)

Submissions must include basic metadata (what the scene or object is, any known labels or measurements) and consent for use in benchmarking. Detailed submission guidelines will be provided to beta participants.

How do I submit a model for evaluation?

Model evaluation is not yet open. The beta focuses on building the verified test set first. Once enough verified cases are available, a sandbox API will be launched where model developers can submit inference results for scoring.

The API will accept model outputs (predicted meshes, point clouds, part labels, assembly trees) and return scores on the metrics defined in the metrics page. Evaluation will be run on held-out test splits so models cannot overfit to the benchmark.

Does it cost anything to participate?

No. Contribution, review and evaluation during the beta will be free. There will be no fee to submit data, verify cases or test models on the benchmark.

Long-term sustainability may come from optional enterprise services (such as private evaluation, custom benchmarks or dataset licensing to companies) while keeping the core protocol and public leaderboard free and open.

Is Sky3 open source?

The protocol, metrics and evaluation code will be published under open licenses once they are stable. The goal is full transparency: anyone should be able to inspect how scores are calculated, reproduce results and propose improvements.

Contributed data remains the property of contributors (subject to the terms they agree to at submission). The archive is structured so that verified cases can be used for benchmarking without requiring contributors to release raw data publicly.