Artificial Intelligence1 min read

Build Training-Ready Physical AI Datasets with Two Prompts in NVIDIA Cosmos

By · Published by Everything Blog

In short

The workflow relies on the NVIDIA Cosmos Data Factory, which integrates five cloud-native blueprints and six composable skills orchestrated by frontier AI agents without human steering. Developers initiate the pipeline by entering an initial setup prompt into a coding agent to handle repository cloning and preflight checks, followed by a second prompt executing a specific blueprint. Integrated tools like NVIDIA Cosmos Curator, Cosmos Dataset Search, and Cosmos Evaluator verify that the generated data maintains physical plausibility and attribute accuracy. This synthetic framework allows teams to safely simulate critical, hard-to-capture edge cases like product leaks and visual defects.

Key points

  • Automated orchestration removes the need for expensive and hazardous real-world staging o…: Automated orchestration removes the need for expensive and hazardous real-world staging of rare physical events.
  • Specialized blueprints target distinct industrial applications ranging from autonomous dr…: Specialized blueprints target distinct industrial applications ranging from autonomous driving to manufacturing defect inspection.
  • Cloud-native workflows resolve critical training dataset gaps by synthesizing physically…: Cloud-native workflows resolve critical training dataset gaps by synthesizing physically plausible edge cases at scale.

Sources checked by our analysis pipeline: 1

World model accuracy starts with high-quality, diverse training datasets. Yet capturing rare defects, nighttime incidents, and other challenging scenarios is costly and time-consuming. Turning raw data into training-ready datasets adds complexity, particularly when using disconnected tools and manual processes for curation, labeling, generation, and evaluation.

NVIDIA technology is helping developers more easily build high-quality, diverse training datasets at scale for physical AI. NVIDIA Cosmos is an open platform of frontier models, datasets, frameworks, and skills for world model development. As part of this platform, NVIDIA Cosmos Data Factory (CDF) is a collection of five cloud-native blueprints and six composable skills to accelerate data curation, processing, and synthetic data generation (SDG). Frontier AI agents can be used to orchestrate these blueprints, enabling prompt-to-data without human steering.

This post walks you through how to curate and augment datasets or create anomalous events for physical AI model training with just two prompts in Cosmos. It features five different use cases.

Set up and configure Cosmos blueprints and skills: Prompt 1

Cosmos Data Factory contains five reference blueprints for common tasks. All five blueprints…

Artificial Intelligence