OmniEmbodied Documentation

OmniEmbodied is a comprehensive platform for embodied AI research, consisting of two main components:

OmniSimulator: A powerful simulation engine for embodied AI agents
OmniEmbodied Framework: An evaluation and training framework built on top of the simulator

Note

This documentation covers both OmniSimulator and the OmniEmbodied Framework. For the latest version, see our GitHub repository.

Quick Links

Installation: Installation
Quick Start: Quick Start
OmniSimulator API: OmniSimulator
Framework Guide: OmniEmbodied Framework

Platform Overview

🎮 OmniSimulator: Core simulation engine providing realistic environments, action systems, and agent interfaces for embodied AI research.
📊 OmniEmbodied Framework: Evaluation and training framework with LLM integration, multi-agent coordination, and comprehensive benchmarking tools.

Key Features

OmniSimulator Engine:

🌍 Rich Environments: Room-based worlds with realistic object interactions and spatial relationships.
🎯 Flexible Action System: Movement, manipulation, observation, and communication actions with validation.
🤖 Agent Interfaces: Clean APIs for integrating various AI architectures and decision-making systems.

OmniEmbodied Framework:

📈 Comprehensive Evaluation: Built-in benchmarks with detailed analytics and performance metrics.
🔧 LLM Integration: Seamless integration with OpenAI, Anthropic, and local language models.
⚖️ Multi-Agent Support: Single-agent and multi-agent scenarios with collaboration patterns.

Getting Started

If you’re new to OmniEmbodied, start with our Installation guide, then follow the Quick Start tutorial to run your first evaluation.

Architecture Overview

The platform has a layered architecture:

┌─────────────────────────────────────────────┐
│           OmniEmbodied Framework            │
│  ┌─────────────────┐ ┌─────────────────────┐ │
│  │   Evaluation    │ │   Agent Modes       │ │
│  │   Framework     │ │   - Single Agent    │ │
│  │                 │ │   - Multi-Agent     │ │
│  └─────────────────┘ └─────────────────────┘ │
│  ┌─────────────────────────────────────────┐ │
│  │         LLM Integration Layer           │ │
│  └─────────────────────────────────────────┘ │
└─────────────────────────────────────────────┘
┌─────────────────────────────────────────────┐
│              OmniSimulator                  │
│  ┌─────────────┐ ┌─────────────┐ ┌────────┐ │
│  │ Environment │ │   Actions   │ │ Agents │ │
│  │  Management │ │   System    │ │   API  │ │
│  └─────────────┘ └─────────────┘ └────────┘ │
│  ┌─────────────────────────────────────────┐ │
│  │           Simulation Core               │ │
│  └─────────────────────────────────────────┘ │
└─────────────────────────────────────────────┘

Use Cases

For Researchers:

Embodied AI Research: Study agent perception, reasoning, and action in realistic environments
Multi-Agent Systems: Investigate collaboration, communication, and coordination strategies
Benchmarking: Evaluate and compare different AI architectures on standardized tasks

For Developers:

Agent Development: Build and test intelligent agents with comprehensive simulation
Algorithm Testing: Validate planning, learning, and decision-making algorithms
Integration: Incorporate embodied AI capabilities into larger systems

For Educators:

Teaching Tool: Demonstrate AI concepts with interactive simulations
Research Platform: Support student research projects with ready-to-use infrastructure
Curriculum Development: Create coursework around embodied AI and multi-agent systems

Community and Support

Issues: Report bugs and request features on our GitHub Issues
Community: Join discussions and get help from the community
Contributing: See our Contributing to OmniEmbodied guide