A real-time warehouse digital twin built on NVIDIA Omniverse. Users can manage inventory (restock / remove cardboard boxes) directly in a 3D USD scene via a web-based dashboard, and query warehouse state using natural language powered by OpenAI.
- Real-time 3D visualization — warehouse scene streamed from Omniverse Kit to a web browser via WebRTC
- Inventory control — restock or remove cardboard boxes on specific rows and shelves using a structured dashboard
- USD scene manipulation — boxes are spawned as USD references and physically placed on shelf surfaces
- AI natural language queries — ask questions about warehouse state in any language (e.g. "Which shelf is almost empty?") powered by OpenAI GPT-4o-mini
- Shelf status variants — shelves automatically update visual state (empty / warning / normal / full) based on inventory level
| Category | Technology |
|---|---|
| 3D Engine | NVIDIA Omniverse Kit |
| Scene Format | OpenUSD (Universal Scene Description) |
| Streaming | Omniverse WebRTC (omni.kit.livestream.webrtc) |
| Kit Events | carb.eventdispatcher, omni.kit.app |
| AI | OpenAI API (gpt-4o-mini) |
| Frontend Framework | React 18 + TypeScript |
| Styling | Tailwind CSS v4 |
| Build Tool | Vite |
| Kit UI | omni.ui |
warehouse-digital-twin/
├── frontend/
│ └── src/
│ ├── App.tsx # Main app, stream state management
│ ├── WarehouseDashboard.tsx # Inventory control + AI query dashboard
│ ├── StreamOnlyWindow.tsx # WebRTC stream + custom event handler
│ └── AppStream.tsx # AppStreamer wrapper
└── source/extensions/
└── demo.semantic_layout_generator/
├── config/extension.toml # Extension config & dependencies
└── demo/semantic_layout_generator/
└── extension.py # Kit extension: event handling, USD scene, OpenAI
🚧 Project Status: Work in Progress (WIP)