Hands Free, AIs Forward: NVIDIA XR AI Brings Agents to AR Glasses

NVIDIA XR AI is now available in public beta, giving developers a framework for building multimodal AI agents for AR glasses and XR devices.  

However, building agentic systems that combine models, skills, harnesses, tools and an agentic runtime to help people perform hands-on work is challenging. To operate effectively in dynamic, real-world environments, these agents must do more than generate responses.  Like human workers, they need knowledge, tools and specialized skills to perceive and understand the world through video, audio and sensor data, interpret fast-changing conditions and spatial context, retrieve information from enterprise systems, reason about the next best action and use software tools to complete tasks. All of this must happen with low latency and in a way that supports the user without creating distraction. NVIDIA XR AI is a developer library that helps developers build these agentic applications. By connecting inputs from AR glasses and XR devices with AI models, enterprise data, tools and accelerated computing, NVIDIA XR AI enables agents that can perceive, reason and act in the flow of work.  It provides a foundation for developers to build or connect skills and tools for enterprise XR applications, and simplifies the integration of multimodal perception, enterprise retrieval, reasoning models and agent orchestration. Together, these capabilities make it easier to build spatially aware, multimodal AI agents that deliver low-latency, context-aware assistance in AR and XR experiences.  NVIDIA NeMo Agent Toolkit enables tool use, reasoning workflows and multi-agent coordination, while NVIDIA accelerated computing platforms — including NVIDIA DGX Spark, NVIDIA DGX Station and NVIDIA RTX PRO systems — provide the infrastructure to run inference across cloud, data center and edge environments. Together, these capabilities enable AI agents that can understand their surroundings, access enterprise knowledge, reason about complex tasks and deliver contextual assistance in real time.  Across manufacturing, science, healthcare, design and immersive learning, developers and enterprises are already tapping NVIDIA XR AI — embedding AI agents where the work happens.