Meta Reality Labs Deploys ExecuTorch for On-Device AI on Quest Headsets and Ray-Ban Smart Glasses

Title

Meta Reality Labs Deploys ExecuTorch for On-Device AI on Quest Headsets and Ray-Ban Smart Glasses

Content

Meta Reality Labs goal is to build the next computing platform. The teams across Reality Labs develop next-generation hardware including Ray-Ban Meta and Oakley Meta glasses, Meta Ray-Ban Display glasses and the Meta Neural Band, and Meta Quest headsets. These products require AI models to run efficiently on a wide range of hardware from high-performance systems-on-a-chip (SoCs) to ultra-low-power microcontrollers. ExecuTorch is an open source, lightweight, and efficient inference engine that enables cutting-edge machine learning experiences across Reality Labs portfolio. On-device AI deployment presents a fundamental challenge that requires balancing supporting researchers and engineers with quick experimentation with optimization flows across diverse hardware targets, while not sacrificing performance and productivity. Traditional approaches require converting PyTorch models to other formats, which introduces numerical mismatches and costly debug cycles. ExecuTorch addresses this challenge by eliminating the conversion step and providing an entirely PyTorch-native flow. ExecuTorch enables Meta Quest 3 and Quest 3S to run advanced AI workloads such as depth estimation and scene understanding directly on the device. This local processing ensures fast, reliable performance, allowing features like Passthrough to seamlessly blend the physical world with virtual content. Real-time AI models enabled by ExecuTorch also drive important features like hand tracking and controller tracking. Another standout feature enabled by ExecuTorch is persistent room memory, where Quest 3 and 3S can remember up to 15 different rooms with their unique layout and boundaries. On Ray-Ban Meta glasses, ExecuTorch enables complex models to run directly to deliver new features like live translation and visual captions shown in real time on the glasses display. Another breakthrough is the text-in-the-wild capability enabled by on-device AI inference.