Inworld and Qwerky Deploy Mojo for Custom GPU Kernel Development

Title

Inworld and Qwerky Deploy Mojo for Custom GPU Kernel Development

Content

Modular AI partnered with Inworld and Qwerky AI to deploy custom Mojo GPU kernels for AI inference optimization. For Inworld, a company building AI products for consumer applications including realistic voice AI, Modular deployed its MAX Framework and Mojo language to create a state-of-the-art speech synthesis pipeline on NVIDIA Blackwell GPU Architecture. The collaboration achieved 70% faster time to first audio and 60% lower cost compared to vanilla vLLM-based implementations, returning the first 2 seconds of synthesized audio in just 200ms. The MAX streaming-aware scheduler minimized time-to-first-token for the Speech-Language Model component while custom Mojo kernels, including a tailored silence-detection kernel running directly on the GPU, enabled highly efficient inference. For Qwerky AI, Modular enabled the deployment of custom Mamba-based models across NVIDIA and AMD GPUs without rewriting native code for each platform. Qwerky engineers wrote a custom selective scan operation in Mojo in just 20-30 lines of readable code that automatically optimizes for both NVIDIA tensor cores and AMD matrix engines, achieving 50% faster GPU kernels and 3x research velocity. The unified MAX compilation pipeline allows Qwerky to iterate on model improvements in hours instead of weeks.

URL