Inflection AI Trains the Models Behind Its Pi Chatbot on Azure A100 GPU Clusters with InfiniBand Networking

Inflection AI built the infrastructure behind Pi, its conversational personal-intelligence chatbot, on Microsoft Azure, according to a Microsoft customer story from August 2023. Pi is designed as a friendly, patient companion that can help people organise their thoughts, research topics and prioritise their day, and people can use it through a website, mobile app, text message, WhatsApp, Instagram direct messages, Facebook Messenger or by voice. Training Pi's language models requires very large amounts of compute. Inflection used GPU-accelerated Azure NC A100 v4 and ND A100 v4 virtual machines powered by NVIDIA A100 Tensor Core GPUs, together with Azure Machine Learning, and cites Microsoft's builds of NVIDIA Quantum-2 InfiniBand networking for fast connections between GPUs. Its largest language models were trained continuously for several months on thousands of AI-optimised virtual machines. "Assembling those into a large-scale connected cluster is extremely difficult and time consuming, and it requires very careful attention to details, which is one of the reasons we use Azure," said Mustafa Suleyman, then co-founder and CEO. Inflection relied on Microsoft's expertise in debugging networking issues on large clusters. "Microsoft tested the GPU cluster for two weeks before handing it over, and as a result, we had very little downtime," Suleyman said, adding that Microsoft "was one of the first to scale NVIDIA Tensor Core A100 GPUs and daisy chain them together." Built-in security, compliance and business continuity also influenced the choice, because the models are expensive to train and their intellectual property is valuable. Inflection worked with a dedicated Microsoft account team, and its own team included people who had worked on GPT-2, GPT-3, LaMDA and PaLM.