Ant Group Deploys Ray for Online Resource Allocation at 6,000+ CPU Cores

Title

Ant Group Deploys Ray for Online Resource Allocation at 6,000+ CPU Cores

Content

Ant Group implemented a flexible, high-performance, stable, and scalable online resource allocation system based on Ray, the open-source distributed computing framework, to support massive scale operations during events like Double 11, the largest online shopping event in the world. The system deployment scale reached more than 6000 CPU cores and is used across a wide range of application scenarios including marketing, search, recommendation, and advertising. The technical challenges addressed included reducing development and maintenance costs across complex engineering links, achieving high-performance solutions for large-scale linear programming problems with up to 100 million variables, and ensuring 99.99% SLA availability with service downtime not exceeding one hour per year. The online resource allocation system uses linear programming duality to make fast online serving decisions, with real-time algorithms for model calibration, traffic prediction, constraint correction, and large-scale LP optimization. The system is built on Ray actors including Master Actors, Router Actors, and Worker Actors, with fault-tolerant recovery ability that senses abnormal actors within one second and forwards requests to alive actors. Ant Group uses Ray for distributed computing across offline, nearline, and online links, enabling LP optimization problems with millions of variables and hundreds of constraints to be solved in seconds using a single GPU card, with ongoing exploration of multi-GPU solutions.

URL