Ada, a Toronto-based customer service automation company, migrated the seven Kubernetes clusters that run its AI platform, with up to 700 nodes each, from self-managed infrastructure to Amazon Elastic Kubernetes Service with near-zero downtime, according to an AWS case study published in 2024. Kubernetes upgrades that took two or three engineers up to five months now take five days, and deployment velocity rose 70% in the 18 months after the move.
"Using AWS Global Accelerator, we could dial traffic incrementally from 0 to 100 percent with very good granularity," said Kosta Djukic, principal engineer of cloud infrastructure at Ada.
Ada's product is a reasoning engine that combines machine learning models and large language model calls to understand customer inquiries and resolve them automatically; since 2016 it has handled more than 4 billion automated customer interactions. Ada commits to 99.9% availability, and version upgrades of its self-managed clusters had consumed up to 30% of its yearly error budget.
Ada completed the migration and several major Kubernetes version upgrades in four months in 2022, using blue/green deployments and AWS Global Accelerator to shift traffic gradually to the new clusters and roll back if needed. Most of its workloads, from general compute to ML inference, now run on EKS, and Argo CD application sets keep applications in sync across clusters. Ada deploys its ML models as containers and slices GPUs so that small models do not each occupy a full GPU, which it estimates raised the cost efficiency of GPU use by 20%. A scaling controller uses Amazon EC2 Spot Instances, and some infrastructure runs on AWS Graviton processors. Since the migration Ada's compute footprint has grown 30% while compute costs fell 15%.
The freed engineering time went into Ada's generative AI agent, which lifted benchmark automated resolution rates from 20-30% with its earlier declarative agents to up to 77%.
Details
- City
- Toronto
- Organization
- Ada Support
- Continent
- North America
- Country