ING Belgium boosts IT incident prediction accuracy by 21 points with synthetic data augmentation

Title

ING Belgium boosts IT incident prediction accuracy by 21 points with synthetic data augmentation

Content

ING Belgium's Site Reliability Engineering team boosted an IT-incident prediction model's F1 score from 76% to 97% — a 21 percentage-point jump — by augmenting imbalanced training data with synthetic replicas produced by DataCebo's Synthetic Data Vault (SDV) Enterprise, addressing a rare-event data shortage that had crippled earlier model attempts. The case study was co-authored by Jan Lennartz, Data Scientist in Site Reliability Engineering at ING Belgium, and Kalyan Veeramachaneni, Co-founder and CEO of DataCebo. Together they framed the goal: predicting whether a planned change would lead to a Major, P1, or P2 incident — enabling SRE to be proactive rather than reactive on hundreds of daily IT infrastructure changes logged across platforms such as ServiceNow and Jira Service Management. Context: ING faced severe class imbalance in its change-vs-incident dataset. The authors describe the team's central challenge: the number of changes linked to incidents was significantly less than what is optimal for finding patterns between the two, and a model trained on such imbalanced data tends to overfit to incident-related patterns, ultimately reducing overall accuracy. The 2024 CrowdStrike outage — which insurers estimated cost Fortune 500 companies around $5.4 billion in a single day — underscored why getting this prediction right matters. Evolution: ING first attempted classical ML training on real, featurized change records and plateaued at 76% F1. The team then explored generative modeling as a way to expand rare-class examples without exposing sensitive production data. SDV Enterprise emerged as the tool of choice for synthesizing realistic change-and-incident records that preserve statistical properties while stripping identifiers. Architecture: The seven-step pipeline ING followed — (1) train/test split, (2) train an SDV generative model on real data, (3) conditional sample synthetic data for Major/P1/P2 incidents, (4) combine real and synthetic into a balanced training set, (5) feature selection, (6) train a LightGBM prediction model, (7) test on held-out data — produced the headline 97% F1 score, a 21-point absolute improvement over baseline. Operational scale: Production deployment lives in ING Belgium's SRE organization, ingesting change and incident records from platforms like ServiceNow and Jira. DataCebo's SDV Enterprise runs the generative layer; LightGBM serves the prediction model. The pattern generalizes to other rare-event ML training scenarios, including fraud detection systems already cited in the article. Future direction: DataCebo has signaled a follow-up technical deep dive covering how much synthetic data to add, at what stage of model development to generate it, and how to validate robustness — pointing toward a roadmap of best-practice guidance for enterprises adopting synthetic-data augmentation for incident prediction and beyond.