Revolut Trains PRAGMA Transaction Foundation Model on NVIDIA H100 GPUs, Achieving 3–5x Faster Development and 2.3x Better Credit Risk Detection

Title

Revolut Trains PRAGMA Transaction Foundation Model on NVIDIA H100 GPUs, Achieving 3–5x Faster Development and 2.3x Better Credit Risk Detection

Content

Revolut, the London-founded global financial super-app serving more than 70 million customers in 40 markets, trained PRAGMA — a transformer-based transaction foundation model family ranging from 10 million to one billion parameters — on NVIDIA H100 GPUs via Nebius AI Cloud. The deployment delivered 3–5x faster model development cycles, 2–5x higher training throughput, 2.3x better credit default detection, 65% more fraud cases caught, and 41% more relevant product recommendations against prior production baselines. The team's architectural philosophy is to learn rich behavioral representations directly from raw event streams and reuse those embeddings across fraud, credit, engagement, product, and recommendations. PRAGMA is a family of encoder-style models that interpret each customer's financial journey as a temporal signal similar to how language models interpret text sequences, with three specialized encoders for user attributes, individual events, and long-term history combining into a unified behavioral representation. Before PRAGMA, Revolut followed the industry-standard pattern of dedicated machine learning pipelines per task, each with hand-crafted features and bespoke ETL. Launching a new use case or entering a new market required months of feature engineering and validation. The fragmentation also meant fraud and credit systems could be working from different representations of the same customer history, making risk-and-growth optimization difficult. PRAGMA was pre-trained on approximately 26 million user records across 111 countries, spanning roughly 24–40 billion events and 207 billion tokens over around 28 months of history. PRAGMA-S (the 10-million-parameter variant) converged in about two days on 16 H100 GPUs, while larger variants took roughly two weeks on 16–32 GPUs. NVIDIA Llama-Nemotron-Embed-1B-v2 is used to embed unstructured text fields like merchant descriptions, delivering a measured 16.1% improvement in credit risk prediction performance. The architecture uses structured tokenization that indexes categorical fields, quantizes continuous values like transaction amounts, and decomposes timestamps into interpretable temporal components. Inter-event timing is modeled with a smooth logarithmic transformation to capture both short-term behavior and long-horizon life events in a single temporal framework. To preserve numerical fidelity and sequence structure, Revolut built shard-based dynamic batching with fixed GPU memory token budgets and used variable-length attention kernels to reduce padding. On Revolut's internal benchmarks, PRAGMA delivered a 64.7% improvement in fraud recall and a 16.7% lift in fraud precision over the prior production model. Sub-second latency is achievable on PRAGMA-S for real-time fraud screening at the point of transaction, while larger variants handle accuracy-sensitive workloads. Teams can either freeze the model and train lightweight linear heads on top of embeddings for rapid experiments, or apply LoRA fine-tuning updating only 2%–4% of parameters. Revolut plans to expand PRAGMA with more multimodal inputs, enable continual pre-training so the model can learn from new events more continuously, and broaden downstream use cases to lifetime value prediction, churn forecasting, and anomaly detection. On the infrastructure side, the team is exploring AutoML integration, embedding versioning, and a full inference pipeline; on the app side, a model-driven interface that adapts in real time to each user's behavior.