China Merchants Bank Wins CNCF Contest for Kubernetes AI Platform

Cloud Native Computing Foundation announced in a press release that China Merchants Bank won its End User Case Study Contest for a Kubernetes platform that combines AI training, fine tuning and online inference. The platform manages nearly 10,000 heterogeneous accelerator cards and covers 99% of the bank's accelerator computing resources.

The bank increased average accelerator utilisation from 35% to more than 60%. It also cut the cost of processing 1 million combined input and output tokens by more than 60% under comparable model and service conditions.

The platform uses Kueue for training admission, queues and quotas, KEDA and Prometheus for demand based inference scaling, HAMi for fine grained shared accelerator allocation, and Fluid for faster access to datasets, model weights and checkpoints. The bank's Twinkle training framework lets five LoRA tenants share one base model instance by default, cutting accelerator use for that configuration by 80% and increasing training density fivefold.

The bank plans to adjust multi tenant training concurrency dynamically, build unit cost based capacity management from utilisation, queue and latency signals, extend KEDA toward scale to zero serverless inference, and broaden HAMi support for more accelerators and additional training and inference backends.

The award was announced during a keynote at the KubeCon, CloudNativeCon, OpenInfra Summit and PyTorch Conference China event in Shanghai.