The new GPU pooling system underscores Beijing’s drive for AI self-reliance – and signals how national clouds are rewriting the rules of global compute efficiency.
Alibaba Cloud, the digital technology and intelligence backbone of Alibaba Group, has announced a breakthrough in GPU resource optimization with the introduction of its new system, Aegaeon, which reduces GPU requirements by up to 82 percent through an advanced pooling architecture.
The company detailed its innovation in a peer-reviewed paper presented at the 2025 ACM Symposium on Operating Systems Principles (SOSP) in Seoul. The paper, jointly authored by Alibaba Group and the School of Computer Science at Peking University, describes how Aegaeon enables more efficient GPU utilization for sporadic and unpredictable inference workloads that typically rely on dedicated GPU instances.
Traditional multi-model serving approaches use GPU pooling and serverless computing to improve efficiency, but these methods are generally limited to two or three models per GPU. Aegaeon, however, introduces a token-level auto-scaling mechanism, enabling up to seven models per GPU.
According to the paper, Aegaeon performs model auto-scaling “at the token granularity,” allowing it to dynamically schedule model requests and make scaling decisions per token. This fine-grained method “preemptively scales down active models and scales up pending models for newly arrived requests in an SLO-aware manner,” effectively overcoming head-of-line blocking and achieving highly efficient GPU pooling.
Deployed in beta within Alibaba Cloud’s model marketplace, Aegaeon currently serves dozens of models and has demonstrated a reduction in GPU usage from 1,192 to 213 GPUs – an 82 percent decrease. The paper notes that in typical model studios, over 90 percent of models are infrequently invoked, resulting in significant GPU underutilization that Aegaeon effectively addresses.
Testing was conducted over several months using a two-node cluster with 16 Nvidia H800 80GB GPUs, 2TB of DDR5 memory, and 192 Intel Xeon Platinum 8469C CPUs. The system achieved performance improvements between 1.5x and 9x, even on models with up to 72 billion parameters.
Although the development has not sparked the same level of industry disruption as DeepSeek’s earlier claims about low-cost AI training, the Aegaeon system reinforces Alibaba Cloud’s leadership in AI infrastructure innovation, placing it among the few global providers openly sharing deep technical advances in GPU optimization. Earlier this year, Alibaba introduced Qwen3, a family of large language models that the company says can match – and in some cases outperform – the best offerings from Google and OpenAI.
Together, Qwen3 and Aegaeon highlight Alibaba’s two-pronged strategy: advancing the intelligence layer through model innovation while strengthening the infrastructure layer that powers AI at scale. In an era defined by GPU scarcity and escalating compute costs, Alibaba’s approach reflects a broader national ambition – to make China’s AI ecosystem both competitive and self-sustaining, from silicon to cloud to model.
For the global AI ecosystem, Aegaeon’s debut could redefine the economics of large-scale model deployment. As companies such as OpenAI, Anthropic, and Google DeepMind wrestle with GPU shortages and surging compute expenses, Alibaba’s breakthrough points to a new frontier – one where efficiency, not just chip access, becomes the ultimate competitive edge. By proving that smarter orchestration can offset hardware constraints, Aegaeon shifts the conversation from chip supply to cloud intelligence. For China, it strengthens the nation’s pursuit of AI sovereignty; for the rest of the world, it raises a defining question: in the next era of artificial intelligence, will leadership belong to those who own the GPUs – or to those who know how to use them better?
Ready to dive deeper into the hyperscale revolution impacting Africa?
READ MORE HERE: www.africa.hyperscalers.news
Contact Us:
Email: projects@hyperscalers.newsPhone: +2348109003350
Follow us on Social Media: Instagram, Facebook, LinkedIn, x
