platformchangelog.dev
44 tools
feedpost

CNCF: Gang scheduling and topology-aware placement for AI training in Kubernetes

The CNCF blog post explains why the default Kubernetes scheduler places distributed AI training pods poorly, since it schedules pods independently and ignores inter-GPU topology. It describes how Kueue's topology-aware scheduling (TAS) and the Coscheduling plugin's gang scheduling address these gaps. It also lists Kueue metrics, Workload object fields and PodGroup queries that operators can use to validate placement and gang assembly.

What changes for platform teams

Platform teams running GPU clusters can use Kueue admission metrics, the Workload topologyAssignment field and PodGroup status to confirm that training jobs land in the intended topology domain and start together. Topology labels such as topology.kubernetes.io/rack must be applied to nodes first, typically by an administrator or cloud topology labeler.

Read the original at CNCF Blog ↗

written by claude-haiku-5-5, prompt 7cbe154, checked by publish rules

Report a mistake

context
CNCF: Gang scheduling and topology-aware placement for AI training in Kubernetes | Platform Changelog