Skip to content
AI supply chain term

Collective communication (all-reduce)

All-reduce is the collective communication step in which every chip in a cluster combines its partial results and ends up with the same final total — the core exchange that keeps distributed AI training in sync.

What it means

When a model is trained across many accelerators, each computes gradients on its slice of data; an all-reduce combines those gradients across every chip and hands each one the identical averaged result before the next step. This collective operation — implemented by libraries such as NVIDIA's NCCL — runs constantly during training, so its speed often decides how efficiently a giant cluster actually trains. It stresses the tightest, fastest links, the scale-up fabric that joins chips into one closely coupled machine, because all-reduce traffic is heavy and synchronous. It is a lever because faster collectives raise the utilization of the whole cluster; it is a constraint because if the interconnect cannot keep up, expensive accelerators sit idle waiting to exchange results, and effective throughput collapses well below the chips' peak.

Why it matters to investors

All-reduce performance ties directly to interconnect quality, so the scale-up fabric that carries it is a competitive battleground. NVIDIA (NVDA) sets the pace with NVLink and NCCL, connectivity supplier Astera Labs (ALAB) sells the links around it, and GPU clouds such as CoreWeave (CRWV) and Nebius (NBIS) depend on it to keep utilization high.

Companies on this part of the chain

NVIDIANVDAAstera LabsALABCoreWeave, Inc.CRWVNebius Group N.V.NBIS

Named to show where the term sits in the AI supply chain — research, not advice, and never a recommendation to buy or sell.

Related terms

NVLinkNVLink is NVIDIA's high-speed interconnect that links GPUs directly so many chips can act as one tightly-coupled machine.Scale-out vs scale-upConnecting many machines into one cluster over a network fabric.Network fabricThe high-speed network that ties accelerators together into a cluster.CUDACUDA is NVIDIA's proprietary software platform that lets developers program its GPUs for general-purpose computing, including AI training and inference.

See Collective communication in the live AI chain.

THE ENTITY maps every constraint onto one live model — which part is tight now, who owns it, and who gets squeezed when it moves. Plain-English reads you can check.

Explore the features
All glossary terms

THE ENTITY is an educational read on the AI supply chain — research, not investment advice. It explains how the chain works and who sits where, never price targets or buy/sell calls.