Pretraining

Pretraining is teaching a model its general skills by running it over huge amounts of data before it is specialized for any task. It is the step that converts raw compute capacity into base-model capability — the foundation on which everything else, from fine-tuning to serving, is later built. It matters now because frontier models increasingly need ever-larger blocks of tightly-connected compute working as one coherent cluster, which ties the pace of capability directly to scarce, hard-to-assemble compute.

If the compute runs short, the pace of frontier progress slows — capability is gated by how much coherent compute a lab can marshal, not by ambition. The exposed names are the frontier and challenger labs whose roadmaps depend on securing those clusters — Safe Superintelligence, DeepSeek, Zhipu AI, Moonshot AI, and MiniMax — alongside data and infrastructure suppliers such as Scale AI that feed the training process.