Skip to content
AI supply chain term

Inference

Running a trained model to answer requests — the recurring cost of AI in production.

What it means

Inference is running an already-trained model to answer real requests — the AI you actually use when a chatbot replies or a feature generates text. Unlike training, which happens once and in a burst, inference happens continuously, at scale, every time someone uses the product. That makes it the recurring cost of AI in production, and it is increasingly the larger share of total AI compute over a model's life. Inference is sensitive to latency and to cost per token, which is why operators invest heavily in efficient chips, custom ASICs and software to make each answer cheaper.

Why it matters to investors

Inference is where AI turns into an ongoing operating cost rather than a one-time build, so it is the better lens on durable, recurring demand for compute. As usage scales, the pressure to drive down cost per token favours efficient accelerators, custom silicon and the unit-economics story — a different investment read than the training spike.

Companies on this part of the chain

NVIDIANVDAAMDAMDBroadcomAVGO

Named to show where the term sits in the AI supply chain — research, not advice, and never a recommendation to buy or sell.

Related terms

Training (pretraining)The large, compute-heavy run that creates a base model from scratch.ASIC (custom AI chip)Application-Specific Integrated Circuit — a custom chip built for one job, e.g. AI inference.Cost per tokenThe all-in price to generate model output — the convergence point of the whole stack.TokenThe unit of text an AI model reads or writes; pricing is per million tokens.LatencyThe delay before a result arrives; low latency matters for interactive AI.

See Inference in the live AI chain.

THE ENTITY maps every constraint onto one live model — which part is tight now, who owns it, and who gets squeezed when it moves. Plain-English reads you can check.

Explore the features
All glossary terms

THE ENTITY is an educational read on the AI supply chain — research, not investment advice. It explains how the chain works and who sits where, never price targets or buy/sell calls.