What it means
Latency is the delay before a result arrives — how long you wait for an AI model to start responding. For interactive uses like chat, code assistance or voice, low latency is essential: a slow answer breaks the experience even if it is correct. Latency is shaped by the speed of the accelerator, how fast it can reach memory, and how efficiently work is moved across the cluster, which is why it ties back to bandwidth and networking. Operators trade latency against cost and throughput constantly, and the balance they strike helps determine which hardware and which custom silicon win.
Why it matters to investors
Latency requirements steer real purchasing decisions — they favour particular accelerators, custom inference silicon and faster memory and links. For investors it is a reminder that the AI hardware race is not only about raw power but about responsiveness, which shifts value toward the parts of the chain that cut delay.
Companies on this part of the chain
Named to show where the term sits in the AI supply chain — research, not advice, and never a recommendation to buy or sell.
Related terms
See Latency in the live AI chain.
THE ENTITY maps every constraint onto one live model — which part is tight now, who owns it, and who gets squeezed when it moves. Plain-English reads you can check.
THE ENTITY is an educational read on the AI supply chain — research, not investment advice. It explains how the chain works and who sits where, never price targets or buy/sell calls.