Agentic Inference Capacity Grew 4.6x in a Year

Chart of agentic inference capacity growth by AI chip family from Q2 2025 to Q2 2026

The amount of agentic inference capacity in AI chips sold to date grew 4.6 times between the second quarter of 2025 and the second quarter of 2026. Peak 8-bit computing power across the same chips rose just 2.6 times. The gap shows that newer hardware is getting better at the long-context, tool-heavy workloads AI agents run, not only at raw arithmetic.

How the capacity figure is built

The estimate uses an index of per-chip inference scores, expressed in GB300 equivalents. Each score reflects a fixed set of agentic workloads with long cached contexts. The same workload mix is applied in every quarter, so the comparison stays consistent over time.

The index accounts for memory, communication between chips and lower-precision computation. It covers both reading the prompt and generating the output. Analysts multiplied the cumulative number of chips sold in each category by its score and summed the results. They then divided by the Q2 2025 total.

For comparison, they ran the same sums on peak 8-bit compute. That is the figure behind the 2.6 times rise.

Blackwell drives the jump

Capacity added by chips sold in Q2 2026 was 3.3 times what Q2 2025 shipments added. Much of that came from the move from Hopper to Blackwell. By Q2 2026, Blackwell and Blackwell Ultra made up roughly 73% of total agentic inference capacity, against 50% of total peak compute.

Fitting a trend to quarterly additions gives annual growth of 3.2 times for inference capacity and 2.1 times for peak compute. Over a longer window, from Q1 2024 to Q2 2026, the fitted rates are 6.8 and 2.5 times a year. Those equal doubling times of 4.3 and 9.2 months. The longer window includes the first Blackwell transition, and the figures describe the past rather than forecast the future.

Why capacity outpaced compute

Inference capacity per unit of peak compute rose about 77% over the year. The analysis credits hardware-model co-design, a catch-all for gains beyond proportional increases in arithmetic throughput and memory bandwidth. Examples include larger scale-up domains, lower-precision math and model architectures that fit the hardware better.

Long agent sessions lean heavily on memory and chip-to-chip links, which is why those upgrades count for more than raw operations per second.

How sure are the numbers?

Shipment figures come from a chip-sales dataset with its own uncertainty ranges. The analysts also varied the share of GB-series chips from 30% to 90% and the H200 share of eligible shipments from 30% to 70%. The 90% intervals came out at 4.2 to 5.1 times for inference growth and 2.5 to 2.7 times for peak compute. A scenario with an earlier, faster H200 transition trimmed the headline from 4.6 to 4.5 times.

There are limits. Several assumptions are judgment calls:

  • 60% of GPUs in each combined pair are rack-scale GB/NVL72 systems and 40% are B/HGX servers.
  • H100 and H200 shipments are split evenly from Q4 2024, and earlier ones count as H100.
  • Software efficiency differences are not modeled.

The intervals cover shipment and hardware-mix uncertainty only. They do not cover error in the modeled performance scores.

Why growth should slow from here

Holding 4.6 times annual growth would require yearly additions equal to 3.6 times the capacity at the start of the year. That bar rises as the installed base grows. Also, the biggest co-design gains came with the Hopper-to-Blackwell shift. Modeled inference capacity per GPU rises 8 times from H200 to GB300 but only 2.2 times from GB300 to Vera Rubin VR200.

Faster shipment growth could make up some of that difference. The expectation is that additions keep increasing, but at a lower percentage rate. Future improvements might come from raising memory bandwidth faster than arithmetic throughput.

What to watch in the next few quarters

The next data points are Vera Rubin shipments and the Blackwell-to-Rubin mix. If the generational gain stays near 2.2 times per GPU, agentic inference capacity growth will depend on how many more chips get built. For buyers of AI services, that points to supply as the main limit on cheaper, longer agent runs.

Source: Epoch AI

Written by

Saksham Bhargava

Saksham Bhargava runs Alteroid and has been writing about consumer technology for over five years, covering iPhone, Android, Windows, AI tools and gaming. With an MCA and six years of hands-on web development experience, he tests the features he writes about on his own devices and focuses on clear, practical guides that solve real problems. Away from the screen, he enjoys travelling to places most tourists skip.