Agentic Inference Capacity Grew 4.6x in a Year
AI chips sold over the past year added 4.6 times the agentic inference capacity, while peak compute grew only 2.6 times. Blackwell and better design drove the gap.
AI chips sold over the past year added 4.6 times the agentic inference capacity, while peak compute grew only 2.6 times. Blackwell and better design drove the gap.
A Hugging Face post shows MoE expert streaming running a 180B model on a 32GB Mac at roughly 14 tokens a second, within 6% of Strata.