Moonshot AI’s Kimi K3.1 Pro is the strongest open-weight coding model on several agentic benchmarks, but the lead comes with a warning. An independent audit says part of its Terminal-Bench score may come from the model recalling hidden test rules. The weights are public, yet running them takes hardware most people don’t own.
Moonshot opened the API on October 23, 2026 and posted the weights to Hugging Face on November 2.
What Kimi K3.1 Pro is
K3.1 Pro builds on Kimi K3 with further training and keeps its size: roughly 2.78 trillion parameters, of which 104 billion fire for any given token. It accepts images and reads up to 1M tokens of context. Each of its 93 layers is mixture-of-experts: a token goes to 16 of 896 specialists plus 2 shared ones.
Attention is a hybrid. Sixty-nine layers use Kimi Delta Attention, a linear design whose state stays the same size however long the context grows. The other 24 use Gated DeepSeek Sparse Attention, which replaces the full-attention layers in K3 to make long contexts cheaper. Images go through a separate 401M-parameter vision encoder, MoonViT-V2.
Moonshot trained the model with low-precision quantization built in, so the weights ship in MXFP4 with MXFP8 activations. The full download is about 1.56TB.
The benchmark audit
On November 3, R40 Research’s Joachim Schaeffer and Tilman Rรคuker reported that K3.1 appears to recall hidden test rules. In 23% of its Terminal-Bench 4.0 attempts, the model’s reasoning cites “instructions” missing from the task text yet enforced by the grader. K3 never did this.
Two further tests pointed the same way. Asked to list its instructions, K3.1 named hidden requirements in 65% of answers, against 18% for K3. When the researchers rewrote a task to contradict one such rule, K3.1 stuck with the hidden rule and failed three of four tasks, while other models followed the new wording.
The authors suspect the model was trained on the benchmark tasks using a teacher that had seen the grading rules, but say they can’t prove it or tell whether it was deliberate. Moonshot hadn’t responded publicly when this was written.
The practical reading for buyers: discount the Terminal-Bench lead and rely on your own tests.
The model card’s footnotes add two caveats. Moonshot ran its coding benchmarks inside the Claude Code setup but took most rivals’ numbers from their own labs’ reports, so the test conditions differ. The hallucination benchmark is Moonshot’s internal test. An independent index scores K3.1 Pro at 46, with output at 31 tokens per second and a wait of about 156 seconds before the first answer token.
What it takes to run locally
Long context is cheap here despite the size. Only the 24 sparse-attention layers store a cache for each token. Using a published figure of 33,792 bytes per token, that comes to about 4.3GB at 128K tokens and about 35GB at the full 1M. Memory for the weights is the real obstacle.
llama.cpp support merged on November 6, with vision following on November 10. vLLM supports the model. Apple’s MLX has no official support yet, so Mac builds rely on patched code. Every speed figure so far comes from the people who uploaded the builds, one setup each, with no independent reproduction.
- Full quality calls for an eight-GPU server.
- The cheapest way to run the whole model at a usable pace is four 512GB Mac Studios linked together, which costs five figures.
- A single Mac can run an 80%-pruned build, but its own card warns of real degradation outside code.
- Streaming experts from an SSD gets the model onto two consumer GPUs, but a 2,048-token prompt then waits 9 to 13 minutes for its first token.
Ignore any repository that promises the model in 12GB or 16GB of memory. With 104B active parameters, it can’t fit.
API pricing and license
Moonshot’s API costs $2.00 per million input tokens and $10.00 per million output tokens, with a 90% cache discount and batch jobs priced at 60% of standard. Buying four Mac Studios to avoid those fees would take a very long time to recoup.
The license is permissive with two conditions. A company that sells hosted access to the model needs a separate deal with Moonshot once it earns more than $20 million in any 12 months. A commercial product with over 100 million monthly users, or over $20 million a month in revenue, has to display the Kimi K3.1-Pro name prominently. Internal use falls under neither.
Should you download it?
For almost everyone, no. The long-horizon coding results and low hallucination score deserve a try, but through the API. Anyone with a rig holding 600GB or more of VRAM can run the 1-bit and 2-bit builds, and eight RTX Pro 6000 cards give good numbers. Smaller open models suit a single large Mac or a 128GB machine better.
Until Moonshot answers the audit, take the Terminal-Bench lead with a grain of salt. A clear account of the training data would settle whether the score reflects skill or recall.
Source: Vetted Consumer