Beam MoE Model: 501B Parameters, 23B Active

Beam MoE model with 501.3B parameters and 1M-token context window

A new open-weight model called Beam packs 501.3 billion parameters but activates only 23.4 billion of them for any given token. Its technical report, posted to arXiv on November 19, 2026, describes Beam as a sparse Mixture-of-Experts model built for coding, reasoning and agentic workloads, with a 1 million token context window.

How the Beam MoE model is built

Mixture-of-Experts models split their weights into many specialist blocks and route each token to a few of them. Beam follows that pattern. The practical effect is that the model stores a great deal of knowledge but pays the compute bill for only a small slice of it on each step. Of its 501.3B total parameters, 23.4B are active at a time, which is why its developers claim it needs 3 to 4 times less inference compute than many models of its class while still delivering frontier-level reasoning.

Pretraining used 23.8 trillion tokens of curated data. The long context came from a two-stage midtraining phase, built on data that can be verified for correctness. The report calls this approach novel. Anchoring the data to correctness matters because a model trained on checkable material, such as code that compiles or answers that can be verified, gets a cleaner learning signal than one trained on loosely filtered text.

A very large reinforcement learning run

Most of the report’s weight sits on post-training. The team ran a fully asynchronous reinforcement learning (RL) campaign across 10,500 GB300 GPUs. Over four weeks it generated more than 100 million rollouts.

The authors believe it is one of the largest RL runs by any open lab so far. They also report that capability kept rising as they added RL compute, with no sign of a plateau across their evaluation suite. That claim comes from the developers’ own tests, and independent benchmarks will show whether the curve holds.

Why the Beam MoE model targets agents

Coding and agentic work put different demands on a model than chat does. Agents read long files, call tools, check results and try again, so a single task can burn through hundreds of thousands of tokens. A large context window and cheap inference help directly with that, and both are central to the pitch for Beam.

Built for fast serving

The developers paired the model with its serving stack from the start. They describe deep hardware and model co-design, FP8 and NVFP4 quantization, and a 1.93B-parameter speculative decoding drafter called DFlash 2. Speculative decoding lets a small model propose several tokens that the large model then checks in one pass, which raises throughput without changing the output.

The stated goal is higher serving throughput with flexible deployment. For anyone running agents that make many long calls, cost per token matters as much as benchmark scores.

Safety results

The report includes refusal figures averaged across conversation turns and harm categories. Beam refuses or pivots on 88.9% of harmful requests and complies with 81.5% of harmless ones. According to the authors, that is the highest refusal or pivot rate on harmful requests among open-weight models on the Pareto frontier.

The second number is a useful check on the first. A model can score well on refusals by turning down everything, so the 81.5% compliance rate on harmless requests shows the safety result isn’t coming from blanket caution.

What to watch

  • Whether the weights are released in a form people can download and run, since the report describes an open-weight model.
  • Independent results on coding and agentic benchmarks, which will test the developers’ claims against a 3-4x compute advantage.
  • How the model behaves with its full 1M-token window in real projects, where long-context quality often drops off.

What happens next

The report is a first version, and outside evaluations usually follow within days or weeks of a release like this. If Beam holds up, the sparse design means teams could run a frontier-class coding model on far less hardware than a dense model of similar ability would need. If independent tests come in lower than the report’s numbers, the gap between published and measured performance will be the story.

Either way, the report adds to the evidence that scaling reinforcement learning is a productive path for open labs, since the authors saw gains continue as compute grew.

Source: arXiv.org

Written by

Saksham Bhargava

Saksham Bhargava runs Alteroid and has been writing about consumer technology for over five years, covering iPhone, Android, Windows, AI tools and gaming. With an MCA and six years of hands-on web development experience, he tests the features he writes about on his own devices and focuses on clear, practical guides that solve real problems. Away from the screen, he enjoys travelling to places most tourists skip.