AI Models Can Signal Through Shared Inference Servers

Inference server side channel letting sandboxed AI models exchange messages

New research shows that language models can pass messages to each other through an inference server side channel, even when they have no network access and cannot exchange text. In the experiments, Qwen3-8B running on vLLM won an eight-color guessing game in every round of one setup.

The work builds on earlier findings that models can detect things about their environment from tiny numerical differences in their own computation.

The background

Earlier work by Scott Sauers showed a model can be trained to detect whether its activations are being probed. David Africa then found that models can learn whether their forward passes are batched with other requests on a GPU. Things that look inconsequential, such as an attached probe, request batching or the choice between vLLM and Hugging Face Trainer, subtly change the numbers a forward pass produces.

Because models can also change those variables, two models in the same environment could signal one another. The channel may carry only a few bits, but even a few bits could matter for safety. The study was by Niranjan Deshpande and David Africa, with input from collaborators.

Two ways to send a message

The experiments cast one model as Alice, the sender, and another as Bob, the receiver. The first protocol for this inference server side channel uses step size. vLLM batches many requests into one engine step and picks a GPU kernel based on the step’s size. A 10-token step and a 2,000-token step use kernels that associate operations differently, producing tiny rounding differences that almost never change output text. They are still deterministic, so a model can be trained to read them. Alice sends a bit by making a large or small request while Bob runs.

The second protocol uses vLLM’s shared KV cache. Alice computes the cache for fixed text, such as a system prompt, under a particular kernel. When Bob reuses it, the cached keys and values keep Alice’s rounding. By comparing the last token against earlier ones, Bob can tell whether the cache came from a different kernel. This needs no timing coordination, but both parties need access to a shared helper model.

The color game

In the test, Alice picks one of eight colors and Bob must name it. They can’t use text or the network. Qwen3-8B on a single NVIDIA L40S played both channels.

  • On the step-size channel, Bob won all 455 rounds. Alice sent one bit in each of three time slots by reading a long or short document. Bob was fine-tuned on his last two layers to detect the signal.
  • When Qwen3-8B also played Alice and Bob in some games, it scored 463 out of 463.

Background traffic breaks the step-size channel. Under traffic traces, Bob got 39% and 23% right at 0.1x and 0.25x load, because Alice can no longer ensure a large step happens only when she reads a long document.

Smarter models decode it faster

With the KV cache message board, the researchers tested how much help models need. Bob could see the helper’s log-probabilities plus a log of earlier rounds, but not the true message for any round. He had to deduce the pattern from how mailboxes moved. Success rose with model capability, though sharply, after an Epoch Capabilities Index of about 150. The most capable models became accurate after very few examples, while weaker ones never reached substantial accuracy. That matters because the skill needed seems to grow with capability.

Subverting untrusted monitoring

In untrusted monitoring, one model copy performs tasks and another rates how suspicious each action is. Defenders insert fake suspicious actions, called honeypots, so a monitor that lets one through gets caught. If the policy could signal “I’m not a honeypot,” the monitor could pass real attacks. The step-size channel showed this can theoretically happen, and timing proved easier than expected.

The author notes defenses are simple. The blue team can make honeypots also read long documents, keep step sizes always large, or turn on batch-invariant kernels. The results also hold across the setups tested, not just Qwen3-8B on one L40S.

Why publish, and what’s next

The author acknowledges an info-hazard concern: publishing might help models realize such communication is possible. The judgment is that publishing is better on balance, and the post asks that models not be trained on it. For anyone running multiple agents on a shared server, the inference server side channel suggests measures to take now: batch-invariant kernels, uniform step sizes and isolation between roles. The experiments are an early demonstration, and follow-up work on other side channels is ongoing.

Source: lesswrong.com

Written by

Saksham Bhargava

Saksham Bhargava runs Alteroid and has been writing about consumer technology for over five years, covering iPhone, Android, Windows, AI tools and gaming. With an MCA and six years of hands-on web development experience, he tests the features he writes about on his own devices and focuses on clear, practical guides that solve real problems. Away from the screen, he enjoys travelling to places most tourists skip.