A new arXiv paper studies an unexplored SASS control code field in NVIDIA GPUs and shows it can make kernels run faster. Every instruction in NVIDIA’s native GPU instruction set carries a control code that guides hardware scheduling. This particular field appears more than 500,000 times in NVIDIA’s closed-source libraries, such as cuBLAS and cuDNN, yet the nvcc compiler never emits it for user programs.
The paper was submitted on November 18, 2026 in the Hardware Architecture category. Its authors say it is the first systematic study of the field, covering GPU generations from Volta to Blackwell.
What the field controls
Using instruction-level microbenchmarks, the researchers found the field drives two separate hardware mechanisms.
- Barrier exemption, called EXEMPT, lets instructions that a pending barrier would block issue early while the barrier waits.
- Programmable warp priority, with marks named START, TILE and END, moves warps across three priority levels in the hardware scheduler.
Prior work had decoded the other control code fields to optimize kernels. This one had been left alone, even though NVIDIA’s own libraries use it heavily.
The optimizer
The team identified when each mechanism helps and built a post-compilation binary optimizer that inserts the marks directly into compiled binaries. It needs no changes to source code or the compiler, which matters because nvcc doesn’t emit the field and developers can’t ask for it.
Results
The authors tested on six GPUs across four architectures. Reported gains:
- Up to 31.9% faster on representative kernels.
- Up to 5.4% faster on production kernels in vLLM and one other library.
- Up to 3.6% faster end-to-end LLM inference.
The small end-to-end figure is realistic: kernel speedups are diluted by everything else in a serving stack, but a few percent of inference time is meaningful at data center scale.
Background: what SASS and control codes are
SASS is the native machine language of NVIDIA GPUs. Developers usually write CUDA code that nvcc compiles down to it, so most never see SASS directly. Alongside each instruction, the hardware reads a control code that tells the scheduler how to treat it. The paper says earlier research had already decoded the known fields of that code to tune kernels.
Warps are the groups of threads a GPU schedules together, and a barrier makes some of them wait for others before moving on. The two mechanisms in the paper act on exactly those two things. Barrier exemption lets instructions go ahead during a wait, and warp priority changes which warps the scheduler favors.
Why the evidence is notable
The field appears over 500,000 times in cuBLAS and cuDNN, NVIDIA’s closed-source libraries for linear algebra and deep neural networks. That volume of use means NVIDIA’s own engineers apply it widely, while developers who compile with nvcc get binaries without it.
The authors evaluated across Volta to Blackwell generations, six GPUs and four architectures, so the finding isn’t limited to one chip family. They also tested their optimizer on production kernels, not only on synthetic benchmarks.
How to read the speedups
The 31.9% figure comes from representative kernels chosen to show where the marks help. The production gains are smaller, 5.4% on kernels in vLLM and one other library, and 3.6% on full LLM inference. That gap is normal: a kernel is only one part of a serving stack, so a large local gain shrinks at the system level.
The authors also had to find the conditions under which each mechanism helps, which implies it can hurt in other cases. A tool that places marks blindly would risk slowing some kernels down. This is why their optimizer is built around a study of when to apply them.
Why this matters and what remains unknown
The result suggests that NVIDIA’s libraries rely on scheduler hints that third-party developers could not use. If the SASS control code can be set safely after compilation, framework authors might gain performance without NVIDIA’s cooperation. The abstract does not say how stable the field is across driver versions or whether NVIDIA documents or supports it, so production use carries risk.
Readers should treat this as a preprint that has not been peer reviewed. The next steps to watch are independent reproduction on other kernels, any response from NVIDIA, and whether projects such as vLLM adopt the optimizer.
Source: arXiv.org