Rust PTX Compiler Matches NVCC, Builds 10x Faster

Rust PTX compiler benchmarked against NVCC on GPU kernels

A student hackathon has put a hard question to GPU programmers: can a hand-built PTX compiler beat NVIDIA’s NVCC? The team behind Emmy, an open-source compiler for LLM inference, ran the contest and then swapped NVCC out of its own pipeline. Its new Rust PTX compiler matches NVCC at the median and builds a typical kernel in 13 ms where NVCC needed 173 ms.

Why Emmy dropped NVCC

Emmy traces a model, fuses its operations into kernels, and picks each kernel’s schedule (tiling, staging, splits) from measured evidence. It writes the result in a typed loop language called Kernel IR. The final stage used to print Kernel IR as CUDA C++ and hand it to NVCC.

That worked, but slowly. NVCC has to parse C++ and repeat optimizations Emmy had already done, and it can’t see the kernel’s structure. Some large kernels took minutes to build at -O3, which dragged down auto-tuning. Emmy also had little say over the PTX that came out.

So the team wrote a small Rust PTX compiler, a crate that compiles Kernel IR straight to PTX, with PTXAS assembling the final code.

Hero image for Can you beat NVCC with a Rust PTX compiler? - GPU, CUDA, Compilers, AI article
Image: kernelspace.substack.com

What the hackathon found

The team worried the new compiler would be slower, since NVCC uses hardware knowledge that NVIDIA doesn’t publish. To test that belief, it ran a student contest. Entrants wrote a PTX compiler for Kernel IR, and organizers scored each one against NVCC on hidden tests. There were 57 entries.

Two won. The first, from shubham-jadhav-FiatLux, ran 1.34x faster than the NVCC build on the hidden tests and 1.54x faster on large shapes. The second, from N00bcak, reached 1.24x and 1.53x.

A 50% gain looked suspicious, and it was. The speedup came from schedule changes, not code generation. The winners recognized specific kernels and applied high-level tricks such as sharing work across lanes, re-tiling attention and fusing sibling loops. Without those, they beat NVCC by only 1.01x and 1.02x across the 257 hidden tests, and by 1.03x and 1.04x on large shapes, measured on an RTX 5090. Most of the remaining entries lost to NVCC.

Entrants also did worse on kernels outside the hidden set, so the team tuned its own compiler on a much larger corpus on six GPU models, from V100 to RTX 5090.

table
Image: kernelspace.substack.com

Why not LLVM

One entrant went through LLVM IR on the way to PTX. LLVM lacks instructions like cp.async and ldmatrix, which forced the author into extensive inline assembly. That entry failed some hidden tests and compiled slowly, so Emmy kept its direct approach.

table
Image: kernelspace.substack.com

What it took to match NVCC

Closing the gap on long-tail kernels was the hard part. Nearly every slow kernel traced back to a short list of causes:

  • Division.: This was the biggest single loss. Emmy compiled each floor division into a signed 64-bit divide with a sign correction, and one attention kernel contained nine of them. NVCC’s CICC instead emits three unsigned divides, each taking a cheaper 32-bit path when the operands are small enough. Emmy had to prove its operands were non-negative to do the same.
  • 64-bit addresses.: PTXAS only folds addresses into base-plus-constant form across unrolled iterations when they’re computed in 64-bit arithmetic. With 32-bit indices it assumes wraparound is possible. The fix cut one split-K matmul from 46.2 to 37.4 microseconds.
  • Unroll budget.: Emmy now writes out each of CICC’s loop copies itself and marks loops nounroll. Without the marker, PTXAS unrolled one SDPA kernel into 11,344 machine instructions and took half a second to assemble it. With it, the kernel shrank to 2,216 instructions, assembled in 0.06 seconds and ran twice as fast.
  • Wider loads.: Loading two fp8 bytes with a single 16-bit read brought the NVFP4 GEMM level with NVCC. On V100, one 128-bit shared-memory load in place of eight 16-bit ones moved Emmy past it.
  • Predicates.: Keeping a compare result in a predicate register, instead of converting it to an integer early, closed the gap on the MoE router.

Where NVCC leaves speed on the table

One change didn’t come from NVCC at all. The .minnctapersm directive tells PTXAS how many blocks must fit on a multiprocessor at once, and by default PTXAS assumes several and holds registers back. Many Emmy launches are a single wave, so no two blocks ever share a multiprocessor. For those, Emmy writes .minnctapersm 1 and lets PTXAS use every register. NVCC never adds that hint, since it can’t know how a kernel will be launched.

Scorecard and compile times

A few outliers still differ from NVCC. On the plus side, V100 Llama-3.1-8B pieces run up to 1.9x faster (76 versus 146 microseconds), an H100 512×512 matmul takes 10.3 versus 11.5 microseconds, and six RTX 5090 lane kernels gain over 5%. On the minus side, one V100 GEMM is 1.11x slower, an H100 fp32 matmul with a dynamic size runs 1.08x slower, and a few kernels run 1.1x to 1.2x slower because PTXAS chooses a different load order for near-identical PTX. Every rewrite the team tried won one case and lost another.

Compile time is the clearer win. Across the 269 corpus kernels NVCC can still build, timed four at a time on an RTX 5090 box with CUDA 13.0, the Rust compiler is roughly 10x faster. Much of NVCC’s time goes into parsing CUDA headers in its C++ front end.

Trying the Rust PTX compiler

The code lives in the cloudrift-ai/emmy repository and ships in the latest Emmy release on PyPI. Install it with pip install emmy-ml, then run emmy compile with the –ir ptx and –target sm_120 flags to see PTX for an RTX 5090.

The Rust PTX compiler is already part of Emmy, so users get the shorter build times without changing anything.

Source: kernelspace.substack.com

Written by

Saksham Bhargava

Saksham Bhargava runs Alteroid and has been writing about consumer technology for over five years, covering iPhone, Android, Windows, AI tools and gaming. With an MCA and six years of hands-on web development experience, he tests the features he writes about on his own devices and focuses on clear, practical guides that solve real problems. Away from the screen, he enjoys travelling to places most tourists skip.