Rust Meets LLVM Offload: Memory-Safe GPU Code Matches CUDA Speed

Researchers have shown that Rust code offloaded to GPUs via LLVM infrastructure can hold its own against finely tuned CUDA and HIP kernels. The work arrives at a moment when high-performance computing teams hunt for ways to curb memory bugs without giving up speed. And the numbers look promising.

A paper released this week details tests where safe Rust kernels delivered performance competitive with hand-optimized C++ on both NVIDIA and AMD accelerators. In several cases the Rust versions pulled ahead. In others they trailed. The gaps, authors say, trace to differences in generated code and intermediate representation rather than any fundamental limit.

Phoronix first reported the findings on the same day the research appeared. Michael Larabel highlighted the central claim: “using LLVM’s Offload infrastructure and the characteristics of the Rust programming language can still lead to competitive performance against native, hand-optimized CUDA and HIP C++ kernels.” The article pointed readers to the arXiv preprint and Rust’s tracking issue.

The team behind the study includes Manuel S. Drehwald, Marcelo Domínguez, Kevin Sala, Alán Aspuru-Guzik, and Johannes Doerfert. Their approach relies on LLVM’s relatively new offload component, the same machinery OpenMP already uses to ship Fortran and C++ code to accelerators. Rust’s front end feeds into that path. The result is a single source base that compiles once for the host and once for the device.

But here’s the twist. The Rust kernels stay safe. No raw pointers in the hot paths. The front end presented in the talk “Taming GPU programming with safe Rust” handles most common patterns without exposing programmers to the usual footguns. Automatic data movement between host and device memory appears high on the wish list. So does support for nearly all Rust types and functions.

Current Rust compiler targets already exist for NVIDIA’s PTX and AMD’s GCN. They sit at Tier 2 and Tier 3 respectively. The offload work builds on them. It adds a cargo wrapper, a manifest that lists kernels, and a three-pass compilation strategy. Host code runs first to emit the list of kernels. Device compilation follows. Then the host links against the offload runtime that manages data layout and transfers.

Some rough edges remain. Developers must sprinkle cfg annotations on device code today. Device compilation re-runs for every kernel instance because monomorphization roots aren’t cached yet. The Rust team lists these as lower priority. Features and runtime performance come first, according to the Rust Compiler Development Guide.

That same guide spells out the ambition. The eventual std::offload module should feel rusty. Safe by default. Convenient. Fast enough without constant tuning. Unsafe escape hatches stay available for experts who need finer control over shared memory or warp intrinsics.

Performance data in the new paper comes from the RAJA Performance Suite. Only a subset of the benchmarks could run under fully safe Rust. Vector addition worked cleanly. More complex indexing patterns required hiding small unsafe blocks inside the compiler or library wrappers. Even then, the generated code stayed competitive.

In one NVIDIA test the Rust offload version beat the CUDA baseline. On AMD hardware the HIP reference sometimes won by low double-digit percentages. Authors attribute the differences to LLVM’s codegen for the two languages and to aliasing information that Rust provides but the offload pipeline doesn’t yet fully exploit.

Separate work at NVIDIA Labs has taken a different route. The cuda-oxide project compiles Rust straight to PTX through a custom pipeline involving Stable MIR and an intermediate Pliron representation. Early results show a naive GEMM kernel hitting 868 TFLOPS on recent hardware. That effort surfaced in discussions on X earlier this year and demonstrates growing industry interest in Rust for GPU kernels.

Johannes Doerfert, a frequent LLVM contributor, has spoken about these ideas at recent developer meetings. His LLVM Dev 2025 talk slides outline how offload intrinsics let the same Rust binary target multiple vendors without separate back ends. The same machinery supports Enzyme for automatic differentiation, opening doors in scientific computing where Aspuru-Guzik’s group works.

Rust’s tracking issue for GPU offload, opened in late 2024, remains open. The feature sits behind #![feature(gpu_offload)] and carries an experimental label. An RFC is expected before any stabilization. Compiler, language, and bootstrap teams all have seats at the table. Progress reports appear in the 2025h1 Rust project goals, which list exposing the LLVM offload capabilities as a concrete target.

Industry watchers see two parallel threads. One is the upstream Rust effort focused on safety and a clean standard-library interface. The other is vendor-specific experiments such as cuda-oxide that chase peak performance today. Both feed the same LLVM offload layer. Both benefit from improvements to LLVM’s GPU code generation that have accelerated in the past two years.

Memory safety on the GPU matters. C++ kernels routinely suffer from use-after-free bugs when data movement and kernel launch ordering go wrong. Rust’s borrow checker can prevent entire classes of those errors at compile time. The new research shows that safety need not come at a steep performance cost once the compiler understands the constraints.

Of course. Real applications will mix safe and unsafe code. Libraries such as ndarray or faer already rely on raw pointers internally. The offload work lets those libraries clone their data structures for the device, preserving Rust’s ownership rules while giving the GPU the pointers it needs.

Optimizations discussed in the talk include copying data only in the required direction, allocating buffers on the device when possible, using shared memory, and fusing kernels. These techniques appear in hand-written CUDA too. The difference is that Rust abstractions could make them easier to apply correctly.

The paper acknowledges remaining work. Better alias analysis inside LLVM could close some of the performance gap. Teaching the offload runtime to reuse device allocations across kernel calls would cut overhead. And a polished user-facing crate that hides the remaining unsafe blocks could accelerate adoption.

Adoption matters. High-performance computing shops already experiment with Rust for host-side control planes. Moving kernels into the same language reduces context switching for developers and for the compiler. It also lets borrow checking span the host-device boundary in ways that static analyzers for C++ cannot match.

Yet challenges persist. Compile times for device code remain higher than many teams accept. Tooling for debugging kernels written in Rust lags the mature CUDA ecosystem. And not every algorithm maps neatly to the safe subset demonstrated so far.

Even so. The competitive performance numbers remove the biggest objection. Teams no longer need to choose between safety and speed. They can start with safe Rust kernels and drop into unsafe only where profiling demands it.

The Rust compiler team has made clear that runtime performance and feature completeness take precedence over internal refactoring for now. That stance aligns with the needs of scientific and financial workloads that dominate GPU usage. Once the interface stabilizes, the focus can shift to squeezing the last few percent from the generated code.

Outside the main research thread, community conversations on X this summer highlighted bounds-check overhead in GPU GEMM loops. One NVIDIA engineer noted that safe Rust slice indexing can impose a 2.4x penalty in hot kernels because the proof of buffer size lives on the host side and does not always propagate into the device IR. Solutions under discussion include new intrinsics or attributes that let the compiler elide checks when safe.

Those concerns echo the paper’s findings. The Rust offload path is capable. It is not yet optimal. Closing the gap will require tighter cooperation between the Rust front end, the LLVM middle end, and the offload runtime.

Still, the trajectory looks clear. Safe Rust on GPUs moved from experimental curiosity to competitive option in a single research effort. With upstream support advancing and vendor experiments delivering real TFLOPS, production use cases cannot be far behind.