The NVIDIA Rust Ecosystem: cudarc & Driver API Wrappers

NVIDIA's Direct Rust Revolution

For years, running CUDA from Rust required cumbersome C++ wrapper libraries or slow PyTorch FFI bindings. Today, NVIDIA directly embraces Rust through ecosystem crates like cudarc, which provide safe, zero-cost abstractions over the raw CUDA Driver API (libcuda.so / nvcuda.dll).

Core cudarc Architecture

use cudarc::driver::{CudaDevice, LaunchAsync, LaunchConfig};
use std::sync::Arc;

pub fn init_cuda_runtime() -> Result, Box> {
    // 1. Initialize CUDA Driver and select GPU ordinal 0
    let dev = CudaDevice::new(0)?;
    println!("Initialized CUDA Device: {}", dev.name()?);

    // 2. Allocate 1,000,000 f32 elements directly in GPU device VRAM
    let host_data: Vec = vec![1.0; 1_000_000];
    let device_slice = dev.htod_copy(host_data)?; // Host-to-Device Copy

    // 3. Inspect GPU memory footprint
    println!("Allocated {} bytes on GPU", device_slice.len() * 4);
    Ok(dev)
}

Asynchronous Streams & Hardware Events

CUDA operations do not execute sequentially on the CPU thread; they are enqueued into CUDA streams. Streams allow concurrent kernel execution and overlapped host-to-device memory transfers without blocking the host Rust thread.