Unified Virtual Memory (UVM) & Zero-Copy Streaming

Bridging Host and Device with Unified Memory

Standard GPU computing requires distinct memory spaces: host RAM (CPU) and device VRAM (GPU), separated by the PCIe or NVLink bus. Unified Virtual Memory (UVM) creates a single virtual address space accessible by both CPU and GPU.

Page Migration Engine Mechanics

When the CPU accesses a UVM pointer, memory resides in system RAM. When a CUDA kernel on the GPU accesses the same pointer, the hardware Page Migration Engine automatically migrates the memory pages across PCIe 5.0 / NVLink into GPU HBM3 memory on demand. In safe Rust, UVM is orchestrated through managed memory allocations:

// Pseudocode for CUDA Managed Allocation FFI
pub struct ManagedBuffer {
    ptr: *mut T,
    len: usize,
}

impl ManagedBuffer {
    pub fn new(len: usize) -> Self {
        let mut ptr = std::ptr::null_mut();
        unsafe {
            // cudaMallocManaged: Accessible by CPU and GPU
            libc_cuda::cudaMallocManaged(&mut ptr as *mut _ as *mut _, len * std::mem::size_of::(), 1);
        }
        Self { ptr, len }
    }
}