High-Throughput Node.js Addons with napi-rs & PyO3

Atomic Component Replacement via napi-rs

Rather than an all-or-nothing rewrite, the Copilot team swapped components one by one using napi-rs. In Node.js applications, napi-rs allows exposing Rust functions as native Node modules with zero V8 serialization overhead:

use napi_derive::napi;

#[napi]
pub fn fast_tokenize(input: String) -> Vec {
    // Executes in native Rust with SIMD acceleration
    let tokenizer = get_active_tokenizer();
    tokenizer.encode(&input)
}

Releasing the Python GIL with PyO3

When authoring Python C-extensions in Rust, the Python Global Interpreter Lock (GIL) prevents true multi-core scaling. PyO3 allows explicitly releasing the GIL during CPU-heavy operations:

use pyo3::prelude::*;

#[pyfunction]
pub fn parallel_matrix_multiply(py: Python<'_>, a: Vec, b: Vec) -> PyResult> {
    // Release the Python GIL so other Python threads can execute!
    py.allow_threads(|| {
        // Native multi-threaded Rust compute (Rayon / SIMD)
        execute_parallel_gemm(&a, &b)
    })
}