Windows CuPy NVRTC Fix

Experiment Overview

Context

CuPy uses NVRTC (NVIDIA Runtime Compiler) for JIT compilation of CUDA kernels. On Windows, the NVRTC DLLs are installed with CUDA but not automatically added to the system PATH. This causes CuPy operations to fail at runtime even when CUDA is properly installed.

Error Signature

cupy.cuda.compiler.CompileException: nvrtc: error: failed to open nvrtc-builtins64_121.dll.
Make sure that nvrtc-builtins64_121.dll is installed correctly.

Or:

cupy_backends.cuda.libs.nvrtc.NVRTCError: NVRTC_ERROR_BUILTIN_OPERATION_FAILURE (7)

Verified Workflow

Solution 1: Add CUDA bin to PATH at runtime (Recommended)

import os
import platform

def setup_cuda_path():
    """Setup CUDA PATH for CuPy on Windows."""
    if platform.system() != "Windows":
        return True

    # Common CUDA installation paths
    cuda_paths = [
        r"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\bin",
        r"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6\bin",
        r"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin",
    ]

    for cuda_bin in cuda_paths:
        if os.path.exists(cuda_bin):
            # Check for nvrtc DLLs
            nvrtc_files = [f for f in os.listdir(cuda_bin) if f.startswith('nvrtc')]
            if nvrtc_files:
                os.environ["PATH"] = cuda_bin + os.pathsep + os.environ.get("PATH", "")
                print(f"Added CUDA to PATH: {cuda_bin}")
                return True

    print("CUDA not found - GPU acceleration may not work")
    return False

# Call BEFORE importing CuPy-dependent code
setup_cuda_path()

# Now CuPy will work
import cupy as cp

Solution 2: For Jupyter notebooks

Add this cell at the very beginning, before any other imports:

import os
cuda_bin = r"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\bin"
os.environ["PATH"] = cuda_bin + os.pathsep + os.environ.get("PATH", "")

Solution 3: System-wide fix (permanent)

Add the CUDA bin directory to your system PATH:

1. Open System Properties > Environment Variables
2. Edit the PATH variable
3. Add: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\bin
4. Restart your terminal/IDE

Failed Attempts (Critical)

Key Insights

• Order matters: PATH must be modified before ANY import that touches CuPy
• CUDA version matching: The nvrtc DLL version (e.g., nvrtc64_121.dll) must match your CUDA toolkit
• Conda environments: Even with cuda-nvrtc package, you may need to add $CONDA_PREFIX/Library/bin to PATH
• Multiple CUDA versions: If you have multiple CUDA versions, ensure PATH points to the correct one for your CuPy version
• Verification command:

  import cupy as cp
  x = cp.array([1, 2, 3])
  print(x * 2)  # Should work without errors
  

Environment Details

Works with:

• CUDA Toolkit 11.x, 12.x
• CuPy 12.x, 13.x
• Windows 10, Windows 11
• Conda and pip installations

References

• CuPy CUDA setup: https://docs.cupy.dev/en/stable/install.html
• NVIDIA CUDA Toolkit: https://developer.nvidia.com/cuda-toolkit
• Related GitHub issues: CuPy NVRTC errors are common on Windows