GPU-Only and Quality-First Processing Principles

Experiment Overview

Context

During performance optimization of Notebook 2 (Cycle Processing), a "fast mode" was proposed that would reduce BaSiC iteration parameters to speed up processing. The user explicitly rejected this approach, establishing core principles for KINTSUGI processing.

Scientific imaging requires quality-first processing. Unlike consumer applications where "good enough" may be acceptable, multiplex immunofluorescence analysis depends on accurate quantification. Quality degradation compounds through the pipeline: illumination correction errors affect stitching, which affects deconvolution, which affects segmentation, which affects all downstream analysis.

Core Principles

1. NEVER Sacrifice Quality for Speed

Quality parameters must remain at their scientifically-validated defaults unless the quality impact is negligible (verified, not assumed).

# CORRECT: Quality parameters (do not reduce)
BASIC_IF_DARKFIELD = True
BASIC_MAX_ITERATIONS = 500
BASIC_OPTIMIZATION_TOLERANCE = 1e-6
BASIC_MAX_REWEIGHT_ITERATIONS = 25
BASIC_REWEIGHT_TOLERANCE = 1e-3

2. ALWAYS Use GPU When Available - No CPU Fallback

If a GPU is available, it must be used. CPU fallback options should be disabled or removed.

# CORRECT: GPU enforcement
if not USE_GPU:
    raise RuntimeError(
        "GPU not available but required for processing.\n"
        "Check GPU status with: from kintsugi.gpu import get_gpu_manager; "
        "print(get_gpu_manager().summary())"
    )
use_gpu = True  # Always True - GPU required

3. Remove CPU Options When GPU Exists

Don't provide use_cpu or use_gpu=False options. If the system has a GPU, use it.

# WRONG: Providing CPU option
def process(use_gpu=True):  # Allows use_gpu=False
    ...

# CORRECT: GPU-only
def process(device_id=None):  # GPU assumed, only device selection
    if device_id is None:
        device_id = GPU_DEVICE_IDS[0]
    ...

Failed Attempts (Critical)

Acceptable Optimizations

These optimizations improve speed WITHOUT sacrificing quality:

Key Insights

• Quality is non-negotiable - Scientific imaging requires accurate quantification
• Speed comes from better hardware, not shortcuts - Invest in GPUs, not reduced iterations
• Errors compound - A 5% error in illumination correction becomes 10%+ by segmentation
• "Fast mode for testing" is a trap - Test with production parameters or you'll miss production issues
• CPU fallback is never needed - If no GPU, the user should know immediately, not get silent degradation

Implementation Pattern

# GPU enforcement at module level
if not USE_GPU:
    raise RuntimeError("GPU required for KINTSUGI processing")

# Function signatures - no CPU options
def process_zplane(
    ...,
    device_id: int = None,  # Which GPU, not whether to use GPU
):
    """GPU is REQUIRED - no CPU fallback."""
    if device_id is None:
        device_id = GPU_DEVICE_IDS[0]

    # Use validated quality parameters
    corrector = KCorrectGPU(use_gpu=True, device_id=device_id)
    flatfield, darkfield = corrector.fit(
        images,
        max_iterations=500,           # Quality parameter - DO NOT REDUCE
        max_reweight_iterations=25,   # Quality parameter - DO NOT REDUCE
        optimization_tolerance=1e-6,  # Quality parameter - DO NOT REDUCE
    )

References

• KINTSUGI Notebook 2: Cycle Processing
• BaSiC paper: Peng et al., Nature Communications 2017
• Skills Registry: basic-caching-evaluation (another quality-compromising approach that failed)