Medical Imaging (AI)

Medical imaging AI refers to machine learning models trained to analyze medical scans like X-rays, CTs, MRIs, and pathology slides. Operators encounter these as specialized model weights (e.g., a DICOM-compatible segmentation model) that process medical images for tasks such as tumor detection, organ segmentation, or fracture classification. These models often require high-resolution inputs and may need 16-bit precision to preserve diagnostic detail, pushing VRAM requirements beyond typical LLM workloads. Running them locally demands careful quantization trade-offs to maintain clinical accuracy.

Medical imaging AI models are typically convolutional neural networks (CNNs) or vision transformers fine-tuned on large, annotated medical datasets. Unlike general-purpose vision models, they must handle DICOM format images with 12-16 bit depth, high resolution (e.g., 512x512 to 2048x2048), and strict regulatory requirements. Operators running these locally often use frameworks like MONAI (built on PyTorch) or ONNX Runtime. Quantization to 8-bit can reduce VRAM usage but may degrade sensitivity for small lesions. Models like U-Net for segmentation or CheXNet for chest X-ray classification are common. Inference latency matters less than accuracy; a single scan might take seconds to minutes depending on model size and hardware.

A radiologist wants to run a lung nodule detection model locally. The model is a 3D U-Net with ~50 million parameters, requiring ~400 MB in FP32. However, input CT volumes are 512x512x200 voxels (16-bit), needing ~100 MB per volume. The operator loads the model in FP16 (200 MB) and processes a volume on an RTX 4090 (24 GB VRAM) in ~10 seconds. Quantizing to INT8 drops VRAM to 100 MB but reduces sensitivity for nodules smaller than 3 mm.

Using MONAI, an operator loads a pretrained segmentation model: model = monai.networks.nets.UNet(spatial_dims=3, in_channels=1, out_channels=3, channels=(16, 32, 64, 128, 256), strides=(2, 2, 2, 2)). They then load DICOM files via monai.data.ImageReader and run inference with model(input_tensor). If VRAM is tight, they set torch.cuda.empty_cache() between slices or use monai.inferers.SlidingWindowInferer to process sub-volumes.