OpenCV

OpenCV (Open Source Computer Vision Library) is a C++ library with Python bindings for real-time image and video processing. Operators encounter it when building pipelines that need to read, transform, or analyze visual data before feeding it into a model. Common operations include resizing, color conversion, face detection, and feature extraction. OpenCV handles camera input, file I/O, and basic ML inference via its DNN module, which can load ONNX or Caffe models. It does not natively run large language models or diffusion models, but it is often used alongside them for preprocessing images or video frames.

OpenCV provides over 2500 optimized algorithms for computer vision. Its core modules cover image processing (filtering, morphology), video analysis (motion estimation, object tracking), camera calibration, and machine learning (k-NN, SVM, decision trees). The DNN module (introduced in OpenCV 3.3) supports inference with models from frameworks like TensorFlow, PyTorch, Caffe, and ONNX. For operators, OpenCV is typically used in the data ingestion stage: reading frames from a webcam or video file, applying transforms (resize, normalize, color space conversion), and passing the resulting tensor to a model. It can also run lightweight models (e.g., MobileNet-SSD for object detection) directly, but for large models (e.g., YOLOv8, Stable Diffusion) the preprocessing is done in OpenCV and inference is delegated to PyTorch or ONNX Runtime. OpenCV is not a deep learning framework itself; it is a tool for preparing visual data.

An operator building a real-time object detection pipeline might use OpenCV to capture frames from a USB camera at 30 FPS, resize each frame to 640x640, convert BGR to RGB, normalize pixel values to [0,1], and then pass the tensor to a YOLOv8 model running in PyTorch. Without OpenCV, the operator would need to write custom code for camera I/O and image manipulation. On a rig with an RTX 3060, OpenCV's GPU-accelerated cv::cuda module can speed up resizing and color conversion, reducing preprocessing latency from ~5 ms to ~1 ms per frame.

In a typical workflow using Hugging Face Transformers for image classification, the operator loads an image with cv2.imread(), resizes it with cv2.resize(), converts color space with cv2.cvtColor(), and then converts the NumPy array to a PyTorch tensor. For example: img = cv2.imread('cat.jpg'); img = cv2.resize(img, (224,224)); img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB); tensor = torch.from_numpy(img).permute(2,0,1).unsqueeze(0). This pipeline is common in both training and inference scripts. In LM Studio or Ollama, OpenCV is not directly used; instead, the runtime handles image preprocessing internally.