DBSCAN

DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is an unsupervised clustering algorithm that groups data points based on density. It defines clusters as dense regions separated by sparse regions, and can identify outliers as noise. Operators encounter DBSCAN when preprocessing text embeddings (e.g., from sentence-transformers) to find clusters of similar documents or to detect anomalous entries without specifying the number of clusters in advance.

DBSCAN works by selecting a point and expanding its cluster if it has at least minPts neighbors within a radius eps. Points with fewer neighbors become noise. It handles arbitrary-shaped clusters and does not require the number of clusters as input. Key parameters: eps (maximum distance between two points to be considered neighbors) and minPts (minimum points to form a dense region). DBSCAN is sensitive to these parameters; too small eps yields many noise points, too large merges clusters. Variants like HDBSCAN use hierarchical density estimation to avoid manual eps tuning. In local AI workflows, DBSCAN is often applied to embedding vectors (e.g., 768-d from BERT) using cosine distance, with eps set between 0.3 and 0.6.

An operator has 10,000 customer support tickets encoded as 768-dimensional embeddings from all-MiniLM-L6-v2. Running DBSCAN with eps=0.5 and minPts=5 might yield 50 clusters of similar issues and 200 noise points (unique tickets). On a laptop CPU, this takes ~2 seconds for 10k points using scikit-learn's implementation. The noise points can be flagged for manual review.

In a Python script using sklearn.cluster.DBSCAN, the operator loads embeddings from a .npy file, then runs db = DBSCAN(eps=0.5, min_samples=5, metric='cosine').fit(embeddings). The resulting db.labels_ array assigns cluster IDs (-1 for noise). This is often followed by visualizing clusters with UMAP or t-SNE, or by extracting representative samples per cluster for summarization.