Graph Neural Network (GNN)

A Graph Neural Network (GNN) is a neural network architecture designed to process data structured as graphs—nodes connected by edges. Unlike standard neural networks that assume fixed-size inputs (e.g., images or sequences), GNNs operate on arbitrary graph topologies by aggregating information from a node's neighbors to update its representation. Operators encounter GNNs when working with relational data like social networks, molecular structures, or recommendation systems. In local AI, GNNs are typically run via frameworks like PyTorch Geometric or DGL, and inference latency scales with graph size and connectivity, making VRAM constraints relevant for large graphs.

GNNs generalize convolution to non-Euclidean domains. The core operation is message passing: each node aggregates features from its neighbors (e.g., via mean, sum, or attention) and updates its own embedding. Common variants include Graph Convolutional Networks (GCN), Graph Attention Networks (GAT), and GraphSAGE. Training GNNs often involves mini-batching with neighbor sampling to fit graphs into GPU memory. For operators, GNN inference on large graphs (e.g., millions of nodes) requires careful memory management—full-batch inference may exceed VRAM, so sampling or partitioning is used. In local AI, GNNs are less common than LLMs but appear in specialized tools like molecule property prediction or node classification tasks.

An operator running node classification on a citation network with 100K nodes using PyTorch Geometric on an RTX 3090 (24 GB VRAM) would need to use neighbor sampling (e.g., NeighborLoader) to fit a mini-batch of 1024 nodes with 10-hop neighbors into memory. Without sampling, the full graph's adjacency matrix (~10 GB for dense storage) would exceed VRAM, causing out-of-memory errors.

In a typical workflow, an operator loads a graph dataset (e.g., Cora or PubMed) using PyTorch Geometric's Planetoid class, defines a GCN model with torch_geometric.nn.GCNConv, and trains using mini-batches. During inference, they call model.eval() and iterate over test nodes with DataLoader. If the graph is large, they switch to NeighborLoader to sample subgraphs, monitoring VRAM usage with nvidia-smi to avoid OOM.