Deep Learning (DL)

Deep learning (DL) is a subset of machine learning that uses multi-layer neural networks to learn patterns from data. In local AI, operators encounter DL as the underlying technique that powers models like Llama, Mistral, or Stable Diffusion. A DL model consists of layers of interconnected nodes (neurons) that transform input data through nonlinear functions. Training adjusts the weights between these nodes using backpropagation and gradient descent. For operators, the practical relevance is that larger models (more layers, more parameters) require more VRAM and compute to run inference, and quantization reduces the precision of these weights to fit within hardware limits.

Deep learning differs from traditional machine learning by automatically learning hierarchical feature representations. In image recognition, early layers detect edges, later layers detect shapes, and final layers recognize objects. For language models, layers learn syntax, semantics, and context. The depth (number of layers) is what makes it 'deep' — typically more than three layers. Popular architectures include transformers (used in LLMs), convolutional neural networks (CNNs) for images, and recurrent neural networks (RNNs) for sequences. Operators running local models benefit from understanding that deeper models generally perform better but require exponentially more compute. Techniques like pruning, distillation, and quantization reduce model size while preserving accuracy, making deep learning feasible on consumer hardware.

A 7B-parameter Llama 3 model is a deep learning model with around 32 transformer layers. Running it at FP16 precision would require 14 GB of VRAM, which fits on an RTX 3090 (24 GB) but not on an RTX 3060 (12 GB). By applying 4-bit quantization, the model size drops to ~5 GB, allowing it to run on the RTX 3060 with acceptable speed (20 tok/s).

When using LM Studio to load a model, the software reports the model architecture (e.g., 'llama'), the number of parameters (e.g., 7B), and the quantization level (e.g., Q4_K_M). These are all properties of the deep learning model. The user selects a model file, and the runtime loads the neural network weights into VRAM, then performs inference by passing tokens through the layers.