Large Language Model (LLM)

A Large Language Model is a neural network with billions of parameters trained on massive text corpora to predict the next token. The "large" generally means at least 1B parameters; modern LLMs range from 1B (edge models) to 1.8T (proprietary frontier models).

Modern LLMs are decoder-only transformers, autoregressive (generating one token at a time), and trained in two stages: pre-training (trillions of tokens, unsupervised next-token prediction) and post-training (RLHF, DPO, or similar — teaching the model to follow instructions and avoid harmful outputs).

For local AI: open-weight LLMs you can run yourself include Llama, Qwen, Mistral, Phi, Gemma, and DeepSeek families. The 7B-32B range fits on consumer hardware; 70B-class needs 24GB+ VRAM or unified-memory Apple Silicon; 100B+ MoE models need workstation-tier setups. See the directory for what runs where.

An operator scoping a local deployment for a customer-support triage bot starts by asking what actually fits the budget hardware: a single RTX 4090 with 24GB VRAM. That rules out 70B-class dense LLMs at any usable quantization, but comfortably fits a 14B or 32B model in 4-bit, or an 8B model with room to spare for a large KV cache. The decision isn't just parameter count — a well-tuned Qwen 2.5 14B often beats a poorly-quantized 70B for structured tasks like ticket classification, because aggressive quantization on a large model can hurt more than simply choosing a smaller model at higher precision. Check /will-it-run before committing to a checkpoint size, since VRAM headroom for context and batching matters as much as the base weight footprint.