Recurrent Neural Network (RNN)

A Recurrent Neural Network (RNN) is a neural network architecture designed for sequential data, where each output depends on the previous hidden state. Unlike feedforward networks, RNNs maintain a hidden state that acts as a memory of past inputs. In local AI, RNNs are rarely used for text generation today—transformers dominate—but they appear in specialized tasks like time-series forecasting or audio processing. Operators encounter RNNs in legacy models or when fine-tuning small sequence models on edge devices, where their sequential nature limits parallelization and makes them slower per token than transformers.

RNNs process sequences step-by-step: at each time step t, they take input x_t and the previous hidden state h_{t-1} to produce output y_t and new hidden state h_t. This recurrence allows them to handle variable-length sequences. However, RNNs suffer from vanishing/exploding gradients, making it hard to learn long-range dependencies. Variants like LSTMs and GRUs introduced gating mechanisms to mitigate this. In practice, for language modeling, transformers have largely replaced RNNs because they process all tokens in parallel via attention, enabling faster training and inference. RNNs still appear in some real-time applications (e.g., speech recognition on microcontrollers) where model size and latency constraints favor their simpler structure. For local AI operators, RNNs are relevant when working with older codebases or deploying tiny models on low-resource hardware.

An operator running a small LSTM-based keyword spotter on a Raspberry Pi 4 might see ~10-20 ms inference per 1-second audio chunk, fitting in under 100 MB RAM. In contrast, a transformer model of similar accuracy would likely exceed the Pi's 4 GB RAM or run at <1x real-time. The RNN's sequential processing keeps memory low but limits throughput.

When loading an RNN model in Hugging Face Transformers, operators might use AutoModel.from_pretrained('some-lstm-model') and see a warning about slow inference. In llama.cpp, RNN support is minimal—most GGUF models are transformers. In MLX, RNN layers exist for custom models, but typical workflows avoid them. If an operator encounters an RNN, it's often in a legacy fine-tuning script using PyTorch's nn.LSTM.