Coding Agent

A coding agent is a language model configured to write, debug, or refactor code autonomously or semi-autonomously. It typically runs in a loop: the model receives a task (e.g., 'add a login endpoint'), generates code, executes it in a sandbox, checks for errors or test failures, and iterates until the task passes. Operators encounter coding agents as tools like GitHub Copilot Chat, Aider, or Open Interpreter. They rely on the model's context window to hold the full codebase and conversation history, so VRAM and context length directly limit how large a project the agent can handle.

Coding agents differ from simple code completion by maintaining state across multiple turns. A typical workflow: the agent receives a prompt describing a feature or bug, generates a diff or full file, runs a build or test command, captures stdout/stderr, and feeds that output back into the model for the next iteration. The agent's effectiveness depends on the model's instruction-following ability, code syntax accuracy, and context size. Smaller models (7B-13B) can handle simple scripts but struggle with multi-file projects; larger models (70B+ or Mixture-of-Experts) perform better but require more VRAM. Quantization (Q4 or Q5) is common to fit larger models on consumer GPUs. Tools like Aider use a 'map' of the codebase to reduce context usage, while Open Interpreter executes arbitrary shell commands. The operator must monitor token usage and iteration count to avoid runaway costs or time.

An operator runs Aider with Llama 3.1 70B Q4 (~40 GB VRAM) on an RTX 4090 (24 GB). Since the model doesn't fit entirely in VRAM, Aider offloads layers to system RAM, dropping tokens/sec from ~20 to ~3. The agent is asked to 'add a /health endpoint to the Flask app.' It generates the route, runs pytest, sees a failure because the test expects JSON, and iterates twice to fix the response format. Total time: ~90 seconds for a 3-iteration fix.

In LM Studio, an operator loads a Qwen2.5-Coder-7B-Instruct GGUF and enables the 'Agent Mode' plugin. They paste a bug report into the chat: 'The sort function throws a KeyError on empty list.' The agent writes a fix, clicks 'Run Code' to execute the script in a sandbox, sees the error, and revises the code. The operator watches the token counter climb and the VRAM usage hover at ~6 GB. If the context fills up (e.g., 8K tokens), the agent starts truncating earlier conversation, potentially losing the bug context.