COURSE · OPS · A006
Custom Agent Frameworks
Learn custom agent frameworks through RunLocalAI's practical lens: agents, framework, runtime and memory, hardware fit, runtime settings, verification habits and local-vs-cloud tradeoffs.
PREREQUISITES
- I002
- B016
Why this course matters
Custom Agent Frameworks is for operators making local AI reliable, measurable and cheaper to run. It connects agents, framework, runtime, memory and planning to the questions RunLocalAI wants every reader to answer before they install, upgrade or scale a model: will it run, what will it cost in memory, what setting changes the result, and how do you verify the answer instead of trusting a demo?
What you will be able to do
By the end, you should be able to explain the main tradeoffs in plain language, choose a safe next experiment, and use the chapter exercises as a repeatable operator checklist. The course favors local evidence, hardware fit, context limits, latency and failure modes over generic AI vocabulary.
How to use this course
Start at chapter one if the topic is new. If you already have a working stack, scan for chapters such as Why Custom Frameworks?, Agent Runtime Design, Agent Loop and Tool Registry and use those lessons as a quality-control pass before changing a workstation, team workflow or production-like local deployment.
CHAPTERS
- 01Why Custom Frameworks?Custom frameworks trade framework support for architectural control. You gain the ability to inspect, profile, and modify every component of your agent's runtime behavior.15 min
- 02Agent Runtime DesignThe runtime loop is a controlled iteration pattern. Every component has a single responsibility, and the orchestrator coordinates their interaction through a defined message protocol.15 min
- 03Agent LoopThe loop is a state machine. Each iteration transitions between phases, and your job is to ensure every transition is well-defined—including the error transitions.15 min
- 04Tool RegistryThe registry is a simple mapping with added schema generation. Its power is in standardization: every tool goes through the same interface, making debugging and testing consistent.15 min
- 05Function Calling IntegrationFunction calling integration is a protocol. Treat it like one: validate inputs, handle errors explicitly, and don't assume the model always produces perfectly-formed output.20 min
- 06Working MemoryWorking memory is ephemeral and fast. Design for fast writes and efficient reads. The truncation strategy is the most critical implementation detail.15 min
- 07Episodic MemoryEpisodic memory turns individual interactions into searchable experience. The storage abstraction lets you swap implementations (in-memory → PostgreSQL → vector database) without changing the agent code.20 min
- 08Semantic MemorySemantic memory is your knowledge base. The embedding-based retrieval is effective but requires careful attention to embedding quality, storage scaling, and retrieval relevance.15 min
- 09Memory PersistencePersistence transforms memory from volatile to durable. The checkpoint pattern ensures you lose at most N tool calls on crash, not an entire session.20 min
- 10Planning SystemsPlanning is explicit commitment to a sequence of actions. Deliberate planning trades flexibility for reliability—use it when tasks are complex enough that zero-shot execution fails.15 min
- 11Task PlannerTask planning is decomposition with validation. Every step must be mappable to an available tool with valid arguments. Confirmation handling separates autonomous execution from actions requiring human oversight.20 min
- 12Re-planningRe-planning is recovery. The trigger determines when to adapt; the replanner uses failure context to generate a new approach. Capping replans prevents infinite loops and enables graceful escalation.20 min
- 13Multi-Agent ProtocolsMulti-agent protocols fail silently until they don't. Explicit message contracts and state machines catch 90% of integration bugs before they hit production.15 min
- 14Agent CommunicationMessage broker architecture prevents coupling, but you pay in debugging complexity. Instrument everything early—you cannot debug what you cannot see.20 min
- 15Agent DiscoveryAgent discovery turns static configurations into dynamic ones, but introduces discovery delay and potential for stale data. Always validate that discovered agents are still responsive before dispatching critical tasks.15 min
- 16ObservabilityObservability is not debugging. Debugging happens after something breaks. Observability tells you something is degrading before users notice. Instrument before you need it.20 min
- 17Error HandlingThe worst failure mode is silent failure. Every error should either be resolved or moved to a dead letter queue with alerting. Errors that "just get retried forever" waste resources and hide problems.20 min
- 18Testing FrameworkTests that always pass are worse than no tests—they provide false confidence. Write tests that can fail, and verify they fail when they should.20 min
- 19BenchmarkingBenchmark results without context are meaningless. A 10ms latency could be excellent or unacceptable depending on your use case. Define SLOs before benchmarking, not after.20 min
- 20DocumentationDocumentation is a liability, not an asset. Every sentence you write must be maintained. Write less, but write things that can't get out of date—like code with type annotations, or decision records that explain intent.20 min
- 21Package DistributionPackage distribution enables collaboration but introduces dependency management challenges. Keep agent dependencies minimal and well-documented. Large dependency trees guarantee conflicts.20 min
- 22Plugin ArchitecturePlugins are untrusted code with your privileges. Sandbox them from day one. Resource limits, hook auditing, and version pinning prevent plugins from becoming attack vectors.20 min
- 23Custom FrameworkCustom frameworks succeed when they embody clear principles. Every "feature" you add must justify itself against the cost of maintenance. Start with the minimum viable abstraction layer.20 min
- 24Agent Framework ProjectThis project demonstrates that agent frameworks don't have to be complex. The value comes from clear communication patterns, observability, and error handling—not from elaborate abstractions.25 min