How to implement human-in-the-loop for AI agents
Agent workflow defined, human review interface
What this does
Implementing human-in-the-loop for AI agents adds approval checkpoints where the agent pauses execution and requests human judgment before performing high-stakes actions. These checkpoints can trigger for actions like deleting files, sending emails, executing financial transactions, or deploying code. The human reviewer sees the proposed action with context and can approve, reject, or modify it before the agent proceeds. This pattern reduces risk while preserving automation for low-risk steps.
Steps
Identify the agent actions that require human approval. Common candidates: file deletion, API calls that modify data, code execution, email sending, and configuration changes. Tag these actions in the agent's tool definitions with requires_approval: true. Implement the approval gate in the agent's execution loop. After the agent selects an action, check the tag: if action.requires_approval: approval = await request_approval(action). The request_approval function serializes the proposed action with all context—the user's original request, the reasoning chain, the action details—and sends it to the review channel. For synchronous review (CLI), present a formatted summary and use input("Approve? [y/n/e(dit)]: "). For asynchronous review, publish to a Redis channel and await a response on a unique correlation ID. Store the pending approval in a database table with fields: id, session_id, action_json, status (pending/approved/rejected/modified), created_at, reviewed_at, reviewer, modified_action_json. Set a timeout (default 30 minutes) after which pending approvals auto-reject with a logged reason. When the response arrives, resume the agent with the approved action, the modified action, or a rejection message instructing the agent to find an alternative path.
Record the local run evidence. Save the exact command, runtime or package version, model name if applicable, and observed output so the result can be reproduced later.
Confirm the local starting state. Print the active binary, package version, model name, or configuration path before changing the workflow.
Run the smallest complete path. Execute the minimum command or script that proves the guide works end to end on the local machine.
Compare against expected output. Check the final line, status code, generated artifact, or model response against the verification section before expanding the setup.
Record the local run evidence. Save the exact command, runtime or package version, model name if applicable, and observed output so the result can be reproduced later.
Verification
Run the agent with a task that triggers an approval checkpoint, e.g., "Delete all log files older than 7 days." Verify the agent pauses and displays the proposed deletion with a list of affected files. Approve the action and confirm files are deleted. Run the same task, reject the action, and verify no files are deleted and the agent reports the rejection. Test the timeout: leave a pending approval for 31 minutes and confirm it auto-rejects. Verify asynchronous approval by sending a task, checking the pending-approvals database, and approving via API—the agent should resume within 5 seconds.
Common failures
Agent hangs indefinitely waiting for approval: Ensure the timeout mechanism is working and that the agent has a fallback path when approval times out. Review channel not receiving approvals: Check the message queue connection and verify the correlation ID is correctly passed in the response. Approval decision lost after agent restart: Persist pending approvals to a database, not in-memory, so restarts can recover them. Reviewer overwhelmed by too many approval requests: Adjust the requires_approval tag to only flag truly risky actions; add a confidence threshold where the agent self-approves actions below the threshold. Race condition with concurrent approval and timeout: Use database-level locks or atomic state transitions (pending → approved/rejected) to prevent double-processing.
- Version mismatch - The installed package or runtime differs from the command shown; check the version first and rerun the smallest verification command.
- Local environment drift - Another service, virtual environment, model, or path is being used; print the active binary path and configuration before changing the guide steps.
Related guides
- build-multi-agent-supervisor-workflow
- implement-guardrails-ai-agents
- setup-agent-observability-opentelemetry