Loop Engineering for AI Agents: Implementation Beyond Theory

A practical architecture for triggering work, executing tasks, verifying results, eenforcing limits, and improving from failures. Loop Engineering A common explanation of Loop Engineering is: Do not design only the next prompt. Design a loop that can continue running, collect feedback, and improve its results. The direction is correct, but it is still too abstract for engineers. Once you start implementing such a system, more concrete questions appear: Which layer is actually running the loop? If the agent says it is finished, is the task truly complete? When verification fails, how does that evidence enter the next attempt? How many times may the agent retry? What triggers the next task? What happens when the budget is exhausted? After one hundred executions, how do we know whether the system is improving? The key idea is: Loop Engineering is not about putting an agent inside while True. It is about designing the system around the agent so that it can discover work, execute tasks, verify results, enforce limits, preserve state, and decide what happens next. Without this surrounding system, the human becomes the outer loop. We manually provide a prompt, inspect the answer, identify mistakes, add missing context, and ask the model to try again. As soon as the human stops operating the process, the work stops. Loop Engineering turns these manual control decisions into explicit, observable, and enforceable system behavior. Four Loops Behind a Production Agent System A practical agent system can be modeled as four connected loops: Trigger ↓ Agent Loop ↓ Candidate Output ↓ Verification Loop ├── Pass → Deliver ├── Fail → Feedback → Retry └── Budget exhausted → Escalate Execution Trace ↓ Improvement Loop ↓ Evaluate and update the harness The four loops answer different questions: Agent Loop: How does the agent complete one task? Verification Loop: How does the system determine whether the result is acceptable? Event Loop: When should work begin? Improvement Loop: How does the system improve across many executions? These loops should not be collapsed into one large agent prompt. They operate at different levels and should have different permissions, budgets, and stopping conditions. 1. Agent Loop: How Is One Task Completed? The Agent Loop is the smallest execution unit: messages → model → tool call → tool result → messages → model The model requests a tool call. The surrounding harness executes the tool, appends the result to the conversation, and calls the model again. def run_agent(messages, tools): while True: response = call_model( messages=messages, tools=tools, ) messages.append(response) if response.has_tool_call: result = execute_tool(response.tool_call) messages.append( to_tool_result( tool_call=response.tool_call, result=result, ) ) continue return response.text This loop answers one question: How does the agent complete a single task? However, the loop usually stops when the model believes it has finished. The model may say: The code has been completed. The bug has been fixed. The analysis is correct. The documentation has been updated. But it may not have run the tests, satisfied the original requirements, or checked whether its changes introduced a regression. The Agent Loop produces a candidate result. It should not automatically decide whether that result is acceptable. 2. Verification Loop: Is the Task Actually Complete? The Verification Loop wraps around the Agent Loop. The worker first produces a candidate output. A separate verification process then checks that output against externally defined completion criteria. If the candidate passes, the result is delivered. If it fails, the failure evidence becomes structured feedback for the next attempt. def verified_run( task, worker, checker, rubric, max_attempts=2, ): feedback = None attempts = [] for attempt_number in range(1, max_attempts + 1): candidate = worker( task=task, feedback=feedback, ) verdict = checker( task=task, output=candidate, rubric=rubric, ) attempts.append({ "attempt": attempt_number, "passed": verdict.passed, "reason": verdict.reason, }) if verdict.passed: return { "ok": True, "output": candidate, "attempts": attempts, "action": "deliver", } feedback = { "previous_output": candidate, "failure_reason": verdict.reason, "evidence": verdict.evidence, "required_correction": verdict.required_correction, } return { "ok": False, "output": None, "attempts": attempts, "action": "escalate_to_human", } The code is simple, but it contains several important architectural decisions. Do Not Rely Only on the Worker’s Self-Assessment The worker may review its own output, but its self-assessment should not be the only completion signal. The same context that produced an incorrect answer often contains the assumptions and reasoning that caused the mistake. Asking the same context whether it is correct may reproduce the same blind spots. For higher-impact tasks, separate the generation and verification contexts. def check_with_fresh_context(task, output, rubric): messages = [{ "role": "user", "content": build_review_prompt( task=task, output=output, rubric=rubric, ), }] return run_checker(messages) The verifier does not always need to be another LLM. Depending on the task, verification may come from: Unit tests Integration tests Schema validation Static analysis Policy rules Database constraints A deterministic scoring function Another model with a fresh context A human reviewer For a coding task, tests may be more trustworthy than an LLM saying, “The implementation looks correct.” For a structured extraction task, JSON Schema validation may be more reliable than another natural-language review. Use the most objective verification mechanism available. Define the Rubric Outside the Loop The agent may try to satisfy the completion criteria, but it should not be allowed to rewrite them. A coding task might use the following rubric: 1. All existing tests must pass. 2. A regression test must be added. 3. Failing tests must not be removed or skipped. 4. Changes must remain inside the specified module. 5. The public API must remain backward compatible. If the agent can modify its own success criteria, it may improve its score by lowering the standard instead of improving the result. The rubric, permissions, and budget should therefore be controlled by an outer harness that the worker cannot modify. Convert Failure Into Structured Feedback A failed verification should not return only: Try again. That instruction contains almost no useful information. The next attempt should know: What the previous attempt produced Which requirement failed What evidence supports the failure What correction is required Which parts already passed and should not be repeated For example: Previous attempt: Added the API endpoint but did not add an authorization test. Verification result: FAIL Failed requirement: Unauthorized users must receive a 403 response. Evidence: The current test suite covers only authenticated requests. Required correction: Add a test for the unauthorized case and verify that the endpoint returns 403. Already satisfied: The successful request path works correctly. The second attempt is no longer generating from scratch. It is correcting a specific failure based on evidence. This is the difference between retrying and learning within a run. Enforce the Budget in Code Every loop needs a limit that the model cannot override. A budget may include: Maximum attempts Maximum tokens Maximum execution time Maximum tool calls Maximum cost Maximum parallel workers Maximum consecutive attempts without new evidence The simplest example is: for attempt_number in range(max_attempts): ... When max_attempts is two, a third attempt cannot occur. This is fundamentally different from writing “Do not try more than twice” inside the prompt. A prompt is guidance. Code is enforcement. When the budget is exhausted, the system should not continue hoping that another attempt will work. It should return an explicit state: ok = false action = escalate_to_human reason = retry_budget_exhausted A human can then inspect the execution trace and decide whether to: Provide missing information Change the task Approve a larger budget Modify the tool set Fix the harness Complete the task manually A loop without verification automates output. A loop without a budget automates the bill. 3. Event Loop: When Does Work Begin? The Agent Loop determines how work is executed. The Verification Loop determines whether it succeeded. Someone still needs to decide when the workflow should start. That is the responsibility of the Event Loop. A trigger may come from: A cron schedule A GitHub webhook A Slack or Discord message A newly created issue A failed CI run A queue A background process A monitoring alert A scheduled wake-up created by the agent def on_event(event): task = convert_event_to_task(event) result = verified_run( task=task, worker=worker, checker=checker, rubric=select_rubric(task), max_attempts=2, ) persist_result( task=task, result=result, ) deliver_report(result) The Event Loop answers: When should the agent begin working? However, receiving an event does not mean the agent should immediately receive unrestricted write access. Triggering work and granting autonomy are separate decisions. Three Levels of Agent Autonomy A useful way to deploy event-driven agents is to increase autonomy gradually. Level 1: Report The system reads data and generates a report. Examples: Summarize failed CI runs every morning Analyze recently created issues Identify outdated dependencies Report potentially risky code changes Detect records that require human review A human decides what action to take. This level is low risk and provides data about whether the agent’s analysis is reliable. Level 2: Assisted The system may prepare a change, but a human must approve it. Examples: Create a branch with a proposed fix Open a pull request Draft a response to an issue Prepare a dependency update Generate a migration plan Fill out a form without submitting it The agent can perform more work, but the final action remains gated. Level 3: Unattended The system may execute specific actions automatically, with humans auditing the results afterward. Examples: Fix known formatting problems Resolve narrow, well-tested regressions Perform low-risk repository maintenance Update generated documentation Deploy changes that pass a complete verification process A new workflow should not receive Level 3 permissions immediately. A safer progression is: Level 1 consistently produces accurate reports ↓ Level 2 consistently produces changes approved without modification ↓ Level 3 receives narrow, task-specific write permissions Autonomy is fundamentally a permissions decision, not a measurement of model intelligence. A more capable model may produce better recommendations, but that does not automatically justify broader access to production systems. 4. Improvement Loop: Is the System Getting Better? Once an agent system runs continuously, it begins accumulating execution traces. Useful trace data may include: Which tasks succeeded Which tasks failed How many attempts each task required Which tools failed most frequently Which requirements were commonly missed How many tokens each task consumed Which tasks required human intervention Which failures repeatedly appeared Which verifier decisions humans later overturned The Improvement Loop uses these traces to improve the surrounding system. Possible changes include: Updating the system prompt Improving tool descriptions Adding a missing tool Fixing a tool schema Changing context construction Adding a reusable skill Improving the verification rubric Adjusting the retry budget Changing model routing Narrowing permissions Adding a regression test The important point is that improvement should also be verified. A proposed harness change should be evaluated against a stable regression set before deployment. def evaluate_harness_change( current_harness, proposed_harness, regression_set, ): baseline = run_evaluation( harness=current_harness, dataset=regression_set, ) candidate = run_evaluation( harness=proposed_harness, dataset=regression_set, ) return compare_results( baseline=baseline, candidate=candidate, ) The system responsible for proposing improvements should not be allowed to modify: Its own permission limits Its own budget The verifier’s pass conditions The regression dataset The deployment approval process Otherwise, it may appear to improve by removing the constraints that expose its failures. For example, if an agent frequently fails a test, deleting that test would improve the measured pass rate while making the system worse. Improvement requires fixed external references. An End-to-End Example Consider a task: Add authorization handling to an API endpoint. Unauthorized users must receive a 403 response. The Event Loop receives a newly created issue and starts the workflow. Attempt 1 Worker: Implemented the endpoint and updated the service layer. Tool results: 18 tests passed. Worker conclusion: Task completed. The Agent Loop believes the task is finished. The Verification Loop checks the result against the fixed rubric. Checker result: FAIL Failed requirement: Unauthorized requests must return 403. Evidence: No test covers unauthenticated or unauthorized access. Required correction: Add an authorization test and verify the response status. Attempt 2 The worker receives the previous output and specific failure evidence. Worker: Added a test for unauthorized access. Updated the endpoint to return 403. Tool results: 19 tests passed. The verifier runs again. Checker result: PASS Evidence: - Existing tests pass. - The new regression test passes. - Unauthorized users receive 403. - Changes remain inside the requested module. The final execution trace might look like this: { "task_id": "issue-184", "status": "passed", "attempts": 2, "tool_calls": 7, "execution_seconds": 84, "human_escalation": false, "final_action": "open_pull_request" } This trace is useful beyond the current task. Across hundreds of runs, the system may discover that authorization tests are frequently omitted. That pattern can then inform the Improvement Loop. Possible improvements include: Adding authorization requirements to the coding skill Updating the rubric Adding a repository-specific test checklist Automatically detecting changed endpoints without permission tests The system does not improve merely because the model receives more prompts. It improves because failures become structured evidence. Common Failure Patterns 1. No Real Stopping Condition The agent repeatedly receives instructions to continue improving until the token or cost budget is exhausted. Better approach: Enforce attempt, time, token, tool-call, and cost limits in the harness. 2. The Worker Grades Its Own Work The same context produces the answer and decides whether it is correct. This can make verification little more than a request for self-confidence. Better approach: Use objective tests where possible. For subjective verification, use a fresh context, a fixed rubric, and periodic human audits. 3. The Checker Always Approves Adding a verifier does not automatically create meaningful verification. A vague rubric such as “Check whether the output is good” may cause the checker to approve nearly everything. Better approach: Require observable evidence. Instead of: Is the implementation correct? Use: Did all existing tests pass? Was a regression test added? Were any tests removed or skipped? Did the changes remain inside the specified module? 4. The Improvement Loop Can Remove Its Own Constraints An agent may improve its measured score by weakening the evaluation process. It could: Remove difficult tests Broaden permissions Increase its own budget Change the success threshold Exclude failed cases from evaluation Better approach: Keep critical gates inside an outer control system that the agent cannot modify. How to Start You do not need to build a fully autonomous agent system on the first day. Start with one narrow task that has clear completion criteria. For example: Check failed CI runs in a repository every day, identify likely causes, and generate recommended fixes. The first version should remain at Level 1: A cron job triggers the workflow once per day. The agent reads failed CI logs. It generates likely causes and recommendations. A verifier checks the output against a fixed rubric. The worker may retry no more than twice. Every attempt is written to an execution trace. The system produces a report whether it succeeds or fails. After the workflow becomes reliable, upgrade it to Level 2: Create an isolated branch or worktree. Generate a proposed fix. Run the tests. Verify the result. Open a pull request. Wait for human approval. Only after these pull requests can consistently be approved without modification should narrow, low-risk tasks be considered for Level 3 autonomy. Loop Engineering Is a Composition Discipline Loop Engineering does not introduce a completely new agent primitive. It composes existing mechanisms into a complete operating system around the model: The Agent Loop performs the work. Tools allow the model to interact with external systems. Verification determines whether the result is acceptable. Feedback carries failure evidence into the next attempt. Budgets constrain resource consumption. Permissions define which actions are allowed. Schedulers, queues, and channels trigger work. Memory and task records preserve state across runs. Observability records execution traces. Evaluation turns historical results into evidence for improvement. Human escalation handles cases outside the system’s reliable boundary. The engineering challenge is not simply keeping the agent alive. The important questions are: What triggers the work? How does the agent execute the task? Who determines whether the task is complete? What evidence is required for approval? How does failure enter the next attempt? How many resources may the workflow consume? When must the task be escalated? How is state preserved? How does the system learn from repeated failures? A production-oriented loop should contain: Trigger + Agent + Tools + Verification + Structured Feedback + Hard Budget + Persistent State + Escalation + Execution Trace + Evaluation The purpose of Loop Engineering is not to remove humans. It moves humans from manually operating every iteration to defining the boundaries, evidence, permissions, and improvement process of the system as a whole. Resources The runnable implementation of these concepts is available in Awesome Agent Architecture , an open-source guide that builds an agent harness progressively from the smallest Agent Loop to tools, permissions, hooks, planning, subagents, skills, context management, memory, scheduling, multi-agent coordination, evaluation, and Loop Engineering. The repository includes runnable Python examples that show how each mechanism changes the behavior of the system. Explore Awesome Agent Architecture on GitHub Loop Engineering for AI Agents: Implementation Beyond Theory was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.