Inside an agent loop

An agent does more than return a reply. It can call a tool, read the result and decide what to do next. The application keeps that process going until the task is done or a limit is reached.

The model is one part of it. You also have to decide which tools it can use, how to handle errors and what counts as finished. Those choices become much easier to understand when you can see each step.

This example uses Peak Bean, the fictional coffee company from the RAG post. Jamie wants a refund for a leaking espresso machine. I have scripted the sequence so you can follow the request without waiting for a real model.

Inspect a request

At turn 3 of a conversation, does the model remember turn 1?

These are the main parts of the example request. Flip between turn 1 and turn 3 and watch what changes.

payload – tokens history included by the application

In this example, the application sends the conversation history with each request. That is where the earlier turns come from. Some APIs manage conversation state for you, but the model still needs the relevant context for its next response.

Follow one request

The transcript below shows the information passed around the loop. The agent looks up the order, checks the refund policy and tries to issue a refund. The first attempt is missing a required field.

The decision notes explain the example. They are not a recording of a model's private reasoning. Step through it and watch how the tool error changes the next request.

Step through the loop

Press Step or Run. Follow the transcript and the current position in the loop.

step 0 / 14

The policy allows a refund and a replacement. Jamie asked for money back. What does the model do next?

system220 tok

You are Peak Bean's support agent. Resolve the ticket using the tools provided. Cite policy for any money decision.

tools · 4 schemas380 tok

lookup_order search_policy issue_refund send_reply

user · jamie60 tok

My espresso machine arrived leaking. I want my money back. My order is #PB-2417.

660

ctx tokens

illustrative turns

$0.000

spend (toy)

The cost counter uses a made-up rate and counts input context only. It is here to show how requests grow.

The first refund call fails because it is missing idempotency_key. The next call adds it. The loop then checks the refund result and sends the reply. A requested action and a confirmed result are separate steps.

The conversation keeps growing

Each tool result adds more text to the conversation. If you keep all of it, later requests get longer. Eventually the history may exceed the context window or make the useful information harder to find.

Summarising older results can make room, but a summary can also lose details. Keep the goal, unresolved issues and facts the next step still needs.

Manage the context

If the conversation gets twice as long, the cost of the next turn…

Jamie's ticket is loaded below. Add more tickets, then tap a history block to shorten it. You can also use the compact button.

more tickets in this session 0

system + tools user model tool results compacted

– / 4,000 tokens window used – next-turn cost –

The next request includes the history you kept. Compacting a block reduces its size in this demo. In a real workflow, you also need to check what the summary left out. Input caching and provider pricing can change the cost.

The 4,000-token limit is chosen for this example. Actual limits depend on the model.

Decide when to retry and when to stop

The missing field in the example is easy to fix. Other failures need different responses. An invalid request needs corrected input. A temporary service failure may need a delayed retry. A permission error may need a person.

Retries also need limits. The simulator below lets you change the failure rate, retry count and total number of tool calls allowed for a ticket.

Try the limits

Tools fail 30% of the time, zero retries, step cap 8. Out of 30 tickets, how many resolve?

Choose the failure rate and limits, then run 30 tickets. The same settings give the same result.

tool failure rate 30%

retries per tool call 1

step cap 8

remove the step cap

Compare the results with zero retries and one retry. A retry can recover from a temporary failure. A step cap limits how much work one ticket consumes. These results come from a small simulation, so they are not a prediction of a real support system.

Before running an agent

a clear condition for successful completion
tool descriptions that explain when to use them
tool results and errors the loop can inspect
limits on steps, usage and repeated failures
retries that address the reason for failure

Problems to watch for

Repeating the same failed call

A retry should have a reason to work. In the transcript, the second refund request includes the missing field. Sending the first request again would not fix it.

Step through the retry

Keeping too much history

Old tool responses can take up most of a request. A shorter history helps only if it keeps the information needed to finish the task.

Try compacting the history

Choosing the wrong tool

Tool names, descriptions and input schemas affect which tool the model chooses. It needs to know when an order lookup is useful and when it should search the policy.

Read the available tools

What I would check before using this

The loop checks tool results before claiming that an action succeeded.
Retries have limits and a reason to succeed.
The context keeps the goal and the information needed for the next step.
The application can stop safely and leave a useful handoff when it cannot finish.

For a refund workflow, I would also check permissions and duplicate-request handling outside the model. A clear final reply is useful only if the refund itself has been confirmed.

About this example

The transcript, tool responses and decision notes are scripted. The ticket simulator uses a fixed random seed and simplified failure rules. Its token counts and costs are illustrative, not measured API usage. An uncapped failed ticket is displayed as a 40-step example rather than running forever.