How to Build an AI Agent That Does One Job Well

Here is how to build an AI agent, in the order that keeps it small. Pick one job with an end you can check by looking at data. Give the model the fewest tools that job needs, and write down which tools you're deliberately leaving out. Write the loop yourself once, with a turn cap and a token cap, so you know exactly what "keeps going" means. Then write a second script that grades the run from the log and the data, never from the model's own summary. Everything below is one Python file of about 150 lines against the Anthropic SDK 1.5.0, plus a 30-line checker, and the job is labeling open issues in a JSON file.

The pages ranking for this query either stay conceptual, like OpenAI's guide to building agents, or build a loop with no stop set at all. OpenAI's definition is the one I'd keep. An agent is a system that independently accomplishes a task on your behalf, and a chatbot that answers one message is not one. What the definition leaves out is the part that costs you money at 3am, which is when the loop stops.

Pick A Job With A Checkable End

The job here is triage. A project has open issues with no label, and each one needs exactly one of bug, feature, or question. That's it. The agent does not comment, close, assign, or edit anything.

I picked this because the end state is a file you can diff. Either every issue has one allowed label or it doesn't, and a script can decide which without reading a single word of what the model said. OpenAI's guide recommends getting the most out of a single agent before adding more, and notes that some deployments run more than 15 well-defined tools while others struggle with fewer than 10 overlapping ones. Three non-overlapping tools is where I'd start every time.

Write the acceptance check before the agent. Mine is four lines long in prose. Every issue has a label from the allowed set. The number of successful label writes equals the number of issues. No issue was labeled without being read first. No tool was called that isn't in the list.

The Tool Ledger

Before any code, decide what each tool may touch. The autonomous AI agent post argued that autonomy is a permission ledger rather than a personality, and OpenAI's guide says the same thing in different words, recommending that you rate every tool low, medium, or high on read-only versus write access, reversibility, permissions, and financial impact. So here is the ledger for this agent, including the rows that are missing on purpose.

The last two rows are the design. A model can be talked into calling a tool it has, by a prompt injection in an issue body or by its own enthusiasm. It cannot be talked into calling a tool that was never declared. That's a harder boundary than any instruction in the system prompt, and it costs nothing.

The Data And The Tools

Create a folder with an issues.json of a few open issues. Each entry has id, title, body, and label, with every label set to null. Five is plenty. I'd write one whose title reads like a bug and whose body asks for a feature, so the read step has a reason to exist.

Now agent.py. The top of the file is the contract and the tools:

import json
import sys
from pathlib import Path

import anthropic

ISSUES = Path(__file__).parent / "issues.json"
RUN_LOG = Path(__file__).parent / "run.jsonl"
LABELS = ("bug", "feature", "question")
MODEL = "claude-opus-5"
MAX_TURNS = 12
MAX_TOKENS_PER_RUN = 60_000

SYSTEM = (
    "You label open issues in a software project. Your only job is to give "
    "every unlabeled issue exactly one label: bug, feature, or question. "
    "Read an issue before labeling it. Do not comment, close, or assign. "
    "When every issue has a label, say so and stop."
)

TOOLS = [
    {
        "name": "list_issues",
        "description": (
            "List every open issue that has no label yet. Returns the id and title "
            "of each one and nothing else. Call this first, and call it again if "
            "you are unsure what is left. Takes no arguments."
        ),
        "input_schema": {"type": "object", "properties": {}, "required": []},
    },
    {
        "name": "read_issue",
        "description": (
            "Read the full title and body of one issue by id. Use it before "
            "labeling, because titles alone are often misleading. Returns an "
            "error if the id does not exist."
        ),
        "input_schema": {
            "type": "object",
            "properties": {"id": {"type": "integer", "description": "Issue id from list_issues"}},
            "required": ["id"],
        },
    },
    {
        "name": "set_label",
        "description": (
            "Set the label on one issue that has no label yet. Allowed labels are "
            "bug, feature, and question. It fails if the issue is already labeled "
            "or the id does not exist, and it cannot change a label once set."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "id": {"type": "integer"},
                "label": {"type": "string", "enum": list(LABELS)},
            },
            "required": ["id", "label"],
        },
    },
]

The descriptions are longer than they look like they need to be. Anthropic's tool definition page calls the description "by far the most important factor in tool performance" and asks for at least three or four sentences each, covering what the tool does, when to use it, and what it won't return. The enum on label is the cheap half of the guard. The expensive half is in the handler:

def load():
    return json.loads(ISSUES.read_text())


def save(issues):
    ISSUES.write_text(json.dumps(issues, indent=2) + "\n")


def run_tool(name, args):
    issues = load()
    if name == "list_issues":
        return [{"id": i["id"], "title": i["title"]} for i in issues if i["label"] is None]
    if name == "read_issue":
        for i in issues:
            if i["id"] == args["id"]:
                return {"id": i["id"], "title": i["title"], "body": i["body"]}
        raise ValueError(f"No issue with id {args['id']}. Call list_issues for valid ids.")
    if name == "set_label":
        if args["label"] not in LABELS:
            raise ValueError(f"Label must be one of {LABELS}.")
        for i in issues:
            if i["id"] == args["id"]:
                if i["label"] is not None:
                    raise ValueError(f"Issue {i['id']} already has label {i['label']}.")
                i["label"] = args["label"]
                save(issues)
                return {"id": i["id"], "label": i["label"]}
        raise ValueError(f"No issue with id {args['id']}.")
    raise ValueError(f"Unknown tool {name}.")


def log(event):
    with RUN_LOG.open("a") as f:
        f.write(json.dumps(event) + "\n")

Every error message says what went wrong and what to do next, because the handling page is explicit that a bare "failed" gives the model nothing to recover with. The same page says Claude will retry an invalid call two or three times with corrections before giving up, which is why a good error string is worth more than a clever prompt.

The Loop With Its Stop Set

This is the part every tutorial on the results page hand-waves. Anthropic's own tool-using agent tutorial uses while response.stop_reason == "tool_use", which is correct and also unbounded. Mine bounds it three ways.

def run(client, task):
    messages = [{"role": "user", "content": task}]
    tokens_used = 0
    for turn in range(1, MAX_TURNS + 1):
        response = client.messages.create(
            model=MODEL,
            max_tokens=4096,
            system=SYSTEM,
            tools=TOOLS,
            messages=messages,
        )
        tokens_used += response.usage.input_tokens + response.usage.output_tokens
        log({"turn": turn, "stop_reason": response.stop_reason,
             "input_tokens": response.usage.input_tokens,
             "output_tokens": response.usage.output_tokens})

        if response.stop_reason == "end_turn":
            return "done", response, tokens_used
        if response.stop_reason != "tool_use":
            return f"stopped: {response.stop_reason}", response, tokens_used
        if tokens_used > MAX_TOKENS_PER_RUN:
            return "stopped: token cap", response, tokens_used

        results = []
        for block in response.content:
            if block.type != "tool_use":
                continue
            try:
                out = run_tool(block.name, block.input)
                results.append({"type": "tool_result", "tool_use_id": block.id,
                                "content": json.dumps(out)})
                log({"turn": turn, "tool": block.name, "input": block.input, "ok": True})
            except Exception as exc:
                results.append({"type": "tool_result", "tool_use_id": block.id,
                                "content": str(exc), "is_error": True})
                log({"turn": turn, "tool": block.name, "input": block.input, "ok": False,
                     "error": str(exc)})
        messages.append({"role": "assistant", "content": response.content})
        messages.append({"role": "user", "content": results})
    return "stopped: turn cap", response, tokens_used


if __name__ == "__main__":
    RUN_LOG.unlink(missing_ok=True)
    status, final, tokens = run(anthropic.Anthropic(), "Label every open issue.")
    text = " ".join(b.text for b in final.content if b.type == "text")
    print(status, tokens, "tokens")
    print(text)
    sys.exit(0 if status == "done" else 1)

Four details carry the loop. The for instead of while is the turn cap, and it counts model calls, which is the thing you pay for. Any stop reason that isn't end_turn or tool_use ends the run with its name in the status, so max_tokens, refusal, and pause_turn never get fed back into another call by accident. A response can hold several tool_use blocks at once, and every result goes back in one user message with the tool_result blocks first, which the docs require. And the assistant's content goes back untouched, thinking blocks included.

The token cap is checked after the call that crossed it, not before, because you can't know the size of a response until it arrives. It's a ceiling on damage, not a budget. At Opus 5 rates, 60,000 tokens is $0.30 if they were all input and $1.50 if they were all output, and a real run sits between.

Prove It Without Asking The Model

The agent's closing message will say something like "all five issues are labeled." Don't grade on that. Grade on issues.json and run.jsonl:

import json
import sys
from pathlib import Path

HERE = Path(__file__).parent
issues = json.loads((HERE / "issues.json").read_text())
events = [json.loads(l) for l in (HERE / "run.jsonl").read_text().splitlines()]

problems = []
for i in issues:
    if i["label"] not in ("bug", "feature", "question"):
        problems.append(f"issue {i['id']} has label {i['label']!r}")

tool_calls = [e for e in events if "tool" in e]
writes = [e for e in tool_calls if e["tool"] == "set_label" and e["ok"]]
if len(writes) != len(issues):
    problems.append(f"{len(writes)} successful set_label calls for {len(issues)} issues")
unknown = {e["tool"] for e in tool_calls} - {"list_issues", "read_issue", "set_label"}
if unknown:
    problems.append(f"unknown tools called: {sorted(unknown)}")
read_before_write = {e["input"]["id"] for e in tool_calls if e["tool"] == "read_issue" and e["ok"]}
for w in writes:
    if w["input"]["id"] not in read_before_write:
        problems.append(f"issue {w['input']['id']} labeled without being read")

turns = max((e["turn"] for e in events), default=0)
print(f"turns={turns} tool_calls={len(tool_calls)} errors={sum(1 for e in tool_calls if not e['ok'])}")
for p in problems:
    print("FAIL", p)
sys.exit(1 if problems else 0)

The "labeled without being read" row is the one I care about most. It catches the agent that skims titles and guesses, a run that looks perfect in the closing message and is wrong on the issue whose title says bug and whose body asks for a feature. It's the same idea as checking that a new test would fail on the old code, applied to the agent's process instead of its output.

Run It, Then Read The Log

uv venv && source .venv/bin/activate
uv pip install "anthropic==1.5.0"
export ANTHROPIC_API_KEY=...
python agent.py && python check.py

An honest note on what I verified. I ran this loop against a scripted fake client that returns real SDK Message objects in six scenarios, including a parallel happy path, a bad id, a double write, a fifty-turn runaway, an oversized response, and each non tool_use stop reason. All six behaved as described, and check.py failed the run that labeled without reading. What I did not do is watch a live model choose its own path through these tools, because there was no credential where I wrote this. Your first live run is where you find out whether it reads before it labels. Watch that one.

What A Run Costs

Tool use is priced as ordinary tokens, with one addition. Anthropic's tool use overview says a tool-use system prompt is added automatically, 286 tokens on Claude Opus 5 with the default tool_choice, and that the tools parameter, the tool_use blocks and the tool_result blocks all count as input on every call. Since the whole conversation is resent each turn, the input grows.

My estimate for the fixed prefix is about 800 tokens. That's the three tool definitions at roughly 430 tokens (the JSON is 1,065 characters, and the models page puts the current tokenizer at about 2.5 characters per token), the system prompt at about 100, the 286-token tool prompt, and the task. From there, two run shapes bracket the cost.

Prices are $5 and $25 per million tokens for Opus 5, $2 and $10 for Sonnet 5, and $1 and $5 for Haiku 4.5, from the models page as fetched on 2026-09-15. Thinking tokens bill as output and aren't in these figures, and Opus 5 thinks by default. Your run.jsonl has the real numbers per turn, which is the reason the loop logs usage at all. Note that the sequential shape lands exactly on my turn cap of 12. If a live run does that, the cap is too tight for this job, and that's a fact you learn from the log rather than from a stack trace.

Where The Loop Breaks

That last row is the mistake I'd expect a first build to make, because answering each tool one at a time feels natural. The docs are strict that a turn's results go back together, first in the content array.

The Shorter Path Once You Understand The Loop

Anthropic's SDK ships a Tool Runner, in beta on the Python side, where you decorate each function with @beta_tool, write a docstring, and call client.beta.messages.tool_runner(...).until_done(). It writes the loop above for you. OpenAI's Agents SDK does the same with Agent and Runner.run, and its loop has a documented max_turns that raises MaxTurnsExceeded when crossed, with a default of 10 in the current source. Ollama's tool calling works the same way for local models, with results sent back as role: "tool" messages.

I'd still write the manual loop once. The turn cap, the token cap, and the log are the parts you'll want to adjust per job, and you can only adjust what you understand. When the tools outgrow one file, the MCP server post covers moving them behind a protocol any host can call.

Is It Free To Build An AI Agent?

The code is. The SDKs above are open source, and the loop is 40 lines you now have. The model calls aren't, unless you run a local model through Ollama, where the cost is your own hardware. On the hosted side the table above is the honest answer for this job, a few cents per run on any current Claude model, and the log tells you the exact figure after the first one.

Can You Build An AI Agent With ChatGPT?

In two senses, yes. You can paste this article's loop into ChatGPT and have it adapt the tools to your data, which is using it as a coding assistant. Or you can build on OpenAI's Agents SDK with an API key, which is the same loop with a different provider. ChatGPT the chat app is still not an agent on its own. The agent OpenAI ships inside it is Codex, which their pricing page lists on every plan from Free upward, with Go at $8 a month and Plus at $20, and Codex is a coding agent with its own sandbox rather than a place to run your triage loop.

Is It Hard To Build An AI Agent?

The loop is easy. I think most people who say it's hard are describing the part after the loop, which is deciding what the agent may not do, writing the check that proves it did the job, and reading the log when it didn't. The hard version is the one where you skip those and try to debug a twelve-turn transcript by reading it.

Are AI Agents Making Money?

Some are, for the people running them, and I can't give you a number that isn't someone's marketing. My judgment is that the agents earning their keep are the boring ones, a triage loop or a nightly report, and they earn it by replacing a recurring twenty minutes. For your own case, price the run with the table above and compare it to what the job costs a person.

What Are The 5 Types Of AI Agents?

That taxonomy, simple reflex through learning agents, describes how an agent decides, and this article's agent is a plain goal-based one with tools. The scheduled piece on types of AI agent takes the list apart with working examples, so I'll leave the definitions there.

Grow It By One Tool

When the labeling agent runs clean three times, add exactly one tool and rerun check.py. If the new tool is a write, put it in the ledger with a rating first. If the job stops fitting in one loop, the multi agent AI post covers when a second agent actually helps and when it just doubles the transcript. Most jobs never get there. The one-file version is the shape every larger agent still has underneath.