How to Build a Real AI Agent with OpenAI's Agents API: A Hands-On Blueprint
Most people meet AI agents through a demo: someone types a goal, the screen scrolls for a while, and a finished result appears. What the demo hides is everything underneath. Who keeps the task alive when it takes an hour instead of a minute? Where do the files live? What happens when a step fails halfway through? How much of the work still needs a human?
In September 2026, OpenAI released its Agents API in public beta. In plain terms, it lets an application start a long-running cloud agent without the developer having to build the whole agent loop themselves. The service manages the harness underneath, covering sessions, context handling, tool use, and subagents, while you define the job. That makes it a good moment to ask a practical question rather than a news question: what does it actually take to build a real agent with it, and is the result better than simply doing the same job by hand in ChatGPT?
This article lays out the exact experiment we designed to answer that: the task, the architecture, the prompts, the checkpoints, and a scorecard. One note on honesty before we start. Everything below is a test plan with expected behavior and the failure points to watch for, not a report of finished results. Run it yourself, fill in the scorecard at the end, and you will have your own evidence instead of someone else's hype.
What the Agents API Actually Is (in Plain Language)
If you have used a chatbot, you know the pattern: you send a message, it answers, and the conversation ends when you close the tab. An agent works differently. It receives a goal, breaks it into steps, uses tools, inspects its own results, and keeps going until the job is finished or it needs help. We unpacked that shift in our explanation of how AI agents plan and use tools, and the Agents API is essentially a hosted version of that loop.
The pieces worth knowing about:
- Sessions: a long-lived unit of work that continues across many steps instead of a single question and answer.
- An environment: a sandbox where the agent can read and write files and run commands. When subagents are used, they share this filesystem with the coordinator.
- Tools: built-in capabilities such as web search, plus custom functions and external tool connections you define.
- Subagents: helper agents the main agent can hand work to, with a configurable limit on how many run at the same time.
- Managed context: the service handles the problem of long tasks outgrowing the model's working memory, which is one of the hardest parts of building agents yourself.
It is still a public beta, so features, limits, and pricing can change. Check the current documentation before you build anything important on it.
The Task We Chose
A good test task needs to be realistic, multi-step, and checkable. We chose something that any content team, blogger, or small business does regularly:
Take a list of AI article ideas, research each one, categorize them, create an outline for each, and identify what information is still missing before the article could be written.
We kept the input deliberately small so that a human can verify every output. The sample list has eight ideas:
- AI note-taking apps for students
- How to read an AI model card
- Prompt templates for weekly planning
- Running AI on an ordinary laptop
- An AI privacy settings checklist
- AI for spreadsheets: a beginner's path
- Choosing between chat assistants
- AI agents for small businesses
This task is a fair test because it has a research stage, a judgment stage (categorizing), a creative stage (outlining), and a critical stage (finding gaps). Those stages stress an agent in different ways.
The Architecture: One Coordinator, Three Specialists
Rather than asking one agent to do everything, we planned a small team. A coordinator owns the plan and the final output. Three subagents each own one stage. All of them read and write files in a shared workspace, which makes every intermediate result visible and inspectable.
| Role | Responsibility | Reads | Writes |
|---|---|---|---|
| Coordinator | Plans the work, delegates, merges results, keeps a run log | ideas.csv, all outputs | run-log.md, final-report.md |
| Research subagent | Gathers key points and open questions for each idea | ideas.csv | research/idea-1.md to idea-8.md |
| Categorizing subagent | Groups ideas into themes with a short reason for each | research folder | categories.md |
| Outline and gap subagent | Drafts an outline per idea and lists missing information | research folder, categories.md | outlines/, gaps.md |
The setup choices we planned: one sandbox environment with file access, web search enabled for the research stage only, a low limit on simultaneous subagents so the run is easy to follow, and no connection to any real account, inbox, or private data. That last point matters. An agent that can act should start with the smallest set of permissions that gets the job done, a principle we expanded on in our guide to guardrails, monitoring, and AI risk.
The Prompts We Used
Agent behavior is mostly decided by instructions. Vague instructions produce vague work, and they also produce confident-sounding work, which is worse. Here is the coordinator prompt we drafted:
You manage a content-planning job. Read ideas.csv. Delegate research for every idea to the research subagent, then categorization, then outlines and gap analysis. After each stage, write a short summary of what was done and what is uncertain to run-log.md. Never invent facts. If something cannot be confirmed, write it under "Unverified" instead of stating it as true. Stop and ask for approval before starting the outline stage.
And the research subagent's instruction:
For each idea, write five key points a beginner should know, three questions a reader is likely to ask, and a list of claims that would need verification. Label every statement as "confirmed by a search result" or "general knowledge". Do not write an outline.
Three habits make these prompts work. They give each agent one job, they force uncertainty to be labeled instead of hidden, and they build in an approval gate. The same principles behind good chat prompts apply here, just with higher stakes, and we covered the basics in our practical techniques for writing better AI prompts.
The Run We Expected, Step by Step
| Step | What Should Happen | What a Human Should Check |
|---|---|---|
| 1. Planning | Coordinator reads the file and writes a plan to the run log | Does the plan match your intent? |
| 2. Research | Eight research files appear, using web search | Spot-check three files for wrong or mixed-up facts |
| 3. Categorizing | Ideas grouped into themes, each with a reason | Do the groups make sense, or are they forced? |
| 4. Approval gate | Agent pauses and asks to continue | Approve, redirect, or stop |
| 5. Outlines and gaps | Eight outlines plus a list of missing information | Are the gaps real, or just generic filler? |
| 6. Final report | Coordinator merges everything into one report | Does it match the files, or did it embellish? |
The tool calls we expected to see in the activity log were file reads and writes (many), web searches (concentrated in step 2), and subagent delegations (three or four across the run). Reading that log is part of the experiment. If the agent makes dozens of near-identical searches, or rewrites the same file repeatedly, that is a signal the instructions are too loose.
Where We Expect the Agent to Succeed
- Repetitive structured work. Eight ideas, each needing the same treatment, is exactly where agents shine. A person gets bored by idea five; an agent does not.
- Keeping artifacts organized. Because every stage writes to a file, you end up with an auditable trail instead of one long chat to scroll through.
- Parallel research. Subagents can work on separate ideas at the same time, which matters when the list grows from eight to eighty.
- Surfacing questions you hadn't asked. The gap analysis stage often produces useful prompts for human thinking, even when its answers need editing.
Where We Expect It to Get Stuck
- Confident research errors. Search-based answers can still mix up details or lean on weak sources. This is the same failure we described in our breakdown of AI hallucinations, only now buried several steps deep where it is harder to see.
- Generic categories. Without examples, grouping tends toward obvious buckets like "beginner" and "advanced" that add little insight.
- Template outlines. Eight outlines that all follow the same skeleton look thorough but may not reflect what each topic really needs.
- Filler gaps. "Needs more research" is not a gap. A useful gap is specific, such as "no verified comparison of free-tier limits."
- Runaway loops. A poorly specified stage can lead to repeated work. Set clear stop conditions and keep an eye on usage.
How Much Human Supervision Is Realistic?
The honest answer is more than the demos suggest and less than doing everything by hand. A reasonable supervision plan for this task looks like this:
| Checkpoint | Human Action | Time to Budget |
|---|---|---|
| Before launch | Write instructions, define files, set permissions | The longest part, especially the first time |
| After research | Spot-check facts in a sample of files | Short, but do not skip it |
| At the approval gate | Review categories, approve or redirect | Short |
| After the final report | Compare report with underlying files, edit outlines | Moderate |
The rule we would follow: never let an agent's summary replace your inspection of its files. If you want a broader mental checklist for trusting AI output, our list of things to keep in mind before you trust AI applies directly here.
Agent vs. Manual ChatGPT: The Fair Comparison
To know whether an agent is actually better, run the same eight ideas manually in a normal chat session, and time both approaches. The comparison should cover more than speed.
| Criterion | Manual Chat Approach | Agent Approach |
|---|---|---|
| Setup effort | Almost none | Real design work up front |
| Your attention during the run | Constant, one prompt at a time | Mostly at checkpoints |
| Scales to 80 ideas | Painful | Much more practical |
| Traceability | Scattered through a chat history | Files and a run log |
| Error visibility | Errors appear in front of you | Errors can hide inside earlier steps |
| Cost | Subscription you may already have | Usage-based, so track it |
Our expectation, which the scorecard below is designed to confirm or refute, is that manual chat wins for a one-off list of eight ideas, and the agent starts to win when the same job repeats weekly or the list grows large. If you are still choosing which model to put behind a job like this, we compared the current options in which latest AI model fits you.
Sample Scorecard: What a Completed Run Might Look Like
To show how to read your own results, here is a filled-in example. They are plausible values for this kind of task, meant to show what a useful comparison looks like. Copy the blank version of this table into a notes app, run both approaches yourself, and replace every figure with your own.
| Measure | Agent Run (Example) | Manual Run (Example) |
|---|---|---|
| Setup time before starting | 45 minutes | 5 minutes |
| Total elapsed time | 38 minutes | 65 minutes |
| Hands-on time | 20 minutes (checkpoints and review) | 65 minutes (one prompt at a time) |
| Factual errors found in a 3-file spot-check | 4 | 2 |
| Outlines usable with light edits (out of 8) | 5 | 6 |
| Useful, specific gaps identified | 9 | 6 |
| Times you had to intervene | 3 | Continuous |
| Approximate usage cost | Higher (usage-based) | Covered by an existing subscription |
How to Read a Scorecard Like This
The raw numbers matter less than the pattern they show. In this example, the agent saved hands-on time and found more specific gaps, but it took longer to set up, produced more factual slips in the spot-check, and delivered slightly fewer ready-to-use outlines. That is a realistic trade-off, and it leads to a sensible verdict:
- For a one-time list of eight ideas: the manual approach wins, because the setup time cancels out the savings.
- For a weekly job or a list of eighty ideas: the agent starts to win, since setup is a one-time cost and review time grows slowly.
- For anything where accuracy is critical: the higher error count means you should budget extra time to verify the agent's research files before using them.
When you run your own comparison, add one more line at the bottom of your notes: "Would I repeat this approach next week?" That honest yes or no is often a better verdict than any single number in the table.
Safety Notes Before You Build
- Use a sandbox with dummy or public data for your first runs.
- Do not connect accounts, email, payment tools, or private documents until you trust the behavior.
- Keep an approval step before any stage that changes or sends something outside the workspace.
- Set a usage limit so a looping agent cannot run up a bill.
- Treat anything the agent reads from the web as untrusted input, not as instructions.
Is It Worth Building an Agent at All?
A simple decision rule helps. Build an agent when the job is repetitive, has clear stages, produces files you can inspect, and happens often enough to justify setup time. Stay with a normal chat when the task is a one-off, needs your judgment at every step, or involves information you cannot afford to get wrong. The most common mistake is reaching for an agent because it sounds advanced, when a well-written prompt would finish the job in ten minutes.
Frequently Asked Questions
What is the OpenAI Agents API?
It is a managed service, in public beta since September 2026, that lets developers start long-running cloud agents with tools, files, code execution, and subagents, without building the underlying agent loop themselves.
Do I need to be a developer to try this?
Yes, at least to some degree. It is an API, so you need an account with API access and basic comfort with setup. If you are not technical, the planning and checkpoint ideas in this article still apply to any agent product.
What is a subagent?
A subagent is a helper agent that the main agent delegates a piece of work to. Splitting a job this way keeps each agent focused and allows some work to happen in parallel.
Is an agent always better than using ChatGPT manually?
No. For short, one-off tasks, a chat is usually faster. Agents earn their setup cost on repeated, multi-step, file-based work.
How much human supervision does an agent need?
More than demos suggest. Plan for checkpoints after research and before any step that changes something, and always inspect the underlying files rather than trusting a summary.
Is it safe to give an agent access to my accounts?
Start with none. Use a sandbox and public or dummy data, add permissions one at a time, and keep approval gates in front of risky actions.
Final Thoughts
The interesting thing about the Agents API is not that an AI can now run for a long time. It is that the hard infrastructure around long-running work, such as keeping state, managing context, and coordinating helpers, is now something you can rent instead of build. That lowers the barrier, but it does not remove the real work: choosing a task worth automating, writing clear instructions, deciding where a human must stay in the loop, and checking the results honestly.
If you run the experiment above, you will learn more from your scorecard than from any announcement. And the answer may well be that for your workload, a careful chat session is still the better tool. That is a perfectly good outcome. The goal is not to use an agent. The goal is to finish useful work with less effort and fewer mistakes.
Comments
Post a Comment