AI agent vs chatbot vs workflow automation: what to build

The difference in the AI agent vs chatbot question is who decides the steps. A chatbot answers questions in a conversation, a workflow automation runs steps you have defined in code (with or without an LLM in some of them), and an AI agent uses a model to decide which steps and tools to use until a task is done. Most businesses should start with the simplest of the three that solves the problem, and move to an agent only when the task genuinely needs flexible, multi-step judgement.

Key takeaways

• A chatbot converses, a workflow follows predefined code paths, and an agent lets the model direct its own process and tool use.
• OpenAI's agent guide says simple chatbots and single-turn LLM features are not agents, because they do not control how a workflow is executed.
• Agents fit tasks with nuanced judgement, unwieldy rule sets or heavy unstructured data; otherwise a deterministic workflow is cheaper and easier to test.
• Agents trade extra latency and cost for flexibility, and they add risks such as excessive agency that need human checkpoints and limits.
• Whichever you build, EU AI Act transparency rules require people to be told they are interacting with an AI system unless it is obvious.

What is the difference between a chatbot, a workflow and an AI agent?

The difference is how much control the model has over what happens next. These working definitions follow the language used by the main model providers:

• A chatbot is a conversational interface that answers questions, often grounded in your documents. It replies; it does not usually act in other systems.
• A workflow automation is a sequence of steps fixed in code. Anthropic describes LLM workflows as systems where models and tools are orchestrated through predefined code paths. Classic automations without any LLM also fall here.
• An AI agent is a system where the model dynamically directs its own processes and tool usage. OpenAI's guide defines agents as systems that independently accomplish tasks on your behalf, and is explicit that simple chatbots, single-turn LLMs and sentiment classifiers are not agents.

In short: a chatbot talks, a workflow follows a script, an agent chooses its own script.

How do they compare side by side?

The three options differ most in predictability, cost and the amount of testing they need.

	Chatbot	Workflow automation	AI agent
Who decides the steps	The user, turn by turn	Your code	The model, within limits you set
Typical job	Answer questions, guide users	Route, extract, transform, notify	Resolve a multi-step task end to end
Takes actions in other systems	Rarely	Yes, fixed actions	Yes, chosen at run time
Predictability	High for answers	Highest	Lowest
Model calls per task	Usually one per turn	A fixed number	Variable, sometimes many
Testing effort	Answer quality	Each step	Whole trajectories, plus edge cases
Main risks	Wrong or made-up answers	Breaks when inputs change	Wrong actions, loops, runaway cost
Good first project	FAQ or help centre assistant	Invoice or email triage	Refund handling with approval step

The "model calls per task" row is where costs diverge. Anthropic notes that agentic systems often trade latency and cost for better task performance. Because each agent step resends instructions and context, an agent can use several times the tokens of a chatbot answering the same question. Our breakdown of the cost to build an AI agent shows how quickly that adds up at current prices.

When is a chatbot enough?

A chatbot is enough when the user mainly needs information and a person, or the user, carries out any action. Help centres, product documentation, internal policy questions and pre-sales questions usually fit. The work goes into grounding answers in your own content, saying "I don't know" when the content is missing, and offering a clear handover to a person. A chatbot that cannot change anything in your systems also limits the damage when it is wrong.

When should you use a workflow automation instead?

Use a workflow when you can write the steps down in advance. If a process always follows "receive email, extract invoice fields, check against the purchase order, post to accounting, flag mismatches", fixing those steps in code gives you predictable cost and behaviour, easier testing and a clear audit trail. An LLM can still do the hard parts, such as reading an unstructured invoice, inside a single step. Anthropic's advice is to find the simplest solution possible and add complexity only when needed, and it notes that optimising single LLM calls with retrieval and examples is often enough.

When does an AI agent make sense?

An agent makes sense when the number and order of steps cannot be predicted in advance. OpenAI's guide suggests prioritising workflows that have resisted automation and lists three signals (practical guide to building agents):

1. Complex decision-making: nuanced judgement, exceptions or context-sensitive decisions, such as refund approvals.
2. Difficult-to-maintain rules: rule sets that have become unwieldy and costly to update, such as vendor security reviews.
3. Heavy reliance on unstructured data: interpreting language and documents, such as processing an insurance claim.

The same guide adds that if a use case does not clearly meet these criteria, a deterministic solution may suffice. A practical test: if you can draw the process as a flowchart with fewer than a dozen branches, build the workflow.

What risks does each option add?

Risk grows with autonomy. The OWASP Top 10 for LLM applications lists prompt injection, sensitive information disclosure and excessive agency among the main risks. Excessive agency, a system with more permissions or autonomy than it needs, mostly applies to agents and to workflows that act in other systems.

Three controls apply to any agent you build:

• Least privilege: give each tool only the permissions the task needs, and prefer read access until write access is proven necessary.
• Stopping conditions: Anthropic recommends limits such as a maximum number of iterations, and a spend cap per task.
• Human checkpoints: OpenAI recommends human oversight for sensitive, irreversible or high-stakes actions such as cancelling orders, large refunds or payments, at least until the agent has a track record. Our guide to human-in-the-loop AI covers how to design those approval steps.

Do EU rules treat chatbots and agents differently?

Not for basic transparency. Article 50(1) of the EU AI Act requires providers to design AI systems that interact directly with people so that those people are informed that they are interacting with an AI system, unless that is obvious from the context. This applies to a chatbot and to an agent that emails or chats with customers. The application dates for parts of the Act have been amended, so check our summary of EU AI Act transparency rules for chatbots for the current position. GDPR applies to all three whenever personal data is processed.

How do you decide what to build first?

Pick the narrowest option that removes real work, then grow. A sensible order for many SMEs:

1. List the task, its inputs, its outputs and who acts on the result.
2. If nobody needs anything done in another system, build a chatbot.
3. If the steps can be written down, build a workflow, with an LLM inside the steps that need language understanding.
4. If the steps genuinely vary case by case, prototype an agent with read-only tools, a step limit and a human approval step for every action.
5. Measure accuracy, cost per task and escalation rate on real examples before you widen its permissions.

Frequently asked questions

Is ChatGPT a chatbot or an AI agent?

It depends on how it is used. Answering questions in a chat window is chatbot behaviour. When an assistant is given tools and decides by itself which ones to call over several steps to complete a task, it is behaving as an agent, which is how OpenAI's own agent guide draws the line.

Can a workflow automation use an LLM?

Yes. Many of the most reliable AI features are workflows where an LLM handles one step, such as classifying an email or extracting fields from a document, while code controls the order of steps. Anthropic calls these workflows, as distinct from agents.

Are AI agents more expensive to run than chatbots?

Usually, because an agent makes several model calls per task and resends its instructions and context each time. Prompt caching and smaller models reduce the gap, but an agent's cost per task is also less predictable, so set a spend limit per task.

Should a small business build an AI agent in 2026?

Only for a task that clearly needs flexible judgement and that you can measure. Many SME use cases are served well by a grounded chatbot or a workflow with an LLM step, which are cheaper to build, test and explain to customers.

Do I need to tell customers they are talking to an AI?

Under Article 50(1) of the EU AI Act, people interacting directly with an AI system must be informed of it unless that is obvious from the context. Check the current application date, which has been affected by later amendments, and disclose it clearly anyway as good practice.

If you are weighing a chatbot, a workflow or an agent for a specific process, book a free 30-minute call and we will help you choose.

Cover photo by Jep Gambardella on Pexels.