Best practices for building a great agent

A good AI agent is not the result of one clever trick. It is the result of a few disciplined habits, repeated. This post collects the practices that consistently produce agents customers trust.

The theme running through all of them is the same: be specific, be honest about limits, and improve from real conversations rather than assumptions.

Organize knowledge so answers are easy to find

Your agent does not reread your entire knowledge base on every message. It looks up the pieces most relevant to the question and answers from those. That means how you organize knowledge directly controls answer quality, more than almost anything else you do.

Practical rules that follow from this:

• Use descriptive headings. A section called "Returns within 30 days" is easy to find. A section called "Policy details" is not.
• Keep each section focused on one topic. One question, one answer, one idea.
• Avoid giant undivided documents. Break a long policy into labeled sections.
• Put the most important sentence near its heading, not buried three paragraphs down.

Tip: Read your own documents the way the agent does. Skim only the headings. If a heading does not tell you what its section answers, the agent will struggle to surface it at the right moment. Rewrite the heading until it does.

Use Q&A pairs for your highest traffic questions

A Q&A pair is stored exactly as you wrote it, question and answer together. That makes them ideal for your most common, most exact questions: store hours, return windows, shipping timelines, warranty terms.

If you know a question gets asked daily and has a precise answer, make it a Q&A pair. You get a clean, predictable response every time.

Keep live data out of static knowledge

Prices and inventory change. If you write them into documents, they go stale, and a confidently wrong price is worse than no price. Connect your store instead, so the agent reads pricing and stock from the source. Let static knowledge hold the things that do not change hour to hour: how returns work, how sizing runs, how to care for a product.

Less, but better

Every plan includes a knowledge allowance, shared across everything you add. More knowledge is not automatically better. A focused base of accurate, well organized content outperforms a sprawling one full of duplicates and noise. Prune aggressively. If a source has never helped answer a question, it is taking up room.

Tip: Duplicate content is worse than missing content. When two sources answer the same question slightly differently, the agent can surface the weaker one. Keep a single source of truth for each fact.

Write instructions that hold up under pressure

Your custom instructions are the agent's personality and its rules. The agent takes them seriously and follows them closely, so write them with that power in mind.

Be concrete about behavior

Vague instructions produce vague agents. Compare:

Be helpful and friendly.

with:

Greet customers by name when you know it. Answer in two or three
sentences, then offer a next step. If a customer is frustrated,
acknowledge it before solving the problem.

The second version tells the agent what to actually do. The first is a wish.

State what the agent must not do

The instructions are also where you set hard limits. Be explicit about claims the agent should never make and topics it should never handle on its own. For most stores that includes medical or health claims, legal advice, and price negotiation. An agent that knows its boundaries is safer than one that is merely told to be careful.

Give it an exit

Tell the agent what to do when it cannot help, and who to point the customer to. An agent with a clear escalation path fails gracefully. An agent without one improvises, and improvisation is where confidently wrong answers come from.

Tip: Write instructions as rules a new hire could follow, not adjectives. "Confirm the order number before discussing an order" is a rule. "Be careful with orders" is an adjective. The agent follows rules far more reliably than vibes.

Make not knowing a feature

The single most damaging thing an agent can do is invent an answer. Customers forgive an agent that says it is not sure and offers to connect them with a person. They do not forgive one that sends them to the wrong place with total confidence.

Design for the unknown explicitly:

• Instruct the agent to say clearly when something is outside its knowledge.
• Give it a next step for those moments, whether that is escalation or a link.
• Test it on questions you deliberately left out of its knowledge, and watch what it does.

If your agent passes the questions it should fail, you have built trust into it.

Choose the model for the job, not the hype

Knowledge and instructions decide what the agent knows and how it behaves. The model decides how well it works with both, and how much each reply costs against your plan.

A bigger model cannot know facts you never gave it, so most quality problems are knowledge problems wearing a model costume. Fix knowledge first. Reach for a stronger model only when the agent has the right information but still struggles on your hard conversations. We cover the full trade off in the companion guide on choosing a model.

Tip: When you investigate a bad answer, ask one question first: did the agent have the knowledge to answer? If not, no model upgrade will help. If yes, then a stronger model is worth testing.

Set up escalation before you need it

No agent should handle everything. The goal is for it to handle the volume of routine questions well, and to route the rest to a human cleanly.

On paid plans, you can enable human handover. When the agent decides a conversation needs a person, or a customer asks for one, it hands the thread over to your team. The customer stays in the same conversation, and your team picks up with the full history visible. There is no repeating, no lost context.

To make this work in practice:

• Decide which situations should always escalate, and put them in the instructions.
• Make sure someone is watching the live chat queue during your support hours.
• Treat escalations as data. A spike in handovers on one topic is a knowledge gap asking to be filled.

Treat the agent as something you tune, not something you ship

The biggest difference between agents that improve and agents that stall is whether anyone reads the conversations.

Build a simple weekly habit:

1. Read a sample of real conversations end to end.
2. Sort the failures into two piles: missing knowledge and weak instructions.
3. Fix the single worst gap in each pile.
4. Note any topic that escalated repeatedly and decide whether the agent should learn to handle it.

Small, steady corrections compound. An agent that gets one real fix a week is dramatically better in three months than one that was launched and forgotten, no matter how polished its launch was.

Tip: Keep a running log for each agent: what you changed, when, and why. When quality shifts, you can trace it to a change instead of guessing. It also stops two people from undoing each other's work.

The practices in one place

If you remember nothing else:

• Organize knowledge into clear, focused, well titled sections.
• Use Q&A pairs for your most common exact questions.
• Keep prices and stock in your store, not in static documents.
• Write concrete instructions, including what the agent must not do.
• Make honest uncertainty a designed behavior, not an accident.
• Fix knowledge before you reach for a bigger model.
• Set up escalation before launch and watch the queue.
• Read conversations every week and fix the worst gap.

None of these are clever. All of them work. The agents customers come to trust are simply the ones whose owners kept doing the unglamorous parts.