A guide to the anatomy of effective commerce agents

频道: Claude Blog
原文: https://claude.com/blog/the-anatomy-of-effective-commerce-agents
原文语言: en
统计: 共 64 轮

[¶1]

Over the past year, we've worked with teams across the commerce industry — retailers, marketplaces, travel, entertainment, and telecom providers — to build commerce agents using Claude.

These agents are in production, and enterprise customers have seen larger carts and more efficient seller operations when using them. They also share a simple architecture: Claude in an agent loop equipped with a set of skills, tools, and a strong eval suite.

This post is for the engineers and engineering leaders building these (or other consumer facing) agents. Part 1 covers the architecture, which you decide once. Part 2 covers latency and cost. Part 3 covers production: memory, safety, evals, and scaling the work across an organization.

过去一年，我们与商务领域各行各业的团队合作——零售商、电商平台，以及旅行、娱乐和电信服务商——用 Claude 构建商务 agent（commerce agent，帮人在线上买卖东西的 agent）。

这些 agent 已经投入生产，企业客户用上之后，购物车规模变大了，卖家运营也更高效了。它们还共享同一种简洁的架构：Claude 运行在 agent 循环中，配备一组 skill、一组工具，以及一套扎实的 eval 套件。

本文写给正在构建这类 agent（或其他面向消费者的 agent）的工程师和工程负责人。第一部分讲架构，这件事只需决定一次；第二部分讲延迟与成本；第三部分讲生产运行：记忆、安全、eval，以及如何在整个组织内规模化推进这项工作。

[¶2]

Reference implementation

We've also provided a blueprint to help build commerce agents on Claude. It contains the harnesses, patterns, and guardrails an engineering team needs to get a commerce agent running in days, with reference implementations of a shopping agent and a merchant agent for retail, travel, telecom, and ticketing platforms.

anthropics/commerce-agents →

In this guide

1. Part 1: The architecture
   1. What is a commerce agent?
   2. Skills, not subagents
   3. System prompt or skill: decide by frequency
   4. Engineering agent tooling
   5. The UI components are tools
2. Part 2: Making it fast and affordable
   1. Minimizing task completion latency
   2. Perceived latency
   3. Prompt caching
   4. Choosing the model and its configuration
3. Part 3: Running it in production
   1. Memory that survives the session
   2. Safety: enforcement lives in the harness
   3. Evals: shipping a non-deterministic system
   4. Shipping with a large organization
4. Looking ahead

参考实现

我们还提供了一份蓝图，帮助大家在 Claude 上构建商务 agent。它包含工程团队在几天内让商务 agent 跑起来所需的 harness（外壳/运行框架）、模式和护栏，并附有购物 agent 与商家 agent 的参考实现，覆盖零售、旅行、电信和票务平台。

anthropics/commerce-agents →

本指南内容

1. 第一部分：架构
   1. 什么是商务 agent？
   2. 用 skill，而不是 subagent
   3. system prompt 还是 skill：按使用频率决定
   4. agent 工具的工程设计
   5. UI 组件就是工具
2. 第二部分：让它又快又省
   1. 最小化任务完成延迟
   2. 感知延迟
   3. Prompt caching
   4. 选择模型及其配置
3. 第三部分：在生产环境中运行
   1. 跨会话留存的记忆
   2. 安全：强制执行落在 harness 里
   3. Eval：发布一个非确定性系统
   4. 在大型组织中发布
4. 展望未来

[¶3]

01 · The architecture

One model in a standard agent loop, with skills for the long tail and tools that call the systems you already run. You decide this once.

01 · 架构

一个模型运行在标准的 agent 循环中，用 skill 覆盖长尾功能，用工具调用你已经在运行的系统。这件事只需决定一次。

[¶4]

What is a commerce agent?

We define a commerce agent as an agent that simplifies buying and selling across an online catalog.

Some agents face consumers: they search, compare, substitute, and assemble the order. That could be a retail cart, a travel itinerary, a mobile plan change, or seats held for a show. Some agents face the business: they answer questions about sales, run promotions and campaigns, and manage inventory and pricing.

The core architecture is a model in a standard agent loop: reasoning about a goal, exploring context, taking actions through tools, learning procedures through skills, asking clarifying questions, and observing the results until the goal is accomplished.

什么是商务 agent？

我们对商务 agent 的定义是：让在线商品目录上的买卖变得更简单的 agent。

有些 agent 面向消费者：它们负责搜索、比较、寻找替代品，并把订单组装好。这个订单可能是一个零售购物车、一份旅行行程、一次手机套餐变更，也可能是为某场演出预留的座位。另一些 agent 面向商家：回答有关销售的问题，开展促销与营销活动，管理库存和定价。

其核心架构，是一个运行在标准 agent 循环中的模型：围绕目标进行推理，探索上下文，通过工具采取行动，通过 skill 习得操作流程，提出澄清问题，并观察结果，如此往复，直至目标达成。

[¶5]

There is no intent router in front of it that segments the conversation and no set of domain specific agents behind it.

在它前面，没有一个负责切分对话的意图路由器；在它后面，也没有一组按领域划分的专用 agent。

[¶6]

Engineering context

上下文的工程设计

[¶7]

Skills, not subagents

A commerce agent has to cover a wide range of capabilities across many categories and intents, which makes it tempting to create one subagent per domain.

In practice this proves suboptimal, because a commerce conversation is one tightly coupled session across multiple intents and turns, and requires considerable shared context.

In a subagent architecture, the orchestrator holds the cart or staged changes, the user's preferences, and the conversation history.

Every handoff to a subagent is a state-lossy operation, which often impacts the quality of the subagent’s response and, consequently, the overall response. On top of that, each handoff can cost several times the tokens and adds seconds of latency.

用 skill，而不是 subagent

商务 agent 要覆盖众多品类和意图下的大量能力，这很容易让人想为每个领域各建一个 subagent（子 agent）。

但实践证明，这种做法并非最优，因为一次商务对话是横跨多种意图、多个轮次的紧耦合会话，需要大量共享上下文。

在 subagent 架构中，orchestrator（编排者）掌握着购物车或暂存变更、用户偏好以及对话历史。

每一次 hand-off（移交）给 subagent，都是一次会丢失状态的操作，往往会拉低 subagent 的回复质量，进而拉低整体回复的质量。除此之外，每次 hand-off 都可能让 token 消耗翻上几倍，并带来数秒的额外延迟。

[¶8]

The domains also rarely separate cleanly. A returns flow might need the order history, the current cart, and the product catalog, meaning a subagent-per-domain approach either duplicates that access everywhere or hands off mid-task.

As models get smarter, they also handle longer context, more skills, and more tools, so the limits behind today's placement rules loosen with each model generation.

Instead, agent skills give you similar per-domain modularity and context control without the handoff tax, because the skill instructions load into the main agent that already holds the entire history.

各个领域之间也很少能干净利落地切分开。一个退货流程可能同时需要订单历史、当前购物车和商品目录，这意味着「每个领域一个 subagent」的做法，要么在每个 subagent 里重复接入这些数据，要么就得在任务进行到一半时 hand-off。

随着模型越来越聪明，它们也能处理更长的上下文、更多的 skill 和更多的工具，因此今天这些放置规则（什么内容该放在哪里）背后的限制，会随着每一代新模型而逐步放宽。

更好的办法是 agent skills：它同样能带来按领域划分的模块化和上下文控制，却不必缴纳 hand-off 这笔「税」，因为 skill 的指令会直接加载进主 agent，而主 agent 本来就掌握着完整的历史。

[¶9]

In our comparisons across several enterprise deployments, a single agent with skills consistently has outperformed both the one-prompt-for-everything design and the subagent design on quality, and often at a lower cost and latency per task.

Where subagents do earn their place is when the orchestrator can call them as a tool for a narrow or self-contained task that would benefit from its own dedicated context window.

A common production example is a deep-research subagent, where the subagent searches and reads documents, writes and runs code, traverses data models, and hits dead ends. All the work happens inside one or more subagents, and only a compact answer comes back to the orchestrator.

在我们对多个企业部署所做的对比中，带 skill 的单一 agent 在质量上始终优于「一个 prompt 包揽一切」的设计和 subagent 设计，而且单个任务的成本和延迟往往也更低。

subagent 真正有用武之地的情形，是 orchestrator 可以把它当作工具来调用，去完成一项范围狭窄或自成一体、能从专属上下文窗口中获益的任务。

生产中一个常见的例子是深度研究 subagent：由它去搜索和阅读文档、编写并运行代码、遍历数据模型，也难免走进死胡同。所有这些工作都发生在一个或多个 subagent 内部，最后只有一个精简的答案回到 orchestrator 手里。

[¶10]

The other exception is a domain that already has its own purpose-built agent. If your pharmacy or financial-services experience runs a dedicated agent with its own compliance surface, the right move can be a hand-off, where that agent takes over the task and works with the user directly through its own loop until the task is done.

The distinction is ownership of the conversation. A hand-off makes the domain agent the user's counterpart, while delegation keeps the orchestrator, bouncing the domain agent in and out within a single turn and degrading on every exchange.

另一个例外，是某个领域已经有了专门为它打造的 agent。如果你的药房或金融服务板块已经运行着一个专属 agent，并且有自己独立的合规边界，那么正确的做法可能是 hand-off：由那个 agent 接管任务，通过它自己的循环直接与用户协作，直到任务完成。

两者的区别在于对话的主导权归谁。hand-off 让领域 agent 成为直接面对用户的一方；delegation（委派）则让 orchestrator 继续掌控对话，在单个轮次内把领域 agent 反复调进调出，每一次往返都会让效果打上折扣。

[¶11]

System prompt or skill: decide by frequency

The main factor when deciding whether to put a set of instructions within a system prompt or skill is how often the agent will need it. Loading a skill costs a model turn, so anything the agent needs on most turns generally goes in the system prompt.

This does, however, depend on how your traffic is distributed, and what agent behavior your evals show. A good starting point is that anything relevant to a third or more of your traffic, whether anticipated before launch or observed in production, goes in the system prompt, and the rest goes in skills.

If a skill is predictable from a signal you already have, such as the page the user arrived from, we recommend injecting it from the harness before the first model call and skipping the extra turn to load the skill.

system prompt 还是 skill：按使用频率决定

决定一组指令该放进 system prompt 还是 skill 时，主要的考量是 agent 需要它的频率。加载一个 skill 要耗费一个模型轮次，所以 agent 在大多数轮次都需要的内容，一般放进 system prompt。

不过，这也取决于你的流量如何分布，以及你的 eval 所呈现的 agent 行为。一个不错的起点是：凡是与三分之一及以上流量相关的内容——无论是上线前预判到的，还是在生产环境中观察到的——都放进 system prompt，其余的放进 skill。

如果某个 skill 能根据你已有的信号预判出来，比如用户是从哪个页面进来的，我们建议由 harness 在第一次调用模型之前就把它注入，省掉加载 skill 的那个额外轮次。

[¶12]

Critical instructions, such as safety and legal rules, brand constraints, and key user facts such as allergies, always go in the system prompt.

For commerce agents, this means product search lives in the prompt, since nearly every session touches it, and skills carry the long tail of features.

In our reference implementation, the shopping agent's prompt holds grounding, cart and checkout semantics, and presentation rules, and the following skills cover the rest: search-discovery, purchase-research, planning-goals, customer-care, and memory-personalization.

The merchant agent splits the same way, with performance-insights, catalog-listings, inventory-operations, pricing-promotions, and marketing-campaigns as its skills, one per operational domain.

关键指令——例如安全与法律规则、品牌约束，以及过敏史这类关键用户信息——一律放进 system prompt。

对商务 agent 来说，这意味着商品搜索应当写在 prompt 里，因为几乎每个会话都会用到它；长尾功能则交由 skill 承载。

在我们的参考实现中，购物 agent 的 prompt 包含事实锚定（grounding）、购物车与结账语义以及展示规则，其余部分由以下 skill 覆盖：search-discovery、purchase-research、planning-goals、customer-care 和 memory-personalization。

商家 agent 也按同样的方式拆分，它的 skill 是 performance-insights、catalog-listings、inventory-operations、pricing-promotions 和 marketing-campaigns，每个运营领域对应一个。

[¶13]

• In the prompt · Shopping agent · Grounding, cart and checkout semantics, presentation rules, and product search.

• Shopping skills · The long tail · search-discovery · purchase-research · planning-goals · customer-care · memory-personalization

• Merchant skills · One per operational domain · performance-insights · catalog-listings · inventory-operations · pricing-promotions · marketing-campaigns

• 放在 prompt 里 · 购物 agent · 事实锚定、购物车与结账语义、展示规则，以及商品搜索。

• 购物 skill · 长尾功能 · search-discovery · purchase-research · planning-goals · customer-care · memory-personalization

• 商家 skill · 每个运营领域一个 · performance-insights · catalog-listings · inventory-operations · pricing-promotions · marketing-campaigns

[¶14]

Engineering agent tooling

Our post on writing effective tools for agents covers tool design in general. Two points have mattered most in commerce:

Build agent tools on top of your core systems and logic.

A commerce company already has search and ranking, a cart, a preferences and profile store, an inventory system, promotion and campaign engines, sales analytics, and more, each encoding logic tuned over years and seeing signals the model never will.

The agent's tools should call those systems, not reimplement them, and the tool boundary is where their logic ends and the model's judgment takes over.

agent 工具的工程设计

关于工具设计的一般原则，我们在为 agent 编写高效工具一文中已有论述。在商务场景中，最要紧的有两点：

把 agent 工具建在你的核心系统和业务逻辑之上。

一家商务公司本来就有搜索与排序、购物车、偏好与用户档案存储、库存系统、促销与营销活动引擎、销售分析等等，每一个系统都沉淀着多年调优的逻辑，也能看到模型永远看不到的信号。

agent 的工具应当调用这些系统，而不是重新实现它们；工具的边界，就是这些系统的逻辑止步、模型的判断接手之处。

[¶15]

For example, when the agent calls search_products, the results should arrive already ranked; its job is to decide which results serve the user's goal, how many to show, and how to present them.

Tool results are context.

Return the fields the model reasons with and drop the rest. Image URLs on every search row are the usual offender.

As needed, reshape the raw response inside the tool, including appending a next step when it isn't obvious from the data.

This is especially relevant for error scenarios, where the model benefits from instructions instead of error codes. For example, add an error instruction "Include a product ID when querying availability," instead of a generic 403.

例如，agent 调用 search_products 时，返回的结果应当已经排好序；agent 的职责是判断哪些结果符合用户的目标、展示多少条，以及如何呈现。

工具结果就是上下文。

只返回模型推理时要用到的字段，其余一律去掉。最常见的冗余，是每一行搜索结果都附带的图片 URL。

必要时，在工具内部对原始响应进行重塑，包括在数据本身看不出下一步该做什么时，附上一条下一步提示。

这一点在出错场景下尤为重要：此时对模型有帮助的是指示，而不是错误码。例如，与其返回一个笼统的 403，不如附上一条错误指示：「查询是否有货时，请附上商品 ID」。

[¶16]

The UI components are tools

Most commerce agent responses are UI components rather than prose, whether a product carousel, an itinerary, a seat map, or a chart. That means the agent has to emit a schema rather than text.

Teams sometimes start by prompting the model to emit custom tags and parsing them on the client-side. This stops working as the surface grows, because:

• The model isn’t as well trained on your markup as it is on tool calls so reliability drops as nested components get added. Well-formed data is not guaranteed just through prompting.

• The tag definitions live in the system prompt, so every new component bloats context and every edit risks regressions elsewhere in the prompt.

UI 组件就是工具

商务 agent 的回复大多是 UI 组件而非文字，比如商品轮播、行程单、座位图或图表。这意味着 agent 要输出的是 schema，而不是文本。

有些团队起步时，会通过 prompt 让模型输出自定义标签，再在客户端解析。随着界面规模扩大，这种做法就会失灵，原因在于：

• 模型在你的自定义标记上的训练远不如在工具调用上充分，所以随着嵌套组件增多，可靠性会下降。单靠 prompt，无法保证输出格式规范的数据。

• 标签定义写在 system prompt 里，于是每新增一个组件，上下文就膨胀一分；每做一次修改，都可能让 prompt 的其他地方出现回归。

[¶17]

• Past conversations end up stored in a format only your parser can read, so loading history means either parsing raw messages on the client or keeping a second copy in a format that isn't native to the model API.

The pattern that has held up is to make each UI component a tool. The model calls present_products, present_itinerary, or present_plan_comparison with typed arguments; your server validates and enriches the call and emits an event; and your client renders it.

As the components are tool calls, they're already in the messages array in native format, so you don’t need to re-parse when you reload an old conversation. An example presentation-tool contract is illustrated below and in the reference repo.

• 历史对话最终会以只有你的解析器才能读懂的格式存储，于是加载历史时，要么在客户端解析原始消息，要么另存一份并非模型 API 原生格式的副本。

经得起检验的模式，是把每个 UI 组件都做成一个工具。模型带着类型化的参数调用 present_products、present_itinerary 或 present_plan_comparison；你的服务端对这次调用进行校验、补充数据，然后发出一个事件；你的客户端再把它渲染出来。

由于组件就是工具调用，它们已经以原生格式存在于 messages 数组中，重新加载旧对话时无需再次解析。下方图示以及参考仓库中给出了一个 presentation tool（展示型工具）契约的示例。

[¶18]

The tradeoff is streaming granularity. Each top-level argument of a tool call buffers on the server for validation, so the sub-components of a presentation tool arrive in steps even with streaming on. This impacts perceived latency.

To get a token-level stream, set eager_input_streaming: true on the tool definition, which skips the buffering and with it the server-side schema guarantee.

In our evals, schema violations are very rare on Claude Sonnet-class models and up, but wrap the call in a retry for the cases where one slips through.

Presentation tools also give the agent a record of what's on screen. When a customer says "the first hotel" or "the third one down on the left," the layout is in the messages array, in the arguments of the last presentation call.

代价在于流式输出的粒度。工具调用的每个顶层参数都会先在服务端缓冲以便校验，所以即使开启了流式输出，presentation tool 的各个子组件也是分段到达的。这会影响感知延迟。

如果想要 token 级的流式输出，可以在工具定义上设置 eager_input_streaming: true，这会跳过缓冲，同时也放弃了服务端的 schema 保证。

在我们的 eval 中，Claude Sonnet 级别及以上的模型极少出现违反 schema 的情况，但还是应该给调用包一层重试，以防偶有违规漏网。

此外，presentation tool 还为 agent 留下了一份屏幕上显示内容的记录。当顾客说「第一家酒店」或「左边往下数第三个」时，布局就在 messages 数组里，在最近一次 presentation tool 调用的参数中。

[¶19]

For that to work, the arguments have to reflect the rendered layout, so structure them the way the UI is structured, as ordered rows and carousels rather than a flat list the client rearranges.

要让这一点成立，参数就必须反映渲染后的布局，所以要按照 UI 的结构来组织参数：用有序的行和轮播来表达，而不是给出一个扁平列表、再让客户端去重新排布。

[¶20]

02 · Making it fast and affordable

Attack latency on two fronts, end-to-end and perceived, and let caching carry the cost. None of it should spend intelligence to get there.

Latency matters in commerce, and consumer surfaces are the least forgiving. However, on agentic surfaces, what we have consistently seen move metrics like retention, engagement, and cart size is the quality of the outcome.

Whether the answer was relevant and the task actually completed was more critical to those metrics as compared to marginal latency gains.

So attack latency on two fronts. Minimize end-to-end latency through good engineering, and pair that with dropping perceived latency (since time spent watching an agent work reads as progress).

02 · 让它又快又省

从端到端和感知两条战线压缩延迟，成本则交给缓存来扛。这一切都不应以牺牲模型智能为代价。

在商务场景中，延迟很重要，而面向消费者的界面对延迟最不宽容。不过，在 agent 化的界面上，我们一再看到，真正能推动留存、参与度和购物车规模等指标的，是结果的质量。

回答是否切题、任务是否真正完成，对这些指标的影响，要比延迟上的边际改进关键得多。

所以，要从两条战线压缩延迟：通过扎实的工程把端到端延迟降到最低，同时降低感知延迟（因为看着 agent 干活的这段时间，会被用户视为进展）。

[¶21]

Every user has a latency budget, and the techniques below keep the agent inside it without spending intelligence to get there.

每个用户心里都有一个延迟预算，下面这些技巧能让 agent 保持在预算之内，同时不以牺牲模型智能为代价。

[¶22]

Minimizing task completion latency

Task completion latency is the sum, over model turns, of time to last token plus tool processing. That gives you three levers to work towards: fewer turns, faster tools, and faster tokens. These levers sometimes compete, so the thing to minimize is the sum rather than any one of them.

• Fewer turns · Load likely context up front, increase model intelligence, and have the model call independent tools in parallel.

• Faster tools · Optimize the tool's own backend, and dispatch tools eagerly as their arguments complete.

• Faster tokens · Choose the model and its configuration by sweeping your eval suite.

最小化任务完成延迟

任务完成延迟，是把每个模型轮次的末 token 耗时（time to last token）与工具处理耗时相加，再对所有轮次求和。由此你有三个可以着力的杠杆：更少的轮次、更快的工具、更快的 token 输出。这几个杠杆有时会相互掣肘，所以要最小化的是总和，而不是其中任何一项。

• 更少的轮次 · 预先加载可能用到的上下文，提升模型智能，并让模型并行调用相互独立的工具。

• 更快的工具 · 优化工具自身的后端，并在每个工具的参数一生成完毕就立即派发执行。

• 更快的 token 输出 · 在你的 eval 套件上对候选模型与配置逐一扫测（sweep），据此选定模型及其配置。

[¶23]

Fewer turns

Query complexity adds turns, and is generally out of your control. Model intelligence and relevant context help the agent get to task completion in fewer turns. Some of our key learnings in this area include:

• Load likely context up front. If the user opened the assistant from a product page, or a merchant opened it from a campaign dashboard, put that page's data in the session context. The conversation is likely about it, and answering from context costs no extra turns.

• Increase model intelligence. Smarter models can decrease overall turns in the completion of a task as the agent can more efficiently plan and issue its tool calls. That often outweighs their slower tokens. If your queries skew complex, or production shows more than about five turns per task, the faster model is frequently the smarter one. Which one that is depends on your traffic, so choose by sweep, as described under "Choosing the model" below.

更少的轮次

查询的复杂度会增加轮次，而这通常不在你的掌控之中。模型智能与相关上下文，则能帮助 agent 用更少的轮次完成任务。我们在这方面的一些关键心得包括：

• 预先加载可能用到的上下文。如果用户是从某个商品页打开助手的，或者商家是从某个营销活动看板打开的，就把该页面的数据放进会话上下文。对话多半就围绕它展开，而直接依据上下文作答，不会多花任何轮次。

• 提升模型智能。更聪明的模型能减少完成一个任务所需的总轮次，因为 agent 能更高效地规划并发出工具调用。这往往足以抵消它们 token 生成较慢的劣势。如果你的查询偏复杂，或者生产环境中每个任务要跑大约五轮以上，那么真正更快的模型往往恰恰是更聪明的那个。具体是哪一个取决于你的流量，所以要通过扫测来选，详见下文「选择模型」一节。

[¶24]

• Have the model call independent tools in parallel. Commerce use cases often require many operations in parallel: be it searching for multiple products, querying many policy docs, or fetching records from many sources of sales data. Parallel tool ensures multiple independent queries don’t burn additional turns. Prompt the model to call many tools within a turn and return the results in one user message as an array of tool results (see the parallel tool use docs).

• 让模型并行调用相互独立的工具。商务场景常常需要并行执行许多操作：无论是搜索多个商品、查询多份政策文档，还是从多个销售数据源拉取记录。并行工具调用能确保多个相互独立的查询不会额外消耗轮次。在 prompt 中引导模型在一轮内调用多个工具，再把各工具的结果作为一个工具结果数组，放在同一条 user 消息里返回（参见并行工具调用文档）。

[¶25]

Faster tools

• Optimize the tool's own backend. Sometimes a tool genuinely fans out – a merchant agent with a  "get today's snapshot" query reads sales, inventory, and campaign status in three independent calls. But we often see the tool boundary become the place where missing backend logic gets stitched together: an availability check that calls the catalog for the SKU, the inventory service per store, and the fulfillment service for cutoffs, then applies substitution rules and pickup eligibility in the tool's own code before answering. That tool is now overloaded with domain knowledge, hard to keep correct as the rules change, and is carrying logic that should sit in an upstream system. When you find yourself writing that logic in a tool, the fix is one backend endpoint that answers the question, and calling that with an agent tool.

更快的工具

• 优化工具自身的后端。有时一个工具确实需要扇出（fan out）——比如商家 agent 的「获取今日快照」查询，要通过三次彼此独立的调用，分别读取销售、库存和营销活动状态。但我们常常看到，工具边界变成了拼凑缺失后端逻辑的地方：一个可售性检查，要调用商品目录查 SKU、按门店逐一调用库存服务、再调用履约服务查截单时间，然后在工具自己的代码里套用商品替换规则和自提资格判断，最后才给出答复。这样的工具被塞满了领域知识，规则一变就很难保证正确，而且承载着本该放在上游系统里的逻辑。当你发现自己在工具里写这类逻辑时，正确的解法是建一个能直接回答这个问题的后端接口，再用一个 agent 工具去调用它。

[¶26]

• Dispatch tools eagerly. Tool arguments stream out of the model like any other tokens, so the harness can execute each tool’s call as its arguments complete and process it while the model is still streaming other, parallel tools or content blocks. We've seen this take multi-second gaps down to a few hundred milliseconds, and the Claude Agent SDK does it by default. You should prompt the model to emit its slowest call first for maximum latency gains.

• 尽早派发工具调用。工具参数和其他 token 一样，是从模型中流式输出的，因此 harness 可以在每个工具的参数一生成完毕时就执行该调用，并在模型仍在流式输出其他并行工具调用或内容块的同时处理它。我们看到，这能把好几秒的空档缩短到几百毫秒，而 Claude Agent SDK 默认就是这样做的。你应该在 prompt 中要求模型先发出最慢的那个调用，以获得最大的延迟收益。

[¶27]

Perceived latency

Perceived latency is the time a user feels until the screen does something. It’s especially critical in consumer-facing use cases where any transaction friction impacts checkout rates and revenue. Two techniques shorten it without touching the model:

• Stream components as they form. A rendered commerce response is typically 500–700 output tokens, which without streaming is five or more seconds of a spinner. Send each parameter of a presentation tool to the client as it streams and render the page progressively.

• Show the work. While the agent is gathering context, render a short progress line for each step in plain language (for example, "finding hotels near the water"). You can build it from the tool's existing arguments (such as the query for a product search), or add an additional user_facing_message parameter tool that prompts the model to write the line.

感知延迟

感知延迟是指用户感觉到的、直到屏幕上有所反应之前的等待时间。在面向消费者的场景中，它尤其关键，因为任何交易摩擦都会影响结账转化率和收入。有两种技巧可以在不改动模型的前提下缩短它：

• 组件边生成边流式输出。一条渲染出来的商务回复通常有 500–700 个输出 token，如果不做流式输出，用户就要对着加载图标干等五秒甚至更久。把 presentation tool 的每个参数在流式生成的同时发送给客户端，渐进式地渲染页面。

• 把工作过程展示出来。在 agent 收集上下文期间，用通俗的语言为每一步渲染一行简短的进度提示（例如「正在查找水边的酒店」）。你可以用工具已有的参数来生成这一行（比如商品搜索的查询词），也可以给工具额外加一个 user_facing_message 参数，让模型来写这行提示。

[¶28]

The two panels above run the same agent with the same tools and prompt; only the harness differs. Total time is about the same, but the time the user sees something is quite different.

上面两个面板运行的是同一个 agent，工具和 prompt 也完全相同，只有 harness 不同。两者的总耗时差不多，但用户看到内容出现的时间点却大不相同。

[¶29]

Prompt caching

Prompt caching is your largest cost reduction candidate and commerce traffic is well-suited for it. Cached input token reads cost a tenth of fresh ones, and while cache-writes carry a premium of roughly 1.25x, a cached prefix pays for itself on its second use. In customer facing applications where volume is large, you have a unique opportunity to hit very high cache levels using the cheapest, default 5 minute cache expiration.

The best commerce deployments we've seen run at 90–99% cache hit rates, and that is the range to design for from the start. Our experience has shown cached token reads are also around 1.5 to 2x faster at ~100k tokens, with relatively linear scaling the more tokens there are.

Prompt caching

Prompt caching（提示词缓存）是你最有潜力的降本手段，而商务场景的流量恰好非常适合它。从缓存读取输入 token 的费用只有全新输入的十分之一；缓存写入虽有溢价（约为基础价格的 1.25 倍），但一段缓存前缀在第二次被使用时就能回本。面向消费者的应用流量大，这给了你一个独特的机会：只用最便宜的、默认 5 分钟的缓存有效期，就能达到非常高的缓存命中水平。

我们见过的最出色的商务部署，cache hit rate（缓存命中率）能达到 90%–99%，这也是你从一开始就应当瞄准的设计区间。我们的经验还表明，在约 100k token 的规模下，读取缓存 token 的速度也要快 1.5 到 2 倍左右，而且 token 越多，这一优势大致呈线性增长。

[¶30]

Caching is prefix-based. A request reads from cache up to the first byte that differs from a previous request, so what matters is not just what is in the context but the order it is in. Think of a request as three segments, ordered by how often they change:

• Global: most of the system prompt and tool definitions, identical across every session. This is your warmest cache and, at scale, will likely not expire. Keep it byte-identical across turns and sessions and put a cache breakpoint at its end.

• Session: per-user context and conversation history, which differ across sessions but stay stable within one. This segment comes after the global one.

缓存是基于前缀的。一次请求能从缓存读取的内容，止于它与此前请求第一个不相同的字节；所以要紧的不只是上下文里有什么，还有这些内容的排列顺序。可以把一次请求看作三个分段，按变化频率从低到高排列：

• Global：大部分 system prompt 和工具定义，在所有会话中都完全相同。这是你最「热」的缓存，在规模化流量下很可能根本不会过期。让它在各轮次、各会话之间保持逐字节一致，并在它的末尾设置一个 cache breakpoint（缓存断点）。

• Session：每个用户各自的上下文和对话历史，不同会话之间各不相同，但在同一会话内保持稳定。这一段排在 Global 段之后。

[¶31]

• Volatile: anything that changes within a session, such as the current time or the current page. Put it at the very end of the request, either as a tagged block in the newest user turn or, on models that support mid-conversation system messages, as a system-role message appended to the messages array. The most common mistake we see is a timestamp or the current page at the top of the system prompt, which silently breaks the cache on every request.

There are two implementation details to remember here. First, skills should be loaded as tool results rather than appended to the system prompt. The skill body then lands in the conversation prefix and is cached along with it.

• Volatile：会话内会变化的一切，比如当前时间或当前页面。把它放在请求的最末尾：要么作为最新一轮 user 消息中一个带标签的块，要么在支持对话中途 system 消息的模型上，作为一条 system 角色的消息追加到 messages 数组末尾。我们见过最常见的错误，是把时间戳或当前页面放在 system prompt 的开头，这会在每一次请求时悄无声息地让缓存失效。

这里有两个实现细节需要记住。第一，skill 应该以工具结果的形式加载，而不是追加到 system prompt 中。这样 skill 正文就会落进对话前缀里，随前缀一起被缓存。

[¶32]

Second, roll your breakpoints forward in each turn: a request allows a limited number of breakpoints, so move the newest one to the end of each user turn. Each round then reads the accumulated history, including long tool results such as search responses, from cache.

第二，每一轮都要把 breakpoint 向前推进：一次请求允许设置的 breakpoint 数量有限，所以要把最新的那个移到每一轮 user 消息的末尾。这样一来，每一轮都能从缓存中读取累积下来的历史，包括搜索响应这类很长的工具结果。

[¶33]

Choosing the model and its configuration

Model size and the effort setting are the same tradeoff – intelligence against latency and cost – and you should choose both by measurement:

1. Pick your metric and your floor. Pick the quality metrics your business runs on (task completion, answer relevance, grounded accuracy), the eval score you won't go below, and your p50 and p99 latency and cost budgets.

2. Sweep. Run your entire eval suite across every model and effort level you'd consider. We recommend starting at Opus for merchant agents, whose tasks are analysis-heavy, and Sonnet for consumer agents, where latency weighs more. If you have production traffic, weigh the results by your real query mix. Then let the numbers decide. Sometimes Opus 5's lift on cart-driving tasks justifies the cost difference over Sonnet, and sometimes it doesn't.

选择模型及其配置

模型大小与 effort（推理力度）设置是同一种权衡——智能与延迟、成本此消彼长——两者都应该通过实测来选择：

1. 先定指标和底线。选定你的业务赖以运转的质量指标（任务完成率、回答相关性、有据可依的准确性）、eval 分数不能跌破的底线，以及 p50 和 p99（中位数与 99 分位）的延迟与成本预算。

2. 全面扫测。在你会考虑的每一个模型和 effort 档位上，把完整的 eval 套件都跑一遍。我们建议：商家 agent 的任务偏重分析，从 Opus 起步；面向消费者的 agent 更看重延迟，从 Sonnet 起步。如果已有生产流量，就按真实的查询构成对结果加权。然后让数字来做决定。有时，Opus 5 在促成加购的任务上带来的提升，足以抵消它比 Sonnet 高出的成本；有时则不然。

[¶34]

3. Read the results carefully. Two things regularly surprise teams. The first is that a prompt is tuned to a model, so a sweep run with one prompt may underperform other models that it wasn't written for. A smaller model usually needs instructions the current model infers on its own, and a larger one will follow instructions to the letter that the smaller one was ignoring. A few rounds of iteration on each candidate's failing cases is a cheap step before ruling any of them out. The second is that a more intelligent configuration sometimes wins on latency (most commonly on p90 and p99) despite slower tokens, because it plans its tool calls better and needs fewer rounds on the most complex requests.

3. 仔细解读结果。有两件事经常让团队感到意外。第一，prompt 是针对特定模型调校的，因此只用一份 prompt 跑扫测，那些 prompt 原本并非为之而写的模型，成绩可能会偏低。换成更小的模型，通常需要把当前模型能自行推断的内容写成明确的指令；换成更大的模型，则会一字不差地执行那些小模型一直在忽略的指令。在淘汰任何一个候选之前，先针对它的失败用例迭代几轮，这一步成本很低。第二，更智能的配置尽管 token 生成更慢，有时反而在延迟上胜出（最常见于 p90 和 p99），因为它能更好地规划工具调用，在最复杂的请求上需要的轮次更少。

[¶35]

Measure cost per completed task rather than per model call, since a cheaper model that needs more turns, or fails more often, is not cheaper. When the result is close, and the cost fits your per-task economics and latency, choose intelligence. Quality is what drives adoption and retention, and allows for room to build for the next 6 months as models become better.

衡量成本应按每个完成的任务来算，而不是按每次模型调用，因为一个更便宜的模型如果需要更多轮次，或者更常失败，其实并不便宜。当结果接近，且成本符合你的单任务经济账和延迟要求时，就选智能更高的那个。质量才是推动用户采用和留存的关键，也能在接下来 6 个月模型不断变强的过程中，为你继续往上构建留出空间。

[¶36]

03 · Running it in production

Memory, safety, evals, and scaling the work across an organization: what gets an agent through production and keeps it there.

Lastly, we talk about what gets an agent through production: memory, safety, evals, and scaling the work across an organization.

03 · 在生产环境中运行

记忆、安全、eval，以及把这项工作推广到整个组织：这些决定了 agent 能否顺利投入生产，并在生产环境中站稳脚跟。

最后，我们来谈谈 agent 要顺利投入生产需要些什么：记忆、安全、eval，以及把这项工作推广到整个组织。

[¶37]

Memory that survives the session

The relationship and interactions you have with your customers matter. Memory is what lets an agent pick up where the last conversation left off instead of starting from nothing. A shopper who mentioned a nut allergy in March shouldn't have to repeat it in June, and a merchant who checks the same three campaigns every Monday shouldn't have to name them each time. Long-term memory, the facts that should survive across sessions, is a system you build and it has three parts: how facts are stored, how they are written, and how they are read.

跨会话留存的记忆

你与客户之间的关系和互动很重要。记忆让 agent 能从上一次对话中断的地方接续下去，而不是从零开始。一位在 3 月提过自己坚果过敏的购物者，不该到了 6 月还得再说一遍；一位每周一都要查看同样三个营销活动的商家，也不该每次都得把它们点名一遍。长期记忆，即那些应当跨会话留存的事实，是一套需要你来构建的系统，它由三部分组成：事实如何存储、如何写入、如何读取。

[¶38]

Storing memories

Memory belongs in your systems, not in the model.

A flat markdown profile works when profiles are small and the agent is the only reader. Most production commerce agents outgrow it, and the practical replacement is the database you already operate. A fact is a small typed record: a key (such as shoe_size, default_store, preferred_report_cadence), a short value, a category, and the session it came from. Some keys you decide up front and every user gets; the rest the extractor discovers. A database stays queryable as the store grows, lets you build deterministic behavior on specific attributes, and joins to the user data you already have.

存储记忆

记忆应该放在你自己的系统里，而不是放在模型里。

在档案规模很小、且 agent 是唯一的读取方时，一份扁平的 markdown 档案（profile）就够用了。但大多数生产环境中的商务 agent 都会超出它的承载范围，务实的替代方案就是你已经在运维的数据库。一条事实就是一条小型的带类型记录：一个键（比如 shoe_size、default_store、preferred_report_cadence）、一个简短的值、一个类别，以及它所来自的会话。有些键是你预先定好、每个用户都有的；其余的由抽取器（extractor）自行发现。随着记忆库不断增长，数据库依然便于查询，让你能基于特定属性构建确定性的行为，还能与你已有的用户数据做关联（join）。

[¶39]

For merchant-facing agents, key memory by person rather than by account. Merchant logins are often shared between operators, so each operator needs their own profile, and reads have to respect that operator's permissions: a store manager's agent should not recall a fact a district manager stated.

In the commerce domain, agent memory holds personal data. The facts worth remembering are often the most regulated ones, and the rules between jurisdictions differ. Treat memory as a data-handling design problem and not just a storage one. In practice that means four things:

• Decide which types of memories you are willing to hold. Enforce that at the write path, with a validator that every save goes through, rather than in the prompt alone.

对于面向商家的 agent，记忆的键要按人来设，而不是按账号。商家的登录账号常由多名运营人员共用，因此每位运营人员都需要自己的档案，读取时也必须遵守该运营人员的权限：门店经理的 agent 不应该调出区域经理说过的事实。

在商务领域，agent 的记忆里存放的是个人数据。最值得记住的事实，往往恰恰是受监管最严的那些，而且不同司法辖区之间的规则各不相同。要把记忆当作数据处理层面的设计问题，而不只是存储问题。落到实践中，这意味着四件事：

• 决定你愿意保存哪些类型的记忆。要在写入路径上强制执行这一点：让每一次保存都经过一个校验器（validator），而不是只靠 prompt 来约束。

[¶40]

• Give users a way to see, correct, and delete what is stored. Wire deletion into your account-deletion and data-request flows.

• Set a retention period. A preference from a few years ago is likely to be outdated, so a retention period helps keep memory facts fresh.

• Memory should be a per-deployment switch. This allows regions that can't take on these obligations to run without it.

• 让用户能查看、更正和删除已存储的内容。把删除功能接入你的账号注销和数据请求流程。

• 设定保留期限。几年前的偏好很可能已经过时，设定保留期限有助于让记忆中的事实保持新鲜。

• 记忆应该能按部署单独开关。这样，无法承担这些义务的地区就可以在不启用记忆的情况下运行。

[¶41]

Writing memory

Write memory asynchronously. At the end of each turn, or every few turns in a long session, an agent in a separate thread or process reads the conversation and creates, updates, or deletes facts in the store, keeping its own working context as the session goes on.

It adds nothing to the conversation's latency, and achieved 13% higher fact recall on our internal commerce memory eval suite.

The obvious alternative, a tool the agent calls to save a fact, is the wrong one for a latency-sensitive commerce agent. Every save is a tool call inside a user-facing turn, and unless the whole store is in context, a save needs a read first to update or dedupe, which is a round of its own.

写入记忆

记忆要异步写入。在每一轮结束时（长会话中则每隔几轮），由一个运行在独立线程或进程中的 agent 读取对话，在记忆库中新建、更新或删除事实，并随着会话推进维护它自己的工作上下文。

这样做不会给对话增加任何延迟，而且在我们内部的商务记忆 eval 套件上，事实召回率高出了 13%。

显而易见的另一种做法，是给 agent 一个保存事实的工具、由它自己调用；但对延迟敏感的商务 agent 来说，这是错误的选择。每次保存都是面向用户的那一轮里的一次工具调用；而且除非整个记忆库都在上下文里，否则保存之前还得先读一次，才能更新或去重，而这本身又是额外的一轮。

[¶42]

It also puts one more decision in front of the agent on every turn, and in our evals that competition for attention showed up as missed memories.

Separating the extractor also lets you prompt it precisely. It reads only the user's and the assistant's text, never tool results, so a product description or a review can't become a fact about the user. Its prompt says what counts as a fact — a stated size, a dietary constraint, a fulfillment preference, a merchant’s usual materialized views — and what doesn't, such as anything from a listing or a one-off detail.

它还会让 agent 在每一轮都多面对一个决策；在我们的 eval 中，这种对注意力的争夺表现为漏记的记忆。

把抽取器单独拆出来，还能让你为它编写更精准的 prompt。它只读取用户和助手的文字，从不读取工具结果，因此商品描述或评论不会变成关于用户的事实。它的 prompt 会写明哪些算事实——用户明确说过的尺码、饮食禁忌、履约偏好、商家惯用的物化视图——以及哪些不算，比如任何来自商品页（listing）的内容，或一次性的细节。

[¶43]

Reading memory

Read memory in three layers.

• Always in context · A small fixed set of facts goes into context on every turn: the ones nearly every request depends on, such as a shopper's default store and fulfillment preference, or an operator's store and role.

• Pre-fetched per turn · Facts relevant to the current request are pre-fetched per turn from the same signals that pre-load a skill: a shoe search pulls sizes and brand preferences, a campaign question pulls the operator's usual metrics.

• Behind a lookup tool · Everything else sits behind a lookup tool.

Since memory is per-user context, all of it goes in the session segment, below the global cache breakpoint.

读取记忆

记忆分三层读取。

• 常驻上下文 · 一小组固定的事实每一轮都会放进上下文：也就是几乎每个请求都依赖的那些，比如购物者的默认门店和履约偏好，或者运营人员所在的门店和角色。

• 每轮预取 · 与当前请求相关的事实，会在每一轮根据预加载 skill 所用的同一批信号来预取：一次鞋类搜索会拉取尺码和品牌偏好，一个关于营销活动的问题会拉取该运营人员常看的指标。

• 放在查询工具背后 · 其余的一切都放在一个查询工具背后。

由于记忆属于每个用户各自的上下文，它全部放在 Session 段里，位于 Global 段末尾的 cache breakpoint 之后。

[¶44]

Safety: enforcement lives in the harness

The prompt is where safe behavior starts, but in commerce it can't be where safety is enforced. The failures are financial and often irreversible, and a prompt rule is one injection or one bad sample away from being skipped. Every rule below is enforced in code, on both the consumer and the merchant agent, and defined once so every runtime shares it.

安全：强制执行落在 harness 里

安全行为始于 prompt，但在商务场景中，prompt 不能成为强制执行安全的地方。这里的失败直接关乎资金，而且往往不可逆；一条写在 prompt 里的规则，离被跳过只差一次注入或一次糟糕的采样。下面的每一条规则都在代码中强制执行，同时适用于面向消费者的 agent 和商家 agent，并且只定义一次，让所有运行时共用。

[¶45]

The model stages; a person or a policy applies

No model tool call moves money or changes the business. Order placement, payments, refunds, price changes, and campaign launches all end in an action the harness controls instead of the model.

On the consumer side this is structural: the checkout tool renders the cart with a button to place the order, and the backend interface the agent calls has no charge method at all.

On the merchant side, every write tool produces a staged change with a server-generated ID, and apply_change succeeds only for IDs that have been approved through a real surface: a button in the operator's portal, a confirmation in the CLI, or the platform's own tool-approval prompt when the agent runs on Managed Agents.

模型只负责暂存，由人或策略来执行

模型发起的任何工具调用都不会让资金发生流动，也不会改变业务。下单、支付、退款、调价和上线营销活动，最终都落在一个由 harness（而非模型）控制的动作上。

在消费者一侧，这一点由结构本身保证：结账工具渲染出的购物车上带有一个下单按钮，而 agent 调用的后端接口根本没有扣款方法。

在商家一侧，每个写入类工具都只会产出一条暂存变更，并附带一个服务器生成的 ID；apply_change 只对已经通过真实界面获得批准的 ID 执行成功，这些界面包括：运营人员后台里的一个按钮、CLI 中的一次确认，或者当 agent 运行在 Managed Agents 上时，平台自带的工具审批确认。

[¶46]

The guardrails are re-checked at apply time against current limits, not the limits in force when the change was staged. Whatever the surface, the shape is the same: the model's most dangerous action is to propose, and the approval routes through the maker-checker flow your business already uses for that kind of change.

各项护栏（guardrails）会在执行时按当前的限额重新校验，而不是沿用变更暂存时生效的限额。无论通过哪种界面，基本形态都相同：模型所能采取的最危险的动作，也不过是提议；而审批要走你的业务针对这类变更本来就在使用的 maker-checker（经办-复核）双人复核流程。

[¶47]

Writes and renders accept only server-issued IDs

The harness keeps a per-session record of every ID the server has handed the model, and that record is the only key any write or render will accept.

The cart accepts only product IDs the server returned to this session, and the merchant tools accept only listing and campaign IDs the agent has actually read. An ID that arrived any other way — hallucinated, pasted by a user, planted in a review — is refused before the backend sees it.

The same rule covers the UI. Presentation tools take IDs, and the server fills in the product, order, or change records itself, so a card only renders records the server itself filled in.

写入与渲染只认服务器签发的 ID

harness 为每个会话维护一份记录，登记服务器交给过模型的每一个 ID；任何写入或渲染操作都只认这份记录。

购物车只接受服务器在本会话中返回过的商品 ID，商家侧工具只接受 agent 实际读取过的商品 listing 和营销活动的 ID。凡是经由其他途径出现的 ID——无论是模型幻觉编造的、用户粘贴进来的，还是被人埋在评价里的——都会在到达后端之前就被拒绝。

同一条规则也覆盖 UI。presentation tool 只接收 ID，商品、订单或变更记录都由服务器自行填充，因此卡片上渲染的只会是服务器亲自填入的记录。

[¶48]

It covers delegates too: the merchant analysis subagent reads data but never adds to the set of IDs the agent may write to.

For fees, disclosures, and other regulated content, the model chooses which product to disclose and the server supplies every word from approved copy. The same fee fields are on the merchant agent's protected list, so neither side of the counter can change or paraphrase them, and evals check the rendered strings byte for byte.

这条规则同样约束被委派方：商家侧负责分析的 subagent 可以读取数据，但绝不会往 agent 可写入的 ID 集合里添加新的 ID。

对于费用、信息披露以及其他受监管的内容，模型只负责选择披露哪个产品的信息，每一个字都由服务器从审核通过的文案中提供。这些费用字段同样列在商家 agent 的受保护清单上，因此买卖两侧的 agent 都无法修改或转述它们；eval 也会逐字节核对渲染出的字符串。

[¶49]

Capped transactions must hold to repeated requests

Most commerce surfaces cap how many of an item one user can buy — for ticket allocations, promotional pricing, or fraud control — and an agent will retry, rephrase, and parallelize in ways a human clicking a button never did.

The cap is therefore enforced on the line as it would be after the write, so a second "add two more" can't stack past it, and cart writes for one session are serialized so parallel tool calls in a single turn can't combine to exceed it.

Merchant changes are checked the same way against caps on price movement, discount depth, restock size, and campaign budget, plus a list of protected fields no change may touch. The rule generalizes: enforce every limit on the resulting state rather than the request, and serialize writes per session.

有上限的交易必须经得住重复请求

大多数商务平台都会限制单个用户购买同一商品的数量——无论是出于票务配额、促销定价还是反欺诈的考虑；而 agent 会重试、换个说法再请求、并行发起请求，这些方式是点按钮的人类用户从未用过的。

因此，上限要按写入之后该购物车行的数量来执行，这样第二次「再加两件」就无法叠加突破上限；同一会话的购物车写入也会被串行化，这样同一轮中并行的多个工具调用就无法合起来超出上限。

商家侧的变更也以同样的方式校验：价格变动幅度、折扣力度、补货数量和营销活动预算都有上限，另外还有一份任何变更都不得触碰的受保护字段清单。这条规则可以推而广之：每一项限制都要针对结果状态而非请求本身来执行，并按会话串行化写入。

[¶50]

Third-party content is sanitized

In commerce most of the context is written by people who aren't you — sellers, reviewers, competitors — so every backend read is untrusted input and goes through one sanitizer.

Every tool result authored by a third party, such as listings, reviews, policies, seller messages, and stored memory,  is sanitized and wrapped in a fence with a fixed label before the model sees it.

The sanitizer strips control and bidirectional characters, removes anything that imitates the fence markers, defuses text that imitates a conversation turn or a tool call, and caps the size, which is designed to stop a hostile listing from impersonating the system or filling the context.

第三方内容一律经过清洗

在商务场景中，大部分上下文都出自他人之手——卖家、评价者、竞争对手——因此每一次后端读取都属于不可信输入，都要经过同一个清洗器（sanitizer）。

所有由第三方撰写的工具结果，例如 listing、评价、政策条款、卖家消息以及已存储的记忆，在模型看到之前都会先经过清洗，再被包裹进一道带固定标签的围栏（fence）。

清洗器会剔除控制字符与双向（bidi）字符，删除任何模仿围栏标记的内容，对模仿对话轮次或工具调用的文本做无害化处理，并限制内容大小。这样设计是为了防止恶意 listing 冒充系统，或是塞满上下文。

[¶51]

The prompt carries the other half of the contract: fenced text is material to report on, never to act on.

契约的另一半由 prompt 承担：围栏内的文本只是供汇报的素材，绝不能据以采取行动。

[¶52]

Evals: shipping a non-deterministic system

Anything from a small prompt change to a new tool can change agent behavior in ways that are hard to predict, and the change you're shipping is often not the one that regresses. Evals are how you find that out before you deploy. Our earlier blog post on evals for agents covers the general practice. This section covers specifics for commerce agents.

Evaluate snapshots, not conversations

The model’s API is stateless, so what the agent outputs is a function of the system prompt, the tools, and the messages array. This means any state a commerce conversation can reach can be constructed directly. So creating an eval case means constructing the test state, appending the test user message, and letting the agent run from there.

Eval：发布一个非确定性系统

小到一处 prompt 改动，大到新增一个工具，都可能以难以预料的方式改变 agent 的行为，而且出现回归的地方往往不在你这次发布的改动之内。要在部署之前发现这些问题，靠的就是 eval。我们此前关于面向 agent 的 eval 的博文介绍了通用做法，本节专门讨论商务 agent 的具体要点。

评估快照，而不是对话

模型的 API 是无状态的，因此 agent 的输出完全取决于 system prompt、工具和 messages 数组。这意味着，商务对话能够到达的任何状态都可以被直接构造出来。所以，创建一个 eval 用例就是：构造出测试状态，追加一条测试用的用户消息，然后让 agent 从那里接着运行。

[¶53]

Then grade the outcome: the final state and the rendered response, including the arguments of the last write. In most cases, we recommend against grading the path the agent took to get there as such test cases are brittle and restricting.

Simulated-user evals, in which a second model plays the user and a judge grades the whole conversation, are a poor tool for measurement. Two non-deterministic systems interacting need larger samples, cost more per trial, are harder to judge, and produce failures that are hard to attribute. They are useful for finding coverage gaps and for a general vibe check on the agent, so use them to discover cases, then write each case as a snapshot.

然后对结果打分：也就是最终状态和渲染出的回复，包括最后一次写入的参数。在大多数情况下，我们不建议对 agent 达成结果所走的路径打分，因为这类测试用例既脆弱又设限过多。

模拟用户式 eval——由第二个模型扮演用户，再由一个评判模型（judge）给整段对话打分——并不是一种好的度量工具。两个非确定性系统相互作用，需要更大的样本量，每次试验成本更高，更难评判，产生的失败也难以归因。它们的用处在于发现覆盖盲区，以及对 agent 做一次整体的观感检查（vibe check）。因此，应当用它们来发现用例，再把每个用例写成快照。

[¶54]

Evaluate for behaviors in tough conditions

Most teams fail to properly test the injected state. A case should encode the preconditions of a failure, not just the task. If a behavior only emerges after a busy first turn with several tool calls, or after a contradiction earlier in the session, a case that starts from a clean state passes on every config and provides no meaningful data.

We've observed most suites to be heavy on such clean-state cases, so make sure a share of yours starts from long, messy, or contradictory histories.

评估艰难条件下的行为

大多数团队都没能充分测试注入的状态。一个用例应当把失败的前置条件编码进去，而不只是编码任务本身。如果某种行为只会在第一轮就密集调用了多个工具之后才出现，或者只会在会话早先出现过前后矛盾之后才出现，那么从干净状态起步的用例在任何配置下都会通过，提供不了任何有意义的数据。

我们观察到，大多数 eval 套件都偏重这类干净状态的用例，所以务必让你的套件中有一部分用例从冗长、杂乱或自相矛盾的历史开始。

[¶55]

Cover the different types of commerce agent evals

Effective evaluation requires testing both desired and undesired behaviors.

For every positive case, write its negative counterpart: a "should serve" for every "should refuse," a "should just do it" for every "should ask." Missing negatives are the most common gap we find in a suite.

Evaluate for the following:

• Core requests that make up the bulk of your traffic, since a failure here affects most sessions. These include simple lookups, multi-constraint requests, product and plan questions, and multi-intent messages. For the questions, check that every price, availability, and attribute traces back to returned data, and that the agent says when data is missing rather than inventing it.

覆盖各类商务 agent eval

有效的评估需要同时测试期望出现的行为和不应出现的行为。

每写一个正例，都要写一个与之对应的反例：有「应当拒绝」，就要有「应当正常响应」；有「应当先询问」，就要有「应当直接去做」。缺少反例是我们在 eval 套件中最常发现的缺口。

需要评估以下几类：

• 核心请求，即占你绝大部分流量的请求，这里一旦失败就会影响大多数会话。其中包括简单查询、多约束条件的请求、关于商品和套餐的问题，以及包含多个意图的消息。对于问题类请求，要检查每一个价格、可售状态和属性都能追溯到返回的数据，并且在数据缺失时 agent 会如实说明，而不是凭空编造。

[¶56]

• Context-dependent requests, such as references to what is on screen, constraints carried over from earlier turns, and writes against an existing cart. Evaluating memory falls into this bucket as well. Check that memories were extracted, retrieved, and changed the answer.

• Safety and brand cases, where a failure costs money or trust. These include attempted injection, attempts to read another user's data, and regulated language, which is checked byte for byte. Split injection into two cases: user-authored injection, where the directive comes from the user's own message, and data-plane injection, where it is planted in product names, reviews, or web snippets that arrive via tool results.

• 依赖上下文的请求，例如指代屏幕上正在显示的内容、沿用前几轮对话中的约束条件，以及对已有购物车进行写入。对记忆的评估也属于这一类。要检查记忆确实被提取、被检索到，并且改变了回答。

• 安全与品牌类用例，这类用例一旦失败，损失的是金钱或信任。其中包括注入尝试、试图读取其他用户数据的请求，以及受监管的措辞（需逐字节核对）。要把注入拆成两类用例：一类是用户撰写的注入，即指令来自用户自己的消息；另一类是数据面注入（data-plane injection），即指令被埋在经由工具结果传入的商品名称、评价或网页片段中。

[¶57]

• Interface evaluations, to ensure the right component is rendered, item caps are respected, and there are no internal identifiers in user-facing text. Test for timeouts and empty results too.

• Requests that belong to multiple capabilities at once. An operator asks "if I mark this down 15%, do I have enough stock to cover the demand?" That is a pricing question and an inventory question together. The right answer stages the markdown with a stock projection attached; the wrong answers do one and skip the other. Evals written per capability won't catch this, because each grades only its own half. Write cases for the requests that need two neighboring capabilities together, and grade both halves of the answer.

• 界面评估，确保渲染出正确的组件、遵守条目数量上限，并且面向用户的文本中不出现内部标识符。超时和空结果的情况也要测试。

• 同时涉及多项能力的请求。一位运营人员问：「如果我把这款商品降价 15%，库存够不够满足需求？」这既是定价问题，也是库存问题。正确的回答是暂存这次降价，并附上库存预测；错误的回答则只做其中一件，漏掉另一件。按单项能力编写的 eval 发现不了这种问题，因为每个 eval 只给属于自己的那一半打分。应当为需要两项相邻能力协同处理的请求编写用例，并对回答的两半都打分。

[¶58]

Write evals with SMEs and use real incidents

Partner with the subject-matter experts who see the failures firsthand, such as team members in Product, Legal, Merchant Ops, Customer Care, and Category Management, to design test cases. Real failures make the best evals, and 50-100 eval cases per user flow is a good starting point.

Make sure to have a variety of cases, as outlined above. Production transcripts are a great stream for sourcing new cases, especially the tricky ones. Coding agents are good at generating additional cases and adversarial variants. The reference repository includes a Claude Code plugin with an eval-authoring skill built with our recommended approach.

与 SME 一起编写 eval，并利用真实事故

与在一线直接看到失败的领域专家（subject-matter expert，SME）合作设计测试用例，比如产品、法务、商家运营、客户服务和品类管理等团队的成员。真实发生过的失败是最好的 eval 素材，每条用户流程先准备 50–100 个 eval 用例是个不错的起点。

务必像上文所述那样，让用例覆盖多种类型。生产环境的对话记录是新用例源源不断的来源，尤其能提供那些棘手的用例。编程 agent 擅长生成更多用例和对抗性变体。参考仓库中包含一个 Claude Code plugin，内含一个按照我们推荐的方法构建、用于编写 eval 的 skill。

[¶59]

Shipping with a large organization

In a commerce enterprise the agent is built by many engineering teams. Search, checkout, pricing, marketing tech, customer care, and the catalog platform each own systems the agent depends on, each ships on its own cadence, and each will want to add or change a tool, a skill, or a prompt rule.

Unlike a service, an agent has no strict module boundary protecting the others: a change made by the pricing team shares a context window with checkout.

The tempting fix is to break the system into many subagents, one per business unit. As discussed in Part 1, we recommend against it for quality reasons. Instead, we outline the process for de-risking multi-team collaboration:

在大型组织中发布

在一家商务企业里，agent 由许多工程团队共同构建。搜索、结账、定价、营销技术、客户服务和商品目录平台，各自拥有 agent 所依赖的系统，各自按自己的节奏发布，也都会想要新增或修改某个工具、某个 skill 或某条 prompt 规则。

agent 不像服务那样有严格的模块边界把其他部分保护起来：定价团队做出的改动，与结账功能共享同一个上下文窗口。

一个诱人的解决办法是把系统拆成许多 subagent，每个业务单元一个。正如第一部分所讨论的，出于质量方面的考虑，我们不建议这样做。下面我们概述一套降低多团队协作风险的流程：

[¶60]

• Ownership follows the systems. Every skill and tool has a single owner team. For example, pricing owns the promotion tools and the pricing skill, care owns the order and returns tools and the customer-care skill. The shared prompt has a single platform-level owner for the common parts and domain owner for the domain-specific section.

• A change ships with its cases and CI runs a set chosen for it. A team contributing a skill also contributes its cases, including the negative cases and the boundary cases against neighboring skills. Running the full suite on every pull request is too slow and too expensive to survive, so build a CI set from it instead. That set will consist of a core set of cases with the highest-traffic requests and every safety case. On top of that, run the cases for whatever the change touched. For a skill, that means its own cases and its neighbors' boundary cases. For a tool, it is every case that calls it. For the shared prompt, it is the full eval suite since everything reads the system prompt. We recommend gating the pass rate over a few trials, and on cache hit rate and cost per turn. It is also a good practice to run the full suite nightly and before every release. Cross-team regressions are caught in these runs.

• 归属跟着系统走。每个 skill 和工具都只有一个负责团队。例如，定价团队负责促销工具和定价 skill，客服团队负责订单与退货工具以及 customer-care skill。共享 prompt 的通用部分归属于单一的平台级负责方，各领域专属的部分则归属于对应的领域负责方。

• 改动连同其用例一起提交，CI 运行专门为它挑选的用例集。贡献 skill 的团队也要一并贡献它的用例，包括反例，以及与相邻 skill 之间的边界用例。每个 pull request 都跑一遍完整套件既慢又贵，难以长期维持，所以应当从中构建一个 CI 用例集。这个用例集由一组核心用例构成，涵盖流量最高的请求和全部安全类用例。在此之上，再运行本次改动所涉及部分的用例：改的是 skill，就运行它自己的用例和相邻 skill 的边界用例；改的是工具，就运行所有调用它的用例；改的是共享 prompt，就要运行完整的 eval 套件，因为所有部分都会读取 system prompt。我们建议以多次试验下的通过率作为门禁，同时把 cache hit rate 和每轮成本也纳入门禁。此外，每晚以及每次发布前都跑一遍完整套件，也是很好的做法。跨团队的回归问题正是在这些运行中被发现的。

[¶61]

• The agent should also be inside the release calendar. It's one deployment unit, so a bad change reaches every user at once. Roll prompt and skill changes to a canary cohort first, keep a switch that turns off one skill without a deploy, and freeze the agent ahead of peak periods the same way you freeze other systems.

For the human side of this arrangement, see Building effective human-agent teams.

• 同样要把 agent 纳入发布日历。它是一个整体部署单元，一次糟糕的改动会同时波及所有用户。prompt 和 skill 的改动要先推送给 canary（金丝雀发布）用户群；保留一个无需重新部署就能关闭单个 skill 的开关；并像冻结其他系统那样，在高峰期来临之前冻结 agent。

关于这套安排中「人」的一面，请参阅打造高效的人与 agent 协作团队。

[¶62]

Looking ahead

Most of what this post describes is not about the model. The tools call systems you already run, the skills encode procedures you already follow, the evals are your product requirements doc written as tests, and the harness enforces policy you would enforce for any client. Models will keep improving, and when a better one ships, the architecture we describe adopts it as a config change with an eval sweep. Everything else keeps working.

It is also important to think about your roadmap for product surfaces. The architecture will outlast the chat panel. The same agent can work over voice, and it can proactively act on a fare drop before the user asks. For a team that already has the evals and the tools, those are presentation-layer projects. Further out, some of the traffic to your storefront will come from agents that shop on behalf of users. The same provenance, staging, and approval rules that keep your own agent in bounds are what will let you open your tools to those agents safely.

展望未来

本文描述的大部分内容其实与模型无关。工具调用的是你已经在运行的系统，skill 编码的是你本来就在遵循的流程，eval 就是你写成测试形式的产品需求文档，harness 执行的则是你对任何客户端都会执行的策略。模型会持续进步；当更好的模型发布时，本文描述的架构只需一次配置变更、外加一轮 eval 扫测就能用上它，其余一切照常运转。

同样重要的是思考产品界面的路线图。这套架构的生命周期将比聊天面板更长。同一个 agent 可以通过语音工作，也可以在用户开口之前就针对票价下降主动采取行动。对于已经具备 eval 和工具的团队来说，这些都只是展示层的项目。再往远看，你线上店铺的一部分流量将来自代表用户购物的 agent。那些让你自家 agent 守住边界的来源校验（provenance）、暂存和审批规则，正是让你能够安全地向这些 agent 开放工具的前提。

[¶63]

Commerce has always rewarded making the buying process as smooth as possible. Agents make that a lot easier. Check out the complete reference implementation, with both the consumer and the merchant agent and runnable examples for retail, travel, telecom, and entertainment.

在商业领域，把购买流程做得尽可能顺畅，一向会得到回报。而 agent 让这件事容易了许多。欢迎查看完整的参考实现，其中既包含面向消费者的 agent，也包含商家 agent，还有零售、旅行、电信和娱乐行业的可运行示例。

[¶64]

Acknowledgements

Written by Matthew Koen and Ali Shazal. Special thanks to Michael Segner, Rodrigo Olivares, Amandeep Khurana, Aiza Usman, John Lopus and others for their contributions.

致谢

本文由 Matthew Koen 和 Ali Shazal 撰写。特别感谢 Michael Segner、Rodrigo Olivares、Amandeep Khurana、Aiza Usman、John Lopus 等人的贡献。