Codex, Behind the Harness — Dominik Kundel, OpenAI
频道: AI Engineer
视频: https://www.youtube.com/watch?v=shRR1e2HXMk
原文语言: en
统计: 共 18 轮 · Dominik Kundel 16
[0:01]
[music]
[音乐]
[0:12] Dominik Kundel
Hi everyone. Uh we're going to start right on time because I'm going to speak basically at 2x. I'm sorry, I have a lot of content. I'm trying to get you out of here on time. I want to start with a quick raise of hands. So, how many of you have built your own agents or are currently building your own agents? Perfectly. You're the right audience for this. Um over the next 20 minutes, I want to talk to you about a couple of different things that we're doing in the Codex harness that hopefully you can learn to apply to your own use cases or even just use the Codex harness with this in um uh in your own projects or at bare minimum learn what happens when you actually use Codex. Uh since we're at AI Engineer World's Fair and we're actually on an energetic engineering track, I'm going to like stop bothering you with like how does an agent work? What is an agent? And instead I want to talk a bit more specifically about some key features that we have in the agent that I think are particularly interesting and are challenges you have to solve. And uh so let's cover it from the lens of like what actually happens when you send off a message. Uh also a quick reminder, if you're unaware, the Codex harness and everything I'm showing you is actually open source. Uh it's MIT it's Apache 2 license and the harness is written in Rust. So, feel free to either learn from it, ask Codex deeper questions about what I'm covering, uh or fork it and make it your own.
大家好。我们准时开始，因为我基本上得用 2 倍速讲。不好意思，我内容太多了，我想让大家能准时散场。开场我先请大家举个手：你们当中有多少人自己搭过 agent，或者正在搭自己的 agent？非常好，那你们就是这场演讲最对的听众。接下来的 20 分钟里，我想跟大家聊几件我们在 Codex harness 里做的事情，希望你们能学到、并用到自己的场景里，或者干脆就在自己的项目里直接用 Codex harness；再不济，至少也能搞清楚你真正用 Codex 的时候背后到底发生了什么。既然我们是在 AI Engineer World's Fair，而且是在一个Agentic Engineering 分会场，我就不再拿「agent 是怎么工作的」「什么是 agent」这类问题来烦你们了，而是想更具体地讲讲我们这个 agent 里几个我认为特别有意思的关键特性——它们也都是你必须去解决的挑战。我们就从「你发出一条消息之后到底发生了什么」这个视角来讲。另外提醒一句，如果你还不知道的话：Codex harness 以及我今天展示的一切其实都是开源的，它是 MIT——准确说是 Apache 2 许可证，而且这个 harness 是用 Rust 写的。所以你完全可以拿它来学习，也可以就我今天讲的内容去问 Codex 更深入的问题，或者 fork 一份改成你自己的。
[1:34] Dominik Kundel
Also disclaimer, before we dive deep into it, this is a current state of affairs. Like things change so quickly. Um you can always refer back to asking Codex what the current state is, but especially with new model releases, we often release new APIs and change sort of how the harness works. Uh so, feel free to follow along as new models come out. If we want to talk about how the Codex agent works, we first need to talk about what actually happens when you send off your message. Um namely there's two protocols that are involved with uh code uh with the Codex agent. The first one is what happens when you send it off in the UI and it goes to the harness. We call that the app server. I talked about that yesterday, so we're not going to spend too much time about it. There will be a talk online that you can follow along. Um the second part is the responses API, which is handles the communication between the harness and the inference. Both of these though are designed for an open ecosystem, meaning if you're building your own UI, you're building your own uh agent interface, you can actually build on top of the Codex uh harness using the app server protocol. We use that same app server to power the Codex app. So it has really all of that functionality uh that you might expect from uh Codex, as well as like we have a lot of third-party community projects that build on top of it, including Theos T3 code or uh Remote X for example.
另外，在深入之前先声明一下：这些都是当下的状态。东西变化太快了。你随时可以回过头去问 Codex 现在的状态是什么样，尤其是每次有新模型发布，我们经常会发布新的 API，也会改 harness 的工作方式。所以新模型出来的时候，欢迎持续跟进。如果我们要讲 Codex agent 是怎么工作的，那得先讲你把消息发出去之后到底发生了什么。具体来说，Codex agent 涉及两个协议。第一个是：你在 UI 里把消息发出去、消息进到 harness 这一段，我们把它叫做 app server。我昨天讲过这块，所以今天不会花太多时间；之后会有演讲视频放到网上，你可以去看。第二部分是 responses API，它负责 harness 和推理（inference）之间的通信。不过这两者都是为开放生态设计的，意思是如果你在做自己的 UI、做自己的 agent 界面，你完全可以用 app server 协议、在 Codex harness 之上去搭。我们自己就是用同一套 app server 来驱动 Codex app 的，所以你对 Codex 期待的那些功能它其实全都有。除此之外，还有很多第三方社区项目建在它之上，比如 Theos T3 code，或者 Remote X。
[2:55] Dominik Kundel
I even used that same app server to put Codex into Cloud code. So if you're uh Cloud code user and you want to leverage uh Codex, you can use that plugin. And if you joined my talk yesterday, you saw me using that same protocol to actually put Codex into Doom. Um which is was a fun adventure as well. I mentioned the other part is the responses API. So responses API was released last year as like a rethinking of the chat completions API in a more agentic world, meaning um we redesigned the slightly the structure, but more importantly we added a lot of like building capabilities that are important for agents like web search, image gen, or other other more complex capabilities that you will see as part of this talk. We also want to make sure that this is like an open ecosystem, so we worked with a lot of partners including Ollama, LM Studio, Nvidia, and others to uh codify an open responses schema and have a governance body for that, so that other uh companies and and platforms can actually build on that same responses API, and you can use any responses API compatible um harness model provider and actually plug it into the Codex harness. So, that's an overview of how these protocols work. We're going from the UI to the harness with the app server protocol, and then from the harness to the LM inference using responses.
我甚至用同一套 app server 把 Codex 塞进了 Claude Code 里。所以如果你是 Claude Code 用户，又想用上 Codex，你可以用那个插件。如果你昨天来听了我的演讲，你还看到我用同一套协议把 Codex 塞进了《Doom》里，那也是一次挺好玩的冒险。我前面提到另一部分是 responses API。responses API 是去年发布的，可以理解为在一个更 agentic 的世界里对 chat completions API 的一次重新思考：我们把结构稍微重新设计了一下，但更重要的是，我们加了很多对 agent 很关键的能力，比如 web search、图像生成（image gen），以及其他一些更复杂的能力——这些你在这场演讲里都会看到。我们也希望它是一个开放生态，所以我们和很多合作伙伴一起，包括 Ollama、LM Studio、Nvidia 等等，把一套开放的 responses schema 规范化下来，并且为它成立了一个治理机构，这样其他公司和平台也能基于同一套 responses API 去建东西；而你可以拿任何兼容 responses API 的模型提供方，直接插到 Codex harness 上去用。这就是这两个协议的整体情况：从 UI 到 harness 走 app server 协议，从 harness 到 LM 推理走 responses。
[4:18] Dominik Kundel
But, what happens in the actual harness? The first step, arguably one of the most important ones, is context construction. And during that, we care about three things quite a lot. The first one is size. We want to make sure that, you know, we don't blast through your token budgets and throw in a bunch of unnecessary content, but also the more context you have in your in your context, the higher it is that you have contradicting information and it causes confusion for the model. The other part is flexibility. We want to make sure that, regardless of how many or how little skills you're using, you have a great experience regardless of how many plugins and MCPs you install. And of course, we want to make sure that things are performing and cost we know you're cost sensitive, so cashability is important as well. To show you this and a couple of other things, I actually built this little nano Codex here, um which functions the same way. It's built on built using the same code that is on the public repo, just turned into TypeScript. But, we can see here when we send over a message, we have a couple of different parts of of the actual context that gets assembled. And some of these are fairly standard and predictable, like the model instructions. Again, these are open source if you actually want to read them.
那么在 harness 内部到底发生了什么？第一步——可以说也是最重要的步骤之一——是上下文构造（context construction）。在这件事上我们非常在意三点。第一点是大小。我们要保证不会一下子把你的 token 预算打爆、往里塞一堆没必要的内容；而且，你上下文里的内容越多，出现互相矛盾的信息的概率就越高，就会让模型犯迷糊。第二点是灵活性。我们要保证不管你用的技能（skill）是多是少，体验都得好；不管你装了多少插件和 MCP，也一样。当然，我们还要保证性能，以及成本——我们知道你对成本敏感，所以可缓存性（cacheability）同样重要。为了展示这些、以及后面的几件事，我实际上做了个小东西叫 nano Codex，它的运作方式跟真的一模一样。它是用公开仓库里的同一份代码搭的，只不过改写成了 TypeScript。我们在这里可以看到，当我们发出一条消息时，真正被组装起来的上下文有好几个不同的部分。其中有些相当标准、也很可预测，比如模型指令（model instructions）。再说一次，这些都是开源的，你想读的话可以去读。
[5:37] Dominik Kundel
But, these are fairly structured and don't really change in size or like mess around with cashability. But, there are some parts that are harder to predict. Namely, for example, how many skills do you have available? Or the tool registry, where especially if you install MCPs, for example, you might have additional context that is like growing as you're installing more MCPs. For that reason, we do two things to maintain cost. The first one is or the size of the context. The first one is having deferred tools. So, we're marking some of these tools as deferred, and that means that they're not added directly to the context window, but instead are available through tool search later on. Um and then the other part is that for available skills, we actually cap the available skills list at 2% of your context total like maximum context window. And that means that if it gets longer, we're reducing slowly the amount of like description that we're putting in there. Tool search specifically is actually something that is available in the responses API. So, even if you're building your own harness, you can leverage this. Since GPT-5.4, you can mark any tool as deferred loading. Um and that means that these tools are only available if you're using tool search, and then you can give the model either our built-in tool search tool or implement your own if you feel like you can better do that discovery yourself.
但这些部分结构相当固定，大小基本不变，也不会把可缓存性搞乱。不过也有一些部分是比较难预测的。比如说，你手上有多少个可用的技能？再比如工具注册表（tool registry）——尤其是你装了 MCP 之后，你可能会有额外的上下文，而且它会随着你装的 MCP 越来越多而不断膨胀。出于这个原因，我们做了两件事来控制成本——或者说控制上下文的大小。第一件是 deferred tools（延迟加载的工具）。我们把其中一部分工具标记为 deferred，意思是它们不会直接被加到上下文窗口里，而是之后通过 tool search 才能拿到。另一件是，对于可用技能列表，我们实际上把它的上限卡在你总上下文——也就是最大上下文窗口——的 2%。也就是说，如果它变得更长，我们就会慢慢削减放进去的描述文字的量。tool search 这件事本身其实在 responses API 里就有，所以哪怕你是在做自己的 harness，也可以直接用上它。从 GPT-5.4 开始，你可以把任何工具标记为 deferred loading（延迟加载）。这意味着这些工具只有在你用 tool search 的时候才可用；然后你可以把我们内置的 tool search 工具给模型，也可以自己实现一个——如果你觉得那套发现逻辑你自己能做得更好的话。
[7:01] Dominik Kundel
Great. We talked about how we're building the context, but an agent really only becomes an agent if it performs actions. And there's three common actions I wanted to talk about. The first one is async actions, things that are happening while the agent has to continue to do work, computer use, and then the file system. For async actions, a good example is sub-agents, where we want to be able to delegate tasks off and then have the main agent continue to do work if necessary. The way that works in practice is that we give the agent a spawn agent tool, which then allows the agent to create new new agent instances, and then use a send input tool to either send new content to that to those new newly created agents, wait for an agent, or shut it back down. Um we use that same concept actually for background terminals as well. So, the Codex agent has a tool to spin up a new uh background terminal and then continuously interact with it by sending new data through standard in to that new agent or wait for a specific amount of time for that agent to finish a task. Computer use is an interesting one because we actually introduced uh computer use in the responses API last year and it was fairly limited. It only allowed you to do one action at a time and um you had to basically declare that you wanted it to do computer use and from there you were up to actually implementing specifically the type of actions that were exposed to that tool.
好。我们讲了上下文是怎么构建的，但一个 agent 真正成为 agent，是因为它会执行动作。有三类常见动作我想讲一讲。第一类是异步动作（async actions），也就是那些在 agent 必须继续干活的同时还在跑着的事情；第二类是 computer use；第三类是文件系统。异步动作的一个好例子是 sub-agent（子 agent）：我们希望能把任务派发出去，同时主 agent 在需要的时候还能继续干活。实际的做法是，我们给 agent 一个 spawn agent 工具，让它可以创建新的 agent 实例；再给一个 send input 工具，用来给这些新建出来的 agent 发送新内容、等待某个 agent，或者把它关掉。同样的思路，我们其实也用在了后台终端（background terminal）上。Codex agent 有一个工具可以起一个新的后台终端，然后持续跟它交互——通过标准输入（standard in）往里送新数据，或者等待指定的一段时间，让它把某个任务干完。computer use 是个挺有意思的东西。我们其实是去年在 responses API 里引入 computer use 的，当时它相当受限：一次只允许你做一个动作，而且你基本上得先声明你要用 computer use，然后从那儿开始，具体暴露给这个工具的都是哪些类型的动作，就得你自己去实现了。
[8:35] Dominik Kundel
This was uh great in terms of like that state uh that point in the journey of of building agents, but since then we've evolved it and uh the recent models and the recent API shapes allow you to use code execution instead to actually do computer use, which means that the agent can script its own interactions with the uh with whatever computer implementation you want to have. You can choose the language like JavaScript or Python and you are you have a much more flexible harness. In fact, that's what we use for browser use. Um so, the what happens when uh Codex uses browser use is it actually interacts with a persistent node repl that gets persisted throughout different uh throughout the turns. And then it writes JavaScript um essentially Playwright code to interact with that uh browser instance in the node repl. Um so, here's an example where on the right side we have a Chromium browser and the first time um it writes some code to get the overall status and like pulls up the right tab. And then on subsequent terms, we can see here that it is able to like reference those new tabs, pull in information, and script the respective actions um to understand what actually has to be done. And so that may speeds up these actions significantly because uh Codex and browser use can actually look at for example one page, understand the structure, and then write a script to perform like scraping for example on subsequent pages more easily.
在 agent 发展的那个阶段、那个节点上，这已经很不错了。但从那以后我们又往前推了一步：最近的模型和最近的 API 形态，允许你改用代码执行（code execution）来做 computer use，也就是说 agent 可以自己写脚本，去跟你想接的任何 computer 实现打交道。你可以选语言，比如 JavaScript 或者 Python，这样你拿到的 harness 就灵活得多。事实上，我们的 browser use 就是这么做的。所以 Codex 用 browser use 的时候，它其实是在跟一个常驻的 node REPL 交互，这个 REPL 会跨多个回合（turn）一直保持存在；然后它写 JavaScript——本质上就是 Playwright 代码——去操作 node REPL 里的那个浏览器实例。这里有个例子：右边是一个 Chromium 浏览器，第一次它先写了一段代码去拿到整体状态，把正确的那个标签页调出来。然后在后续的回合里，我们能看到它可以引用这些新的标签页、把信息拉进来，并且写出相应的动作脚本，去搞清楚到底该做什么。这样一来，这些动作的速度就快了很多，因为 Codex 和 browser use 可以先看一个页面、理解它的结构，然后写一个脚本，在后续的页面上更轻松地执行比如抓取（scraping）这类操作。
[10:08] Dominik Kundel
The third thing is file system interactions. I think if you're using Codex or any coding agent, of course you want it to be able to interact with the file system. In the case of um our models, all of the recent models starting with GPT-5 have been trained on the concept of an apply patch tool to do file editing, which means that they're used to using that to change files by giving it a a diff, and then um also using that same thing to create new files. And then for everything else, it uses a shell tool where it will uh do a file search or other file system navigation. Uh you will see the model naturally trying to use Ripgrep um since that's what what it got used to during training. So, we're actually in the Codex harness shipping Ripgrep with uh with the harness if you don't have it installed on your own. On Windows, we also trained the model to use PowerShell natively. So, if you're running it on Windows, you'll see it uh start writing uh PowerShell code instead. One of the things you're seeing here is um the sandbox pulling in. So, all of the interactions that you see with the file system actually go through our sandbox layer. Um and on macOS, we use Seatbelt for that, similar to most agents. And on Linux, we use Bubblewrap.
第三件事是文件系统交互。我想，如果你在用 Codex 或者任何编码 agent，你当然希望它能跟文件系统打交道。就我们的模型而言，从 GPT-5 开始的所有近期模型，都是围绕 apply patch 这个工具的概念训练出来做文件编辑的，也就是说它们习惯用它、通过给出一份 diff 来改文件，同样也用它来新建文件。除此之外的其他所有事情，它用的是 shell 工具——用来做文件搜索或者其他文件系统层面的操作。你会看到模型很自然地想去用 Ripgrep，因为那是它在训练过程中用惯了的。所以我们在 Codex harness 里其实是把 Ripgrep 一起打包发出去的，以防你自己机器上没装。在 Windows 上，我们还专门训练了模型原生使用 PowerShell，所以如果你在 Windows 上跑，你会看到它开始写 PowerShell 代码。你在这里看到的另一件事，是沙箱（sandbox）开始起作用了。你看到的所有跟文件系统的交互，实际上都会走我们的沙箱层。在 macOS 上我们用的是 Seatbelt，这跟大多数 agent 一样；在 Linux 上我们用的是 Bubblewrap。
[11:25] Dominik Kundel
On uh Windows, it's slightly different where we actually had to build our own custom um open open source Windows sandbox. It's in the same GitHub repository if you want to take a look. And there's a many reasons why we had to do this, and I could probably fill a whole talk about that. So, instead, I would recommend you to actually check out the check out the article that David wrote about this. Um it goes into all of the other Windows alternatives and why we actually had to build our own sandbox there. Highly fascinating if you really want to nerd out about Windows internals. With sandboxes though, uh one of the biggest complaints is always approval fatigue. Um how many of you get like annoyed by approvals, especially for long tasks and like have running full access? All right. And I don't think the audience gets filmed, so like we're going to do this again. Like how many of you know that like IT and security really hates that you turn on full access?
在 Windows 上情况稍有不同，我们实际上不得不自己造了一个开源的 Windows sandbox。它就在同一个 GitHub 仓库里，想看的话可以去翻。我们之所以非这么干不可有很多原因，光这一块我大概就能讲满一整场演讲。所以这里我建议你们去看 David 写的那篇文章，里面把 Windows 上其他所有替代方案、以及我们为什么最终还是得自己造一个 sandbox 都讲透了。如果你真想深挖 Windows 内部机制，那篇文章非常有意思。不过说到 sandbox，最大的抱怨之一永远是审批疲劳（approval fatigue）。你们当中有多少人被审批弹窗烦到过？尤其是跑长任务的时候，恨不得直接开全权访问（full access）？好的。而且我觉得观众席应该没被拍进镜头，所以我们再来一次：你们当中有多少人知道，IT 和安全团队其实特别讨厌你打开全权访问？
[12:19] Dominik Kundel
Yeah.
是啊。
[12:20] Dominik Kundel
Yeah, I know. Uh so, for that reason and uh you know, our own security team wanting to be happy as well, um we worked on um an auto approval mode. And the idea here is really to um derisk some of the like full access things because there are things that can go wrong. As much as these models have gotten better, especially if you're trying to push the model through prompting to really have high agency, that can be misinterpreted by the by the agent to not match up with your expectation like you asking a model to send out a file to through an email, pushing it to have high agency, and it realizes it can't attach the file, so it uploads it to a file share, um and like uses that instead, or messes up some escaping and accidentally deletes too much data. Those things can still happen even with the models getting better. And so, you want to still be careful with these um full access modes. And so, instead, we built auto review with this goal to cover all of the things that are harder to predict that an agent might do and especially in the long run, and um approve those automatically without having to escalate them to you while keeping things like uh data exfiltration outside of it. The way this works is when we um kick off a task that runs into a sandbox like deleting a file in this case, um the model when it tries to escalate spins up an auto review uh subagent.
是啊，我知道。所以出于这个原因，也因为我们自己的安全团队也想开心一点，我们做了一个自动审批模式（auto approval mode）。这里的想法是给全权访问这类东西降降风险，因为确实有些事会出岔子。尽管这些模型已经变强了很多，但尤其是当你试图通过 prompt 把模型往高自主性（high agency）上推的时候，agent 可能会把你的意图理解偏，跟你的预期对不上。比如你让模型通过邮件把一个文件发出去，同时又推着它高自主性，结果它发现自己没法把文件作为附件加上，于是就把文件传到某个文件共享上，改用那种方式；又或者转义（escaping）搞错了，一不小心删掉了太多数据。哪怕模型越来越好，这些事情还是可能发生。所以对这类全权访问模式，你仍然得小心。因此我们做了 auto review，目标是覆盖住那些更难预测的、agent 可能做出的动作，尤其是长时间运行的情况下，把这些动作自动放行、不用升级到你这里来确认，同时把数据外泄（data exfiltration）这类事情挡在外面。它的工作方式是：当我们启动一个任务、任务在 sandbox 里跑起来，比如这个例子里是删除一个文件，模型在试图申请提权（escalate）的时候，就会拉起一个 auto review 的 subagent。
[13:49] Dominik Kundel
And uh let's see. Didn't do it yet. There we go. So, it spins up an auto review subagent. And this subagent runs entirely separate and can't spin up other subagents and has read permissions only. Um but it we're giving it a bunch of context around sort of how what what is user authorization, which we'll see in a second. How do things like uh risk taxonomies work? How do we want it want it to judge these things? Um and then eventually we give it um the transcript as well as sort of the tool calls that are actually happening. And this is important because the context matters, right? Like in some cases you want the agent to actually delete a file. In other cases you don't. If you ask it to or if it like is part of the project, it makes sense. Especially like things like if you ask it to delete a dot git folder, great. If you didn't ask it to, it should probably not touch that part and like completely delete your history. Um and so we're giving it all of this context and ask it to then come up with um a judgment based on like the user authorization. In this case, it's high because we explicitly told it to delete the file. Um what is the impact of the deletion or like the action itself and what should we do? So, this could apply both to file system actions, but also to network calls for example where like curling Google to see if it if the internet works is fine, but maybe not uploading a file for example.
我们来看一下。还没触发。好，来了。所以它会拉起一个 auto review 的 subagent。这个 subagent 完全独立运行，不能再拉起别的 subagent，而且只有读权限。但我们会给它一堆上下文：什么算是用户授权（user authorization）——这个我们马上会看到；风险分类体系（risk taxonomy）是怎么运作的；我们希望它按什么标准来判断这些事情。然后我们最终会把对话记录（transcript）以及实际发生的那些工具调用（tool call）也一并给它。这一点很重要，因为上下文是关键，对吧？有些情况下你就是想让 agent 去删掉那个文件，另一些情况下你并不想。如果是你要求的，或者这本来就是项目的一部分，那就说得通。特别是像你让它删 .git 文件夹，那很好；但如果你没让它删，它大概就不该碰那块、把你的历史记录整个删掉。所以我们把这些上下文全都给它，然后让它基于用户授权给出一个判断。在这个例子里授权级别是高，因为是我们明确让它删这个文件的。再就是这次删除、或者说这个动作本身的影响是什么，以及我们该怎么处理。这套东西既适用于文件系统操作，也适用于网络请求——比如 curl 一下 Google 看看网通不通没问题，但上传一个文件可能就不行。
[15:18] Dominik Kundel
This is a gross oversimplification of all of the thoughts and uh work that went into this by engineers and and the research team. So, if you do want to learn more about it, check out the auto review um blog post that we wrote. The next thing I wanted to talk about is speed. Um because agents can do a lot of tool calls, and while we are doing a lot of work on speeding up inference, it's only part of the equation. We noticed this when we uh launched GPT 5.3 Codex Spark, uh and it's running on Cerebras at 1,000 tokens per second. So, with that, we realized that with all of these tool calls and the interactions, inference wasn't no longer was no longer the bottleneck. It was actually the network. So, for that reason, we introduced WebSocket mode, which means that the responses API doesn't run through service-side events and HTTP, but instead uses uses a persistent WebSocket connection, which allows us to save both on network overhead, but also provides stateful context, which means we only have to send the data that actually changed. So, for example, if there's a tool call, we only send back the result of the tool call rather than sending all of the items back. So, in practice, oh Great. In this case, my demo server crashed.
这只是对工程师们和研究团队在这上面投入的全部思考与工作的一个极度简化的说法。所以如果你确实想了解更多，去看我们写的那篇 auto review 博客文章。接下来我想讲的是速度。因为 agent 会做大量的工具调用，而虽然我们在推理（inference）提速上做了很多工作，但那只是方程式的一部分。我们是在推出 GPT 5.3 Codex Spark 的时候注意到这一点的，它跑在 Cerebras 上，每秒 1,000 个 token。有了这个速度我们才意识到，在这么多工具调用和交互之下，推理已经不再是瓶颈了，真正的瓶颈是网络。所以出于这个原因，我们引入了 WebSocket 模式，也就是说 responses API 不再走 server-sent events 和 HTTP，而是用一条持久的 WebSocket 连接。这样我们既能省掉网络开销，又能获得有状态的上下文（stateful context），意味着我们只需要发送真正变化了的那部分数据。举个例子，如果有一次工具调用，我们只把这次工具调用的结果发回去，而不是把所有 item 都发一遍。所以在实际使用中……哦，好极了。这次我的 demo 服务器崩了。
[16:36] Dominik Kundel
Um let me see if I have a backup here. Um Oh, spoiling the rest of the demo. Nope. Uh All right. Um in practice, you will see that it's going to send um one like it's going to send only one item after another. Oh, wait. Sorry, I pressed the wrong button for this one. There we go. Here's the backup. Um So, it's going to only send like one item after another, rather than sending like in this case, for example, like nine items back. Um while for the other one, we're only sending one back. And so, over time, this actually speeds up things quite significantly. Um and can have a pretty drastic impact on the performance. All right. We're at AI Engineer World's Fair in 2026. The only thing like I have to talk legally about is loops. So, we're briefly going to talk about it. Um specifically {slash} goal, because I got the questions a couple of times during this event. How does this actually work? Um and of course since the demo server crashed, which also shows that that was real data. We're going to show you the demo here. This is a hosted demo, by the way, so you'll be able to replay this afterwards. Um but basically what happens is in this case we're having a try to guess a number, and it's only achieving that goal when it's actually done with guessing that number.
我看看这儿有没有备份。哦，这把后面的 demo 剧透了。不是这个。好吧。在实际使用中你会看到，它会一个 item 一个 item 地发。哦等等，抱歉，这个我按错键了。好，备份在这儿。所以它只会一个 item 接一个 item 地发，而不是像这个例子里那样一次发回九个 item；而另一边我们只发回一个。所以时间一长，这确实能显著加快速度，对性能的影响可以相当剧烈。好。我们现在是在 2026 年的 AI Engineer World's Fair，我从法律上唯一必须讲的就是 loops，所以我们简单说两句。特别是 /goal，因为这次活动期间我被问到过好几次：这东西到底是怎么工作的？当然，既然 demo 服务器崩了——这也正好说明刚才那些是真实数据——我们这就把 demo 演示给你们看。顺便说一句，这是一个托管的 demo，所以之后你们可以自己回放。基本上这里发生的是：这个例子里我们让它去猜一个数字，只有当它真的把那个数字猜出来了，这个目标才算达成。
[18:07] Dominik Kundel
And while it until it's done with that, um it will actually automatically the harness will inject this continuation prompt. And this continuation prompt includes, among other things, your objective. That's the goal that you set. And then uh we continue to do this until the model itself calls an update plan update goal tool, which specifies that the plan was actually or the goal was actually achieved. So, that is the reason why you actually don't want to, you know, write full essays like I know a lot of you have been trying to um into your goal, but instead have very concrete and very fiable um prompts, so that uh it's easy to detect when things are done. The last thing I want to talk about is compaction. If we have these agents run for hours or days at a time, you don't want to stand there and actually approve everything all the time. Um and because of that we introduced uh end end of last year auto compaction, and this has been uh used by Codex since then to automatically trigger compaction on the server side in a way that the model got trained with so that the performance stays the same. Um the way this works is you can either trigger this manually or automatically and it will turn your transcription into or your your previous context window into a new one that you're going to use instead on subsequent turns and it contains a compaction item that has all of the necessary information in it that you need.
而在它完成之前，harness 会自动注入一个续跑 prompt（continuation prompt）。这个续跑 prompt 里包含的东西之一就是你的目标（objective），也就是你设定的那个 goal。然后我们会一直这样继续下去，直到模型自己调用 update plan、update goal 这个工具，声明这个计划、或者说这个目标确实达成了。所以这就是为什么你其实不该在 goal 里写长篇大论——我知道你们很多人一直在试着这么干——而是应该写非常具体、非常可验证的 prompt，这样才容易判断事情什么时候算做完了。最后我想讲的是压缩（compaction）。如果让这些 agent 一次连续跑上几个小时甚至几天，你肯定不想一直守在那儿、一遍遍地去批准所有东西。因为这个原因，我们在去年年底引入了自动压缩（auto compaction），从那以后 Codex 一直在用它，在服务端自动触发 compaction，而且触发的方式和模型训练时的方式一致，这样性能能保持不变。它的工作方式是：你可以手动触发，也可以自动触发，它会把你之前的上下文窗口（context window）变成一个新的，后续轮次就改用这个新的，而且这个新窗口里包含一个 compaction item，装着你需要的所有必要信息。
[19:37] Dominik Kundel
All right. To wrap things up, thank you for staying with me speed running through these slides. Um three things I want you to take away from this. Uh the first one is the Codex app server and the harness are open source. You can use this as a blueprint to learn how we built our agents or you can actually use it as the harness itself that you can build on top of. The other part is that most of the features that are stand out for Codex are actually features that are exposed in the responses API. So even if you want to build your own agent, you can leverage these things like tool search, apply patch, web sockets, or server side compaction. You can use those directly regardless of what harness you're using. And the last thing is as models evolve, keep an eye out on how we're evolving the responses API, how we're evolving Codex, and uh use that as a way to understand how you want to update your own agents to make make use of these new capabilities. And with that, thank you so much. That's the link to the slides and I'll be heading down to the booth if you have any questions.
好，最后收个尾。谢谢你们陪我一路速通完这些幻灯片。我希望你们从中带走三件事。第一，Codex 的 app server 和 harness 都是开源的。你可以把它当作蓝本，去了解我们是怎么造 agent 的；也可以直接把它当成 harness 本身，在上面搭你自己的东西。第二，Codex 上那些亮眼的特性，大部分其实都在 responses API 里开放出来了。所以即使你想造自己的 agent，你也照样能用上这些东西——比如 tool search、apply patch、WebSocket，或者服务端 compaction。不管你用的是哪个 harness，都可以直接用上它们。最后一点是，随着模型不断演进，请留意我们是怎么演进 responses API 的、怎么演进 Codex 的，把它当作一条线索，去想清楚你自己的 agent 该怎么更新，才能用上这些新能力。就讲到这里，非常感谢大家。这是幻灯片的链接，我等下会去展台，有问题可以来找我。
[20:52]
[music]
[音乐]