What is an AI harness? I build one live in less than 30 minutes
频道: How I AI
视频: https://www.youtube.com/watch?v=ofS-4RRw9zw
原文语言: en
统计: 共 13 轮 · Claire 13
[0:00] Claire
A harness is some code around an AI agent that makes it more effective. Why we've seen people build these specific use case harnesses is sometimes with the specific job, you just want to micromanage a little bit. You just want to be more prescriptive about how that job gets done. [music] I'm going to show you how it works and then we will talk about how I built it. So the interface I built for my harness is a terminal UI. The harness core is run on cloud agent SDK and then it's connected to real tools. So it's connected to [music] Sentry, Vercel, and then it's connected to linear and GitHub in terms of getting [music] tasks done. I think we all have done good work, but then now I've realized that these agents can help us solve very very specific problems by constraining that work. It's really like changed my mind about how work gets done.
所谓 harness，就是包在 AI agent 外面的一层代码，让它变得更好用。为什么大家都在给特定用例造这种专用 harness？因为面对某些特定的工作，你就是想稍微「微观管理」一下，想对这件事怎么做规定得更明确。[音乐] 我会先给你们演示它怎么运行，然后再聊聊我是怎么把它搭出来的。我给自己的 harness 做的界面是一个终端 UI。harness 的核心跑在 Claude Agent SDK 上，然后接上了真实的工具：接了 Sentry、Vercel，在推进任务方面还接了 Linear 和 GitHub。我觉得我们大家一直都干得不错，但现在我意识到，只要把工作范围约束好，这些 agent 能帮我们解决非常非常具体的问题。这真的彻底改变了我对「工作是怎么被完成的」这件事的看法。
[0:52] Claire
[music]
[音乐]
[0:54] Claire
Everybody's saying it's not the model, it's the harness. But you know what not everybody is saying? What is [music] a harness? In today's how I AI episode, I'm going to demystify the idea of a harness, write my own harness and show you how you can do the same, and explain to you why a custom harness makes sense and could be better than using Claude code or Codex alone. Let's get to it. This episode is brought to you by Bolt.new, the AI app builder for people who have ideas and want to ship them. Most AI tools spit out code that looks great in the demo and falls apart the second [music] you try to do anything real with it. Or they lock you into their own platform with no real way out. Bolt is different. [music] You describe what you want to build, a startup MVP, a landing page, an internal tool, a side project, and Bolt generates production-ready code in minutes. Connect Stripe or any other MCP, hook up your domain, and deploy it live. Founders are using Bolt to build businesses doing real [music] revenue. Product managers are shipping prototypes their teams actually use. Designers and marketers are launching campaigns without waiting in line. Anyone can build, engineering can ship, everyone [music] wins. You just need an idea and a weekend. Check it out at bolt.new/howiai. Before I get into how to build a harness, let's talk about what a harness is and I'm going to make it as simple as I can for all of you. A harness is some code around an AI agent. Yes, you heard it here first. A harness is just code around an AI agent that makes it more effective. Can that code have AI in it? Sure. Does that code have to have AI in it? Not necessarily. What is the goal of a harness? To make the AI better. It is so simple and I feel like the way that people have been talking about this have made it such a mystery that I wanted to make it just very clear to you all. It is just writing more code around your AI to make it more useful for a specific use case.
人人都在说「关键不在模型，而在 harness」。但你知道没人说什么吗？——harness 到底是什么？在今天这期 How I AI 里，我要把 harness 这个概念讲透，现场写一个我自己的 harness，教你怎么照着做，并且解释为什么定制 harness 是有道理的，甚至可能比单独用 Claude Code 或 Codex 更好。开始吧。本期节目由 Bolt.new 赞助——一个给「有想法、想发布」的人准备的 AI 应用构建器。大多数 AI 工具吐出来的代码在 demo 里看着很美，可你一旦想拿它干点真事就散架了；要么就是把你锁死在它们自己的平台里，根本没有退路。Bolt 不一样。你描述你想做什么——一个创业 MVP、一个落地页、一个内部工具、一个副业项目——Bolt 几分钟就能生成可上生产环境的代码。接上 Stripe 或任何其他 MCP，绑定你的域名，直接部署上线。有创始人正在用 Bolt 做出有真实营收的生意；产品经理在发布团队真正会用的原型；设计师和市场人员不用排队等资源就能上线活动。人人都能构建，工程能发布，皆大欢喜。你只需要一个想法和一个周末。去 bolt.new/howiai 看看吧。在讲怎么构建 harness 之前，先说说 harness 是什么，我会尽量给大家讲得最简单。harness 就是包在 AI agent 外面的一些代码。没错，你在这里最先听到这个说法：harness 就是包在 AI agent 外面、让它更好用的代码而已。这些代码里可以有 AI 吗？当然可以。必须有 AI 吗？不一定。harness 的目标是什么？让 AI 表现得更好。就这么简单。我觉得大家谈论这件事的方式把它搞得太神秘了，所以我想把它给各位讲得清清楚楚：它就是在你的 AI 周围多写一些代码，让它在某个特定用例上更有用。
[3:05] Claire
So, what are the parts of a harness? Well, a harness is going to have specific context, it's going to be able to take specific actions, and it's going to have a goal of specific outcomes. It's just as simple as that. And I want to talk about when it makes sense to build a harness and when it doesn't. And I think you want to build a harness when the same workflow needs the same setup and the same outcomes. And so, it's kind of similar to when you would build an AI agent. In fact, harness agent, sometimes you can interchange some of these concepts, but really it's when there is a sort of combination of deterministic and non-deterministic workflow, step-by-step process tools use cases, you want your AI to follow up to do a specific job. Usually those jobs are like slightly more complex. And this is why you've seen these coding harnesses come out like coding is a job to be done. It needs specific tools. It typically goes through kind of a standard workflow. And so coding harnesses are very popular, but you could also do things like managing production incidents where you need to go through a specific process, getting PRs ready for release, of handling support escalations, managing migrations. Even non-technical use cases like doing research in a very specific way or consolidating docs in a very specific way. That's how you and why you would use a harness. So, how did I decide what kind of harness I would build? Well, I looked across my business at Chat Priority and I thought, "What am I doing sort of repeatedly and consistently that I think AI could be good at? That I think we could be doing better if we were more structured about the AI and how we used it." And I thought that fixing bugs, you all if you've listened to this podcast, look, I ship code so I ship bugs. Fixing bugs is a very specific workflow where we've built some custom internal tools that I've been generally doing with Claude Coder Codex, but I had a had this hypothesis that I could do a better job of triaging bugs if I built my own harness. And so, I picked Sentry debugging and sorry for the Claude slot content here. Um Sentry debugging and debugging Sentry issues, really figuring out the issue, using some of our custom internal tools, and then doing all the follow-up actions we do when we close bugs was like a good first harness. It had coding in it. It needed custom content and custom context. There were like specific outcomes I wanted to make sure that we followed like tracking everything in linear and writing follow-up docs that the rest of the engineering team could use. And so, we chose uh debugging our
那么，harness 由哪几部分组成？一个 harness 会有特定的上下文，能执行特定的动作，并且以特定的结果为目标。就这么简单。我还想聊聊什么时候值得构建 harness、什么时候不值得。我认为，当同一个工作流需要同样的设置、同样的产出时，你就该建一个 harness。这跟你什么时候该做一个 AI agent 有点类似——事实上 harness 和 agent 这两个概念有时候可以互换。但真正的场景是：存在一种确定性和非确定性混合的工作流，有一步一步的流程、有工具、有具体用例，你希望你的 AI 按这个套路去完成一个特定的工作。通常这些工作会稍微复杂一些。这也是为什么你看到市面上出现了这些编程 harness——写代码就是一个「待完成的工作」，它需要特定的工具，通常也遵循一套比较标准的流程，所以编程 harness 特别流行。但你也可以做别的事，比如管理线上生产事故——那需要走一套特定流程；比如把 PR 准备好发布、处理客服升级工单、管理数据迁移；甚至一些非技术的用例，比如按某种非常特定的方式做调研，或者按特定方式整合文档。这就是你会用 harness 的场景和原因。那我是怎么决定要造一个什么样的 harness 的呢？我审视了我在 ChatPRD 的整个业务，然后想：「有什么事是我在反复、持续地做，而且我觉得 AI 能干得不错的？如果我们对 AI 的用法更结构化一点，哪些事能做得更好？」我想到了修 bug——各位如果听过这个播客就知道，我天天发代码，所以我也天天发 bug。修 bug 是一个非常特定的工作流，我们为此做了一些定制的内部工具，我平时一般用 Claude Code 或 Codex 来干，但我有个假设：如果我构建自己的 harness，bug 分诊（triage）可以做得更好。所以我选了 Sentry 排障——抱歉这里全是 Claude 相关的内容——Sentry 排障、调试 Sentry 上报的问题：真正搞清楚问题所在，用上我们一些定制的内部工具，然后把我们关闭 bug 时该做的所有后续动作都做完。这是一个很好的第一个 harness：它包含写代码，需要定制的内容和上下文，还有一些我想确保被执行到位的特定产出——比如所有东西都记录进 Linear，写好跟进文档让工程团队其他人可以用。于是我们就选了调试
[5:53] Claire
Sentry bugs, by we I mean me and Codex, chose debugging Sentry as a good use case to demonstrate how to build a harness. Now, why wouldn't I just use an AI coding tool directly? Well, I have been using AI coding tools directly. And I think the problem with using a general-purpose coding tool and why we've seen people build these specific use case harnesses is sometimes with a specific job, you just want to micromanage a little bit. You just want to be more prescriptive about how that job gets done. And so, if you can identify the right workflows, you can actually be more efficient, more consistent, and have better outcomes if you build a harness. So, for this specific use case, you know, with a direct AI tool like Claude Code, um I would have to explain what I want the the agent to do. So, I'd say like, "Dear agent, please fix this bug. Here it is." and send a link. Instead of this harness, I can literally just paste in the link and the agent already knows my intent, already knows what the job to be done. A second thing that I wasn't that worried about, but is interesting when you build your harnesses, you can be really prescriptive about what tools it's allowed to do and what it's allowed to execute and not. So, for example, if you wanted to build an investigate-only harness, you could make sure that your harness, your code editor, never actually wrote code. It only explored and explained root cause. You can also repeat the same process over time if you encode it in a harness. And so, if you want like a very precise step-by-step flow, including outcomes, so for us, every time we fixed a Sentry bug, we want it documented in Linear, we want a very specific report, we might even want to follow up with customers that it was impacted with. You could encode that in a skill, but then again, you have to babysit it. When we build this harness, we knew it would happen every time. And then, from a model perspective, you can do multimodal routing and all sorts of interesting things and ways that you couldn't with a general-purpose AI model. So, I'm going to show you how it works, and then we will talk about how I built it. Okay, so the interface I built for my harness is a terminal UI, again, like Claude Code or Code Geeks, something you would run in your eye in a UI.
Sentry bug 这个方向——「我们」指的是我和 Codex——选了调试 Sentry 作为演示如何构建 harness 的好用例。那么，我为什么不直接用 AI 编程工具呢？其实我一直在直接用 AI 编程工具。用通用编程工具的问题——也是为什么大家开始造这些特定用例的 harness——在于：面对某个特定的工作，你有时就是想稍微「微观管理」一下，想对这件事怎么做规定得更明确。所以只要你能找准合适的工作流，构建一个 harness 实际上可以让你更高效、更一致、拿到更好的结果。具体到这个用例：如果直接用 Claude Code 这类 AI 工具，我得先跟 agent 解释我想让它干什么，比如说「亲爱的 agent，请修这个 bug，链接在此」，然后发个链接。而有了这个 harness，我只要把链接粘贴进去，agent 就已经知道我的意图、知道要完成的工作是什么。第二点——我倒不是特别担心这个，但你构建自己的 harness 时会发现很有意思——你可以非常明确地规定它允许用哪些工具、允许执行什么、不允许执行什么。举个例子，如果你想造一个「只调查不动手」的 harness，你可以确保你的 harness、你的代码编辑器永远不真正写代码，它只做探索、只解释根因。另外，把流程编码进 harness 之后，你就能长期重复同一套过程。如果你想要一个非常精确的、包含产出物的分步流程——对我们来说，每修完一个 Sentry bug，我们都要在 Linear 里记录，要出一份特定格式的报告，甚至可能要跟受影响的客户做跟进——你当然可以把这些写进一个 skill，但那样你还得盯着它执行。而当我们把它做进 harness 里，我们知道它每一次都会发生。再从模型的角度看，你还可以做多模型路由之类各种有意思的事情，这些是用一个通用 AI 模型做不到的。好，接下来我演示它怎么运行，然后再聊我是怎么搭的。我给这个 harness 做的界面是一个终端 UI，就像 Claude Code 或 Codex 那样，是你在终端里跑的东西。
[8:13] Claire
And just so you know, your harness does not have to be a TUI. It doesn't have to be a CLI. It doesn't even have to have letters. It could be a web app. I did it in a TUI one because I haven't built one in a while. I thought it would be fun. And two, I just want to show that building your own custom harness means you can build your own custom interface into these AI agents as well. So, the harness is the whole experience, including the human experience that makes it more useful and easier to use. And so, um this TUI is pretty easy to invoke. I just run TUI. You can see it here. It's kind of cute. It's been made cute. Um I use this library called Ink, which helps you make cute TUIs. I don't think they would say cute, but I'm going to say cute. And you can see here that this terminal UI really reflects the structure of the harness itself. So, you see all the runs um that it's done so far, errors and how it's fixed things. And then, sort of our process, which is it gathers evidence, it streams in activities, and then it builds some artifacts. And so, I'm going to actually have it investigate this Sentry error over here. It's one where our edit um operations are getting dropped sometime by the agents. And that has now kicked off our specific harness. So, what it's going to do is it's going to start this investigation run. It's going to kick off a Claude SDK session, which is a fundamental part of how I built this. It's going to go ahead and start gathering evidence and coming up with a root cause hypothesis of what's causing this issue and how we might fix it. Now, as you can see, I chose I investigate, not F fix. So, the investigation should not touch and modify files. And again, this is something that I would have had to like prompt to the agent and say, I only want you to investigate. I do not want you to ship a fix. But instead, I can just click I, paste in that Sentry issue, and it's off to the races. This episode is brought to you by Customer.io.
顺便说一句，你的 harness 不一定非得是 TUI，也不一定是 CLI，甚至不一定要有文字界面——它完全可以是个 web 应用。我做成 TUI 有两个原因：一是我好久没做过 TUI 了，觉得会很好玩；二是我想说明，构建你自己的定制 harness 意味着你也可以为这些 AI agent 构建你自己的定制界面。harness 是整个体验，包括让它更好用、更易用的「人的体验」这一层。这个 TUI 调起来很简单，我直接运行 tui 命令，你们看，挺可爱的——我特意把它做得很可爱。我用了一个叫 Ink 的库，它能帮你做出可爱的 TUI。他们官方大概不会自称「可爱」，但我要这么说。你可以看到，这个终端 UI 其实反映了 harness 本身的结构：你能看到它到目前为止跑过的所有 run（运行记录）、出过的错、修过的东西；然后是我们的流程——收集证据、实时流式显示各项活动、生成一些产出物（artifacts）。现在我要让它去调查旁边这个 Sentry 报错，是一个我们的编辑操作有时会被 agent 弄丢的问题。好，它已经启动了我们这个专属 harness。它接下来会开始这次调查 run，拉起一个 Claude SDK 会话——这是我构建方式的核心部分——然后开始收集证据，提出一个根因假设：这个问题是什么导致的、我们可能怎么修。注意，你看我按的是 I（investigate 调查），不是 F（fix 修复）。所以这次调查不应该碰任何文件、不应该改任何文件。同样，这本来是我需要专门向 agent 提示的事——「我只要你调查，不要你发布修复」——但现在我只要按一下 I，把那个 Sentry issue 粘进去，它就跑起来了。本期节目由 Customer.io 赞助。
[10:18] Claire
You're here because you'd rather use AI than talk about it. With Customer.io, you describe the campaign you want to build, and the AI agent creates for you the audience, the messages, and the timing. You review it, make any changes you want, and launch. Instead of spending hours stitching together tools and workflows, you can focus on the work that actually drives growth. Every campaign is tied back to results, so you can see what's working and what to do next. [music] More than 9,000 brands use Customer.io to turn the data they already have into messages customers remember. Visit customer.io/howiai to try it today. Customer.io, more impact from every message. While this is running, I'm going to just go and show you a little bit about how this works and how I have actually built it. Okay, so this is the high-level architecture of the app. So, the front end is a terminal UI or a C CLI. Each invocation of the harness we call a run, so it's running a task. Each task has a specific input, usually that's a Sentry issue. And then there are specific flags I put on the harness that allow it to edit the source, modify the inputs, or even message customers only if I flag and approve it. So again, this is just a little bit more control over how the agent works. The harness core is run on Claude agent SDK and so all the agentic planning is run through the Claude agent SDK which has some of the primitives of Claude code including wrapping files and writing files and all those sorts of things that we find useful. And then what's really interesting about this harness, and you've seen in other harnesses like open claw, is it can create its own artifacts in its file store. And so we have this artifact store, I will show it to you in a minute, and it basically saves all the evidence from these runs to the file system for the agent to use in the future. And then it's connected to real tools, so it's connected to century, for sale, the Claude SDK, it's running Sonnet 46. I think that's the right the right model for the job. And then it's connected to linear and GitHub in terms of getting tasks done. Now what's really interesting as well is you can prompt this in a custom way. So instead of the general like you are Claude code, make no mistakes, you are our, you know, sort of model genius.
你来看这个节目，是因为你更想用 AI，而不是空谈 AI。用 Customer.io，你描述你想做的营销活动，AI agent 就会帮你生成受众、消息内容和发送时机。你审一遍，想改就改，然后发布。你不用再花几个小时把各种工具和工作流拼起来，而是可以专注在真正驱动增长的工作上。每个活动都能关联回结果，所以你能看到什么有效、下一步该做什么。已有超过 9000 个品牌在用 Customer.io，把手头已有的数据变成客户记得住的消息。访问 customer.io/howiai 今天就试试。Customer.io——让每条消息都更有影响力。趁它在跑，我来给你们讲讲这个东西的原理和我实际是怎么构建它的。这是这个应用的高层架构。前端是一个终端 UI，或者说 CLI。harness 的每次调用我们称为一个 run，也就是跑一个任务。每个任务有特定的输入，通常是一个 Sentry issue。然后我在 harness 上设了一些特定的开关（flag），只有在我明确标记并批准的情况下，它才被允许编辑源码、修改输入，甚至给客户发消息。所以说，这只是在 agent 的工作方式上多加了一层控制。harness 的核心跑在 Claude Agent SDK 上，所有的 agent 规划都通过 Claude Agent SDK 完成——它带有 Claude Code 的一些基础能力（primitives），包括读文件、写文件之类我们觉得有用的东西。这个 harness 特别有意思的一点——你在 OpenClaw 等其他 harness 里也见过——是它可以在自己的文件存储里创建自己的产出物。我们有一个 artifact store（产出物仓库），待会儿给你们看，它基本上把每次 run 收集到的所有证据都存进文件系统，供 agent 将来使用。然后它接上了真实的工具：接了 Sentry、Vercel；Claude SDK 这边跑的是 Sonnet 4.6，我觉得这是干这活最合适的模型；在推进任务方面还接了 Linear 和 GitHub。另外还有一点很有意思：你可以用定制的方式给它写提示词。不是那种泛泛的「你是 Claude Code，不要犯错，你是我们的天才模型」……
[12:51] Claire
I'm saying specifically that you're working inside the Chat Purity engineering harness, it's Chat Purity specific, it's not an open-ended coding system. We want to use these artifacts as a source of truth and here's the plan to attack a very specific problem. And what I want you to return is X, Y, and Z. And again, I don't have to copy and paste this, I don't even have to put it in a skill where hopefully it will get invoked in the right way. I've actually encoded this in a very specific step in the harness to make sure that the model falls it every time. And so there's several of these types of custom prompts inside my harness. There is um the artifacts that get generated. There are tool policies around like what tools can be called and which ones can't. And then um I have just decided again to use Claude Sonnet 4 6, um which is really I think the right model for this particular workflow. Okay, I want to talk a little bit about the code and how you generate this. And then just like a peek behind the scenes. I actually ran dueling Claude code and Codex sessions and essentially said like help me build a harness. I think I want to use the Claude agent SDK. Here's what I would like it to do. And then like closed my eyes and tried to get it done. Honestly, it was not a one-shot. I don't know if it was my prompting or the models were being funky. It was GPT 5.5 and Opus, but both of them really wanted to build something super deterministic. So, they like really resisted putting any AI in the harness and I had had to really prompt it very, very specifically to get what I want. So, I would say if you were trying to do this, I would be very specific about the workflow. I would be very specific about the tools. I would be very specific about where custom prompts make sense. And then I would suggest using an agent SDK either from Claude or from OpenAI to run most of it because without that prompting, I just did not get what I wanted out of these models. The second thing I will say, funnily enough, Codex did the best job at building the agent, but it used Claude agent's SDK to actually implement the agent. So, we are spanning across models and spanning across coding agents here.
我明确告诉它：你是在 ChatPRD 的工程 harness 里工作，这是 ChatPRD 专属的，不是一个开放式的编程系统；我们要把这些 artifacts 当作事实来源（source of truth）；这是攻克这个特定问题的计划；我要你返回的是 X、Y、Z。而且，这些我都不用复制粘贴，甚至不用写进一个 skill 然后祈祷它被正确调用——我把它直接编码进了 harness 里一个非常具体的步骤，确保模型每一次都会遵循。我的 harness 里有好几处这样的定制提示词，还有会生成的 artifacts、关于哪些工具能调用哪些不能的工具策略（tool policies）。然后我再说一遍，我选择了 Claude Sonnet 4.6，我确实认为它是这个工作流最合适的模型。好，我想稍微聊聊代码，以及这个东西是怎么生成的——给你们看点幕后。我实际上是同时开了 Claude Code 和 Codex 两个会话「对打」，跟它们说：帮我构建一个 harness，我想用 Claude Agent SDK，我希望它做这些事。然后我就闭上眼睛，试着让它们把活干完。说实话，这不是一次就成的（one-shot）。我不知道是我的提示词问题还是模型抽风——用的是 GPT-5.5 和 Opus——但它们俩都特别想构建一个超级确定性的东西，非常抗拒往 harness 里放任何 AI，我不得不非常非常明确地反复提示，才拿到我想要的结果。所以我建议，如果你要做这件事：把工作流写得非常具体，把工具写得非常具体，把哪里该用定制提示词写得非常具体；然后建议用一个 agent SDK——Claude 的或 OpenAI 的都行——来跑主体部分，因为不这样提示的话，我从这些模型那里根本拿不到我想要的东西。第二件事，说来好笑：Codex 在构建这个 agent 上干得最好，但它实际实现 agent 用的是 Claude Agent SDK。所以我们这是横跨多个模型、横跨多个编程 agent 在干活。
[15:09] Claire
But the actual harness itself is pretty simple. It's got sort of a high-level index of how you get to the TUI. And then it's got like, I don't know, eight files of specific things it can do. So, it can hunt for bugs in Sentry. Um it has a Sentry adapter to effectively use the Sentry API in a very specific way. So, instead of using the MCP generally, instead of like having your coding agent wander through all these traces, I'm just very precise about exactly what I think you need to pull from a bug report perspective, what's useful, what's not, and made that connector really opinionated. It's got similar a linear integration and a Vercel integration and a GitHub integration. So, again, not like generally how you can use these tools, but specifically how you would use these tools when you are searching for a bug. And then, after those tools and data sources are used, the bug is identified and triaged, then there is this artifact file here that outputs and spits out the specific artifact I want to see after a bug run is done. And that artifact bundle looks something like this. So, it's literally just uh the task run, um which is all the messages, the report, so what was the Sentry issue, here's a brief on what we discovered, here's any logs that we think are relevant, what the Cloud Worker ended up doing, and then the summary of the output. And then, we also output this beautiful HTML file um that I can show you that shows you what happened and how it all worked, as well as a worker report. So, I will show you those outcomes, as well. Just pulling up this code for you again, it's pretty straightforward. It's giving me all the instructions on where to put my specific API keys, and then, I can just run it in this very opinionated way. So, in addition to running the TUI, which lets me sort of like navigate through the UI and use this harness, something I might want to do as a human, it also has built these really easy command line tools, where if I just quickly want to run this harness against specific issues with specific flags on tool use, I can definitely do that. And what's kind of interesting about this is yes, I built this harness and you can see here I built this like fun UI so that I could use it in a fun way and it makes for a better demo, but really this harness is a structured way to give agents the job of running these investigations on an on a simpler basis.
但这个 harness 本身其实非常简单。它有一个高层的入口索引，告诉你怎么进到 TUI，然后大概有八个文件，每个负责一件具体的事。比如它可以去 Sentry 里找 bug——它有一个 Sentry adapter，以一种非常特定的方式去调 Sentry 的 API。也就是说，我没有泛泛地用 MCP，没有让 coding agent 在一堆 trace 里到处乱逛，而是非常精确地定义了：从 bug 报告的角度，我认为你需要拉取什么、什么有用、什么没用，把这个 connector 做得非常有主见（opinionated）。类似地，它还有 Linear 集成、Vercel 集成和 GitHub 集成。同样，不是「这些工具一般能怎么用」，而是「当你在排查一个 bug 的时候，具体该怎么用这些工具」。等这些工具和数据源都用完、bug 被定位和分诊（triage）之后，还有一个 artifact 文件，会在每次 bug 排查跑完后输出我想看到的特定产物。这个 artifact 包大概长这样：就是这次任务运行的记录——所有的消息、报告（这个 Sentry issue 是什么、我们发现了什么的简报、我们认为相关的日志）、Claude worker 最后做了什么，以及输出的总结。另外我们还会输出一个很漂亮的 HTML 文件——我可以展示给你看——它会告诉你发生了什么、整个流程是怎么跑的，还有一份 worker 报告。这些结果我等下都会展示。再把代码调出来给你们看一眼，真的很直白：它告诉我具体的 API key 该放在哪里，然后我就可以用这种非常有主见的方式去运行它。除了运行 TUI——让我可以像人类一样在界面里导航、使用这个 harness——它还内置了非常方便的命令行工具：如果我只是想快速地针对特定 issue、带上特定的工具使用 flag 来跑这个 harness，完全可以做到。有意思的地方在于：没错，我造了这个 harness，你们也看到我做了这个好玩的 UI，让我可以用一种有趣的方式使用它，演示效果也更好；但说到底，这个 harness 是一种结构化的方式，把「跑这些排查」这项工作，以更简单的形式交给 agent 去做。
[17:46] Claire
And so you can imagine while I design the TUI for human, actually giving a kind of all intelligent agent a specific harness to solve a specific problem with agents in that, I think that's how you're going to get real leverage and really custom outcomes out of things like coding agents like Claude Code. And so going through this process has really opened my mind to we've gotten so used to like the open chat field. Like if I just type in, the agent will do good work. And I think we all have done good work. But then now I've realized that these agents can help us solve very, very specific problems using other agents and by constraining that work, we can actually get specific jobs done really efficiently and then use the general purpose agent to sort of orchestrate it. So it's really like changed my mind about how work gets done. As you can see here again, it's just a couple files. It's really not too much. The adapters to the data sources, um a couple workflows. In particular, this bug hunter workflow which just goes through exactly how we want to hunt bugs, including how we want to put together summaries of bug reports and then some files here in terms of running the TUI or the CLI. And then as I said, we have this artifacts folder that gets updated every time a run happens where I can click in and actually see exactly what happened out of a run. So, let's go and see if this run happened well and what I can find out. So, now I have the full context. Here's the investigation brief and I can go look for it. So, this is Bug Hunter C7. Let's see if I can find this one. Here it is. Here's the investigation brief on that edit document operations dropped. I have confirmed evidence. So, it's saying, "Yes, there was definitely a Sentry warning. It's impacted 150 users. It's still happening hourly. Um it's a warning, so it's not an actual error."
所以你可以想象，虽然这个 TUI 是我为人类设计的，但真正的玩法是：给一个全能的智能 agent 一个专门的 harness，让它带着里面的 agent 去解决一个特定问题——我认为这才是你从 Claude Code 这类 coding agent 身上榨出真正杠杆、拿到真正定制化结果的方式。走完这个过程真的让我打开了思路。我们已经太习惯那个开放的聊天输入框了——好像我只要打几行字，agent 就会把活干好。而且说实话，我们确实也干出过不少好活。但现在我意识到，这些 agent 可以帮我们用「其他 agent」去解决非常非常具体的问题；通过约束这些工作，我们可以让特定的活干得非常高效，然后再用通用型 agent 去做编排（orchestrate）。这真的改变了我对「工作是怎么被完成的」的看法。你再看这里，就是几个文件而已，真的不多：对接数据源的 adapter，几个 workflow——特别是这个 bug hunter workflow，它完整写清了我们想怎么排查 bug，包括怎么组织 bug 报告的总结——然后就是运行 TUI 或 CLI 的几个文件。另外我说过，还有一个 artifacts 文件夹，每次运行都会更新，我可以点进去看每次运行到底发生了什么。那我们现在就去看看这次运行跑得怎么样、能查出什么。现在我拿到了完整的上下文，这是调查简报（investigation brief），我可以去找它。这是 Bug Hunter C7，看看能不能找到——在这儿。这就是关于「编辑文档操作被丢弃」那个问题的调查简报。我已经拿到了确凿的证据：它说，是的，Sentry 里确实有一条 warning，影响了 150 个用户，而且现在每小时都还在发生。不过它是一个 warning，不是真正的 error。
[19:54] Claire
And the Versel logs were unavailable and so we weren't able to use that data. And then it found likely root causes. So, invalid original range or overlapping original range. And so, it's identified a couple potential root causes as well as a blind spot in this particular function. It's told me exactly where in the product surface um the issue is and then how I would actually verify this by fetching a raw Sentry event to see if the issues that they've identified are correct. It's identifying should it um issue a linear issue and it says, "Yes, we should definitely make a linear issue to fix this." And so, this should get assigned to somebody. And then it doesn't recommend turning on patch mode and actually fixing this. So, again, this is like a very specific outcome I wanted. I wanted to say like, "What's all the evidence? Priority rank the root causes. Make a suggestion on the next step if we need to verify this more. Tell me if I need to assign it to somebody in Linear and then tell me if you can fix it." And they're saying, "No, I don't think I can fix it yet. I need a little bit more information." And all of that is built because I have done this like very specific workflow and encoded that in what we're calling a harness, which is just code around an agent. So, how would you you build your own harness? I feel like hopefully you're still with me not too much of that went over your head. Just to reiterate, I just identified a specific workflow. I determined what the run against the task would look like. I made very opinionated calls to tools or data sources, so I didn't just say like use an MCP, although that could be part of your harness. But what I did is I made adapters that made the calls to these external APIs and tools very specific. I thought about what the structured artifacts out of that workflow might be. I decided what rules and permissions I wanted to give this harness and which ones I didn't. I decided whether I wanted to use Claude Code or Codex or a model router to actually run these things. And then I built a surface to interact with this agent. So, I built a TUI so I could actually look and work with this harness in a way. It could be a TUI, it could be a CLI, it could be a web app, but I built some way to interact with this.
而 Vercel 的日志当时拿不到，所以我们没能用上那部分数据。然后它找出了可能的根因（root cause）：无效的原始 range，或者重叠的原始 range。它识别出了几个潜在根因，还发现了这个函数里的一个盲区。它精确地告诉我问题出在产品的哪个位置，以及我该怎么验证——去拉取一条原始的 Sentry 事件，看看它们识别出的问题是否正确。它还在判断要不要建一个 Linear issue，结论是：「要，我们绝对应该建一个 Linear issue 来修这个问题」，所以这事应该被指派给某个人。然后它不建议开启 patch 模式直接去修。你看，这正是我想要的非常具体的产出。我想要的就是：把所有证据摆出来，按优先级排列根因，如果还需要进一步验证就给出下一步建议，告诉我要不要在 Linear 里指派给谁，再告诉我你能不能直接修。它的回答是：「不行，我觉得现在还修不了，我还需要更多信息。」这一切之所以成立，是因为我把这个非常具体的工作流编码进了我们所说的 harness 里——harness 说白了就是包在 agent 外面的一层代码。那么，你自己该怎么搭一个 harness 呢？希望到这里你还跟得上，前面的内容没有太超纲。重新梳理一下：我先确定了一个具体的工作流；我想清楚了针对这个任务的一次运行长什么样；我对工具和数据源做了非常有主见的调用——我没有只说「用个 MCP 吧」（虽然 MCP 也可以是你 harness 的一部分），而是做了 adapter，把对这些外部 API 和工具的调用变得非常具体；我想清楚了这个工作流应该输出什么样的结构化 artifact；我决定了要给这个 harness 哪些规则和权限、不给哪些；我决定了到底用 Claude Code、Codex，还是一个模型路由器（model router）来实际执行；最后我搭了一个和这个 agent 交互的界面——我搭了个 TUI，好让我能实际查看并使用这个 harness。它可以是 TUI，可以是 CLI，也可以是个 web 应用，总之我搭了某种交互方式。
[22:18] Claire
So, this is what you need to do. Identify a workflow. Uh really write it down on, you know, proverbial paper, HTML or markdown. Figure out what sources of data you want and then plug it all into Claude Code or into Codex as I did and have it build your own harness and then test it against real data. So, that's it. I just I really hope that you walk away from this realizing that these mystery terms like harness are not that mysterious. A harness is simply putting some structure around how AI works. Yes, Cursor is like a really complex harness. Yes, Codex and Claude Code are very complex coding harnesses. But at the end of the day, they're code that wraps these AI agents and these AI calls to make them more efficient in doing a very specific job. And so whether you're doing that in a very prescriptive way like I just showed where I want to show you how I triage sentry bugs, do the investigation and pass it on to the team, or you're doing it a broad way like these general purpose coding agents that just have access to tools and context and methods that make the coding workflow better. That's all harnesses. You can think of harnesses that you can build. You can build them in the terminal. You can build them for CLIs. You can even build them as web apps. I'm starting to hypothesize that a wrapper is just a harness and that is going to upgrade everything that I've vibe coded over the last 3 years. This has been totally a learning experience for me here on How AI. This is my very first harness that I've built live on the show. I hope it's useful for you. And if you're interested in me building other and demystifying AI terms, let me know in the comments. Thanks for joining How AI. Thanks so much for watching. If you enjoyed the show, please like and subscribe here on YouTube or even better, leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify, or your favorite podcast app. Please consider leaving us a rating and review which will help others find the show. You can see all our episodes and learn more about the show at howiaiipod.com.
所以你要做的就是这些：确定一个工作流，认认真真把它写下来——写在纸上也好，HTML 或 markdown 也好；想清楚你需要哪些数据源；然后像我这样把这一切丢进 Claude Code 或 Codex，让它帮你搭出你自己的 harness，再拿真实数据去测试。就是这样。我真心希望你看完之后能意识到，harness 这类神秘词汇其实一点都不神秘。harness 无非就是给 AI 的工作方式加上一些结构。没错，Cursor 是一个非常复杂的 harness；没错，Codex 和 Claude Code 是非常复杂的 coding harness。但归根结底，它们就是包裹在 AI agent 和 AI 调用外面的代码，让它们在完成某项特定工作时更高效。所以，无论你是像我刚才演示的那样，用非常规定性的方式来做——我想展示我是怎么分诊 Sentry bug、做调查、再交接给团队的——还是像那些通用 coding agent 一样做得很宽泛，只是给它们配上让编码工作流更顺畅的工具、上下文和方法，这些统统都是 harness。你可以想想自己能搭哪些 harness：可以搭在终端里，可以做成 CLI，甚至可以做成 web 应用。我开始有一个猜想：所谓的套壳应用（wrapper）其实就是一个 harness——这个认知会让我过去三年 vibe coding 出来的所有东西都升级一遍。这期对我自己来说完全是一次学习之旅，这是我第一次在 How I AI 节目里现场搭 harness，希望对你有用。如果你想让我继续搭点别的、继续拆解那些 AI 术语，请在评论区告诉我。感谢收看 How I AI。非常感谢观看。如果你喜欢这期节目，请在 YouTube 上点赞和订阅，或者更好的是，留言分享你的想法。你也可以在 Apple Podcasts、Spotify 或你喜欢的播客应用上找到这档播客。欢迎给我们打分和评论，这会帮助更多人发现这个节目。你可以在 howiaipod.com 查看我们所有的节目并了解更多信息。
[24:32] Claire
See you next time.
我们下期见。