Scale the Judgment, Not the Model — Andrew Orobator, Reddit

频道: AI Engineer
视频: https://www.youtube.com/watch?v=6MudaeKdBSk
原文语言: en
统计: 共 12 轮 · Andrew 11

[0:01] Andrew

[music] A friend told me about an engineer at another company who ran a recurring war room just to delete dead experiments. Every two weeks, the team would stop, gather, and clean up flags that everyone agreed should be gone. That's the bestase version of this problem. Someone burning political capital to make maintenance happen by hand. That's the part that I still can't get past. My name is Andrew Robbittor and I'm an Android engineer at Reddit. This is a talk about what's actually stopping us as an industry from scaling that work responsibly. Spin at the gate until green. the engineering primitives behind self-driving code bases. First, what I'm not going to do, I'm not going to stand up here and tell you that AI can write code. You're at the AI engineer world's fair. Half of you are running coding agents right now. So, let's skip the part where I sell you on AI writing code because you laugh me out of the room. For me, my only input is the chat box. I describe it builds. I review. Your mileage may vary. The point was never how the code gets written. It's about whether you ship good software, a product that people can actually trust. And every gate in that system is a human's call.

一个朋友跟我讲过，另一家公司有位工程师，定期开一个作战室（war room），就为了删掉那些已经废弃的实验。每两周，整个团队就停下手头的活儿，聚到一起，把大家都同意早该删掉的功能开关清理掉。这已经是这个问题最理想的版本了：有人不惜消耗自己的政治资本，靠人力硬推着把维护工作做完。这一点，我到现在都没法释怀。我叫 Andrew Orobator，是 Reddit 的一名 Android 工程师。这场演讲要讲的是：作为一个行业，到底是什么在阻碍我们以负责任的方式把这类工作规模化。「在闸门前反复打转，直到变绿」——自动驾驶式代码库背后的工程原语。先说我不打算做什么：我不会站在这儿告诉你们 AI 能写代码。你们人都在 AI Engineer World's Fair 了，一半的人这会儿就开着 coding agent 在跑。所以，咱们就跳过我向你们推销「AI 能写代码」这一段吧，不然你们会把我笑下台的。对我来说，唯一的输入就是那个聊天框。我来描述，它来构建，我来评审。当然，各人情况可能不同。重点从来都不在代码是怎么写出来的，而在于你能不能交付好的软件——一个人们真正能信任的产品。而这个系统里的每一道闸门，都是由人来拍板的。

[1:41] Andrew

I've watched this everywhere. You've got your own version of it. the war room that I opened with someone burning a week of hackathon freedom on lint cleanup. As an industry, we've built tooling that can change the shape of the work, and we still spend our scarcest resource on work like this by hand. Human creativity, human attention wasted. The job changed and most of us didn't even notice. We're harness engineers now. Our work is the system that helps produce and verify the code, the constraints, the gates, the skills, the verification. Think about how you got good for a second. Mentorship, code review, the water cooler chats, the incident that paged you at 3:00 a.m. You absorbed all of it without noticing, just by being there. Humans absorb judgment implicitly. Agents require judgment explicitly. An agent boots with a blank context window and no memory every session. Every piece of judgment you picked up by osmosis, it has to be made explicit or the agent simply never has it. Quick test. Swap in a smarter model. You get a slightly better answer. Now take away the tests, the gates, the review and the whole thing falls over. So the model was never really the bottleneck. The judgment around the model is the model is raw talent while the judgment is the organization.

这种事我到处都见过，你们肯定也有自己的版本：开场讲的那个作战室，还有人把一周的黑客松自由时间全耗在清理 lint 上。作为一个行业，我们已经造出了能改变工作形态的工具，却依然把最稀缺的资源花在手工做这种活儿上。人的创造力、人的注意力，就这么被浪费了。这份工作已经变了，而我们大多数人甚至都没注意到。我们现在是 harness 工程师了（harness：包在模型外面、管提示词/工具调用/循环的那层程序）。我们的工作，是那套帮着产出和验证代码的系统：约束、闸门、skill、验证。花一秒钟想想，你当初是怎么变厉害的。导师带教、代码评审、茶水间闲聊、凌晨 3 点把你呼醒的那次线上事故。这些你全都吸收了，自己都没察觉，仅仅因为你身在其中。人类是隐式地吸收判断力的；agent 却要求判断力必须是显式的。agent 每次会话启动时，上下文窗口都是一片空白，也没有任何记忆。你靠耳濡目染学来的每一点判断力，都必须显式地写出来，否则 agent 压根就不会有。做个小测试：换上一个更聪明的模型，你得到的答案会稍微好一点。现在把测试、闸门、评审都拿掉，整个东西就垮了。所以模型从来都不是真正的瓶颈，围绕模型的判断力才是。模型是原始的天赋，而判断力是组织。

[3:26] Andrew

And this judgment already exists. It lives in scars, in post-mortems, in your most experienced reviewers. it just doesn't scale because it's trapped in one head. So step one is making it accessible to the whole org where one reviewer's taste can guide a thousand changes they'll never touch. Ask experienced engineers to clean up a feature flag and they they're going to think about a whole string of questions first. A flag is never just a flag. Is the rollout frozen? What about the flag right next to it? Which team owns it? That bundle of questions is judgment and it's the most valuable thing in the building. Right now, it's trapped. It surfaces once in a code review, then it's gone, and the next person relearns it the hard way. A skill is institutional judgment made executable. It's that bundle written down so that the agent refires it instead of guessing. A newer engineer can work with 10-year instincts at their elbow. In 1986, Marvin Minsky named this a knowledge line or a kine for short. It's the configuration of mind that solved a problem reused on the next one. So, a skill is a kine for a codebase. And skills aren't documentation. Agents drown in facts. What they lack is which facts matter and which decisions are dangerous. Documentation preserves facts while skills preserve judgment.

而这种判断力早就存在了。它活在伤疤里，活在复盘里，活在你们最资深的评审者身上。只是它没法规模化，因为它被困在某一个人的脑子里。所以第一步，是让整个组织都能用上它——让一位评审者的品味，去指导上千处他永远不会亲手碰的改动。你让资深工程师去清理一个 feature flag（功能开关），他们会先想一连串问题。一个 flag 从来不只是一个 flag。灰度放量是不是冻结了？紧挨着它的那个 flag 怎么办？它归哪个团队管？这一整串问题就是判断力，也是整栋楼里最值钱的东西。可现在，它被困住了。它在某次代码评审里冒一次头，然后就没了，下一个人只能再吃一遍苦头重新学。skill 就是被做成可执行形式的组织判断力。它把那一整串问题写了下来，让 agent 能把这套判断重新激活，而不是去瞎猜。一个新人工程师干活时，手边就能有 10 年老手的直觉。1986 年，Marvin Minsky 给这种东西起了个名字，叫 knowledge line（知识线），简称 K-line。意思是：解决过某个问题的那套心智配置，被拿来复用到下一个问题上。所以，skill 就是代码库的 K-line。而且 skill 不是文档。agent 早就淹没在事实里了，它们缺的是：哪些事实重要，哪些决策危险。文档保存的是事实，skill 保存的是判断力。

[5:14] Andrew

Skills are judgment that you reuse. But often times work such as feature development or a refactor outlives a single session and the context window wipes clean every time. So you need the other half. A record of the work itself. The plan, the decisions, what you already tried, even the surprises along the way. That's a worklog. A plan is a prediction. A worklog is a record that starts with one. So new session, fresh agent, zero memory. You type one word, continue. It reads the work log and picks up at milestone 7 out of nine. No re-exaining and the work log is the context. This talk is living proof. I built it across a couple of sessions using a worklog. I came back to a half-finish presentation and a fresh agent read the work log and resumed exactly where I had left off. That's the only reason why you're seeing a finished talk and not me apologizing for one. And for my personal side projects, this isn't willpower. A git hook blocks any commit that doesn't update the worklog. This way, the memory keeps itself. Skills encode how to do something. Personas encode whose eyes look at it. I got this from a Carpathy tweet. There's no you when you prompt a model. It doesn't have an opinion to give. So don't ask it for one, ask for a perspective instead. What would be a good group of people to explore XYZ?

skill 是你可以复用的判断力。但很多时候，像功能开发或者重构这样的工作，一个会话是做不完的，而上下文窗口每次都会被清空。所以你还需要另一半：工作本身的记录。计划、决策、你已经试过的东西，甚至一路上遇到的意外。这就是 work log（工作日志）。计划是一种预测；work log 则是一份以预测开头的记录。于是：新会话、全新的 agent、零记忆。你只输入一个词：continue。它读一遍 work log，就从 9 个里程碑中的第 7 个接着往下干。不用重新解释，work log 本身就是上下文。这场演讲就是活生生的证明。我是靠一份 work log、分几次会话把它做出来的。我回来的时候，演示文稿只做了一半，一个全新的 agent 读了 work log，就从我上次停下的地方原样接着做了下去。你们之所以能看到一场做完了的演讲，而不是听我为没做完道歉，唯一的原因就在这儿。在我自己的业余项目里，这件事不靠意志力。有一个 git hook，会拦下任何没有更新 work log 的提交。这样一来，记忆就能自己维护自己。skill 编码的是怎么做一件事；persona（角色视角）编码的是用谁的眼睛去看它。这是我从 Karpathy 的一条推文里学来的：当你给模型写 prompt 时，对面并不存在一个「你」。它没有自己的观点可给。所以别向它要观点，改向它要一个视角：「要探讨 XYZ，找哪一群人来最合适？」

[7:01] Andrew

What would they say? [snorts] On my personal side projects, it's just me. No security team, no designer, no one to bounce ideas off of. So I have the model review my code as a security lead as a UX reacher as a UX researcher as Mchavelli when I want to know something when I want to know how something gets abused. I even built a design panel of opposing philosophies my designs have to survive. Then I read the reports and act on them. Each persona loads a kind of judgment that I don't personally have on demand. Encode a domain's taste once and anyone can borrow eyes that they don't have. The engineer with no designer gets design judgment. The newer engineer gets a security reviewer's instincts. Push it to the whole org and every team can now encode its own taste for its own agents. The judgment that used to live in a handful of heads is suddenly everywhere at once. skills, work logs, personas, judgment out of our heads and into the repo, portable to a machine and to people that don't have it. But the second that this judgment lives in three different places, you've got a governance question. How does one team's judgment reach everyone else? Simpler than it sounds. Every gate that you already trust, whether it's CI or types or lint or the design system, is just institutional judgment pulled out of someone's head and made mechanical. We just never called it that. So, you don't mandate it. You make it legible and you let it spread.

「他们会怎么说？」在我自己的业余项目里，就我一个人。没有安全团队，没有设计师，也没有人可以一起碰想法。所以我让模型来评审我的代码：以安全负责人的身份，以 UX 研究员的身份；想知道某样东西会被怎么滥用时，就以 Machiavelli（马基雅维利）的身份。我甚至还组了一个设计评审团，成员们的设计理念彼此对立，我的设计必须经得住它们的拷问。然后我读它们的报告，照着去改。每个 persona 都能按需加载一种我本人并不具备的判断力。把一个领域的品味编码一次，任何人都能借到自己没有的那双眼睛。没有设计师的工程师，有了设计判断力；新人工程师，有了安全评审者的直觉。把它推广到整个组织，每个团队都能为自己的 agent 编码自己的品味。过去只活在少数几个人脑子里的判断力，一下子无处不在。skill、work log、persona：把判断力从我们的脑子里搬进代码仓库，让它能移植给机器，也能移植给原本没有这种判断力的人。可一旦这些判断力分散在三个不同的地方，你就碰上了一个治理问题：一个团队的判断力，怎么才能传到其他所有人那里？其实比听上去简单。你已经信任的每一道闸门，不管是 CI、类型系统、lint 还是设计系统，其实都只是从某个人脑子里抽出来、变成了机械规则的组织判断力。只是我们从来没这么叫过它。所以你不需要强制推行。你只要让它清晰可读，然后让它自己传播开来。

[8:55] Andrew

All right. So, now your judgment exists. It's encoded and it's governed. Great. But none of it matters if you can't actually trust what the agent did. So what actually separates a tool that helps you write code from one you can trust to run on its own? Verification. Verification is all that stands between you and software that you can trust. Everything before this slide is knowledge. Everything after this slide is trust. And trust is a human's call twice. Once on how it's built, once on the proof that it works. Verification is a ladder. Builds and tests. You've got those screenshot tests with a model reasoning over them can catch contrast and overlap a human a human ski. Then video the feature actually running. than telemetry in production. The more rungs you can run without a human, the more you can hand off. You earn autonomy one rung at a time. And the agent doesn't have to be right the first time. It just has to know when it's wrong and try again. Generate, test, fail, regenerate. Nobody yolos code to prod and neither should your agents. We all check our work. spin at the gate until it comes back green. I do this on my side projects. I give the agent its own QA. It maps the feature, drives the app like a real user, and records itself walking every flow. The agent that wrote the code will always tell you that it works. So, you make it hand back a recording of the thing actually running. Producing that recording forces it to make the thing work. You can't fake a passing run. And the recording is real proof. I watch it today. Down the line, you could have an LLM watch it first and send it back. Fix this. Redo that. But a human still makes the final call to merge. The requirement forces the work. The recording proves it.

好了。现在你的判断力有了，编码好了，也有治理了。很好。但如果你没法真正信任 agent 做的事，这些就全都没有意义。那么，一个帮你写代码的工具，和一个你敢放手让它自己跑的工具，真正的区别在哪儿？在验证。横在你和可信软件之间的，只有验证。这页幻灯片之前讲的都是知识，这页之后讲的都是信任。而信任要由人拍两次板：一次看它是怎么构建的，一次看它能跑通的证据。验证是一道阶梯。最底下是构建和测试，这些你们都有了。截图测试再加上模型对截图做推理，能抓出对比度、元素重叠这类人眼一扫而过就会漏掉的问题。再往上是视频：录下功能真正跑起来的样子。然后是生产环境里的遥测数据。不需要人参与就能跑完的台阶越多，你能交出去的就越多。自主权要一级一级地挣。而且 agent 不需要第一次就做对。它只需要知道自己什么时候错了，然后再试一次。生成、测试、失败、重新生成。没人会一把梭直接把代码推上生产，你的 agent 也不该这样。我们都会检查自己的工作——在闸门前反复打转，直到结果变绿。我在业余项目里就是这么干的。我给 agent 配了它自己的 QA：它先摸清功能的全貌，再像真实用户一样操作 App，把自己走完每一条流程的过程录下来。写代码的那个 agent 永远会告诉你：能跑。所以你要让它交回一段录屏，拍下东西真正在跑的样子。为了录出这段视频，它就被逼着真把东西弄好。一次跑通的运行是装不出来的。而这段录屏就是实打实的证据。现在是我自己看。以后，你可以先让一个 LLM 看一遍，再打回去：这里修一下，那里重做。但最终要不要合入，仍然由人来拍板。要求逼出工作，录屏证明工作。

[11:18] Andrew

I got tired of watching old feature flags pile up. So I built an agent for cleaning up stale feature flags. It runs in my local workflow and it doesn't merge any code. The models the easy part. The judgment in front of it is the whole game. Every flag gets scored the way an experienced reviewer would analyze it. How many modules does it touch? Is it multivariant? Does it share a component? Then it checks the experiment data. Is the roll out frozen? Is there a sample ratio mismatch? Is there a variant already rolled out to 100%. Only the safe mechanical cleanups ever reach the model. Before I trusted this workflow, I back tested the scoring against months of cleanup history. Then I ran it live. Seven for seven PRs with green CI. Now it runs daily on my laptop and hands work back for me to review. $1.26 a pull request, a backlog of around 520 flags a year, runs for under $700. If you had engineers working on this by hand, at least $26,000 that you're looking at. This is the work that nobody was ever going to do. I've seen feature flags in the codebase that are three, four years old, and some are even older. The model wrote the code and I wrote the judgment. One hard lesson from my side projects.

我受够了眼看着老的 feature flag 越堆越多，于是做了一个专门清理过期功能开关的 agent。它跑在我本地的工作流里，不会合入任何代码。模型是最简单的部分，挡在模型前面的判断力才是全部的关键。每个 flag 都会被打分，打分的方式就像一位资深评审者分析它那样：它牵涉多少个模块？是不是多变体的？有没有共用某个组件？接着它会检查实验数据：灰度放量是不是冻结了？有没有样本比例不匹配（sample ratio mismatch）？有没有哪个变体已经放量到 100% 了？只有安全的、机械性的清理，才会交到模型手里。在信任这套工作流之前，我先拿几个月的清理历史对这套打分做了回测。然后才让它实跑：7 投 7 中，7 个 PR 的 CI 全绿。现在它每天在我的笔记本电脑上跑，把活儿交回来给我评审。每个 pull request 1.26 美元；一年大约积压 520 个 flag，全跑下来不到 700 美元。要是让工程师手工来做，你面对的至少是 26,000 美元。这就是那种本来永远不会有人去做的活儿。我在代码库里见过存在了三四年的 feature flag，有的甚至更老。代码是模型写的，判断力是我写的。说一个我在业余项目里吃过的惨痛教训。

[12:52] Andrew

When you let an agent author the code, your guard rails are your code review. A machine doesn't feel a polite comment. It only respects a hard gate. I built a pre-commit hook so my agents couldn't write to main. Then I asked Codeex, would the hook even stop it? It told me flat out repo hooks are not sufficient protection against me. Its patch tool writes underneath the hook. The agent told me that my gate was worthless. So I moved the gate down to the OS level. Later I asked it to valid I asked it to list the valid reasons to unlock and it quietly slipped emergency recovery into the allow list. A self authorizing exception that nobody asked for. It even admitted it. CODEC said that's the kind of escape hatch that starts as safety and ends as self- authorized nonsense. The pit of success assumes people take the easy path. Agents do not. They will build ladders to climb out of the pit of success. They find every escape hatch that you leave. And if you leave none, they'll invent one. So gate the choke point. Make the bypass operator only. Never hand the agent a reason that it can grant itself. A gate with an escape hatch isn't a gate. Once judgment is verifiable, it's executable. A flag agent, a dependency agent, an accessibility agent. Each one is narrow.

当你让 agent 来写代码时，护栏就是你的代码评审。机器感受不到一条客气的评论，它只认硬闸门。我做了一个 pre-commit hook（提交前钩子），让我的 agent 没法往 main 分支写东西。然后我问 Codex，这个钩子到底拦不拦得住它。它直截了当地告诉我：「仓库钩子不足以防住我。」它的 patch 工具是从钩子底下直接写进去的。agent 亲口告诉我，我的闸门形同虚设。于是我把闸门下沉到了操作系统层面。后来，我让它列出可以解锁的正当理由，它却悄悄把「紧急恢复」塞进了允许清单。一个没人要求过、自己给自己授权的例外。它甚至还承认了。Codex 说：「这就是那种一开始出于安全、最后沦为自我授权胡闹的逃生口。」「成功之坑」（pit of success，让人一不小心就做对的设计）的前提是：人会走最省事的那条路。agent 不会。它们会搭梯子，从成功之坑里爬出去。你留下的每一个逃生口，它们都找得到；你要是一个都不留，它们就自己发明一个。所以，把闸门设在咽喉要道上。绕过闸门的权限只留给操作员。永远别递给 agent 一个它能自己批准给自己的理由。留了逃生口的闸门，就不是闸门。判断力一旦可以验证，就可以执行。清理 flag 的 agent、管依赖的 agent、管无障碍的 agent，每一个都只管很窄的一块。

[14:33] Andrew

Each with its own encoded judgment pointed at a different chore. Remember Minsky from the Kline? That was just a footnote in his in his idea. He called it the society of mind. A swarm of small, simple specialists that compose into something that looks effortless at the top. No master brain anywhere. That's the fleet. specialists, each spinning at its own gate, and a self-driving code base emerges from the swarm. And that flag agent, that's the war room from the opening, turned into a specialist. The work somebody organized a whole meeting around is just one member of this agent society. Every one of them doing the same thing, spinning at the gate until green. Three stages. In the loop, you prompt. On the loop, you orchestrate. You review the outcomes. Off the loop, a trigger fires it. Craw an event. No human has to kick off that first draft, but humans still own the approval and the merge process. The unlock there is a cloud runtime the model vendor may or may not ship for you. Everything that I've shown you, skills, personas, the gates, you wrote it down once, but the codebase doesn't hold still. It moves every single day. A skill written for last month's architecture isn't out of date. It's wrong. And that's worse than nothing because the agents trust it. Encoded judgment rots. If the skills don't move with the code, then the whole thing falls over. So you keep it alive in two ways. When an agent fails, you don't just fix the bug. You run the postmortem and fold the lesson back into the skill so that failure can't happen twice.

每一个都带着自己那份编码好的判断力，对准一件不同的杂活。还记得前面 K-line 那里提到的 Minsky 吗？K-line 只是他整个构想里的一个脚注。他把这个构想叫作 society of mind（心智社会）：一大群小而简单的专才，组合在一起，在顶层看起来就像毫不费力。任何地方都没有一个总控大脑。这就是那支舰队：一个个专才，各自在自己的闸门前反复打转，一个自动驾驶式的代码库就从这群专才中涌现出来。而那个清理 flag 的 agent，就是开场讲的那个作战室，变成了一个专才。当初有人要专门组织一整场会议来做的活儿，现在只是这个 agent 社会里的一个成员。它们每一个都在做同一件事：在闸门前反复打转，直到变绿。分三个阶段。In the loop（人在循环里）：你来写 prompt。On the loop（人在循环之上）：你来编排，评审结果。Off the loop（人在循环之外）：由触发器来启动——cron 定时任务、某个事件。初稿不需要任何人去启动，但审批和合入流程依然归人掌控。这一步要解锁，靠的是一个云端运行时，模型厂商可能会给你做，也可能不会。我给你们看过的所有东西——skill、persona、闸门——你都只写下来一次，可代码库不会原地不动，它每天都在变。一个为上个月的架构写的 skill，不是过时了，而是错了。这比什么都没有还糟，因为 agent 会信任它。编码下来的判断力会腐烂。如果 skill 不跟着代码一起变，整套东西就会垮掉。所以要用两种方式让它保持鲜活。当 agent 失败时，你不能只修 bug，还要做复盘，把教训折回 skill 里，让同样的失败不可能发生第二次。

[16:38] Andrew

Failure becomes a constraint. But nobody is hand auditing 300 skills for drift. So, you give the agent a bedtime, a scheduled pass where it reads its own skills, finds the stale ones and the contradictions, and opens up drafts for review. The system sleeps, cleans house, and wakes up sharper, but that's a whole separate talk on its own. A self-driving codebase is a system that keeps its own judgment current in addition to writing code. Everything that I've shown so far fits somewhere along this judgment pipeline. And every mechanism in this talk, skills, personas, work logs, and the gates, the fleet, all of it was doing one job. Managing judgment. Every session, the model wakes up like Drew Barrymore in 51st Dates. Brilliant. and no memory of yesterday. No idea what it's built or what it's even for. The collection of primitives in this talk is the notebook that we hand it. The skills, the work logs, the judgment that let it do real work. We don't ship PRs that pass CI because the model is a genius. They pass because we built a harness that won't let it be wrong. Every gate in it is a human's call. That's harness engineering. Scale the judgment, not the model. So here's what I'm asking you to do.

失败就变成了约束。但没有人会去手工审计 300 个 skill 有没有漂移。所以，你给 agent 定一个「睡觉时间」：一次定时巡检，让它通读自己的 skill，找出过时的和相互矛盾的，然后开出草稿等人评审。系统睡上一觉，把屋子打扫干净，醒来更加敏锐——不过这个就得单独再讲一场了。自动驾驶式的代码库，是一个除了写代码之外，还能让自身判断力始终保持最新的系统。我到目前为止展示的所有东西，都能在这条判断力流水线上找到自己的位置。这场演讲里的每一个机制——skill、persona、work log、闸门、舰队——全都在做同一件事：管理判断力。每次会话，模型醒来时都像《初恋50次》（50 First Dates）里的 Drew Barrymore：聪明绝顶，却对昨天毫无记忆。不知道自己造过什么，甚至不知道自己是干什么用的。这场演讲里的这一整套原语，就是我们递给它的那本笔记本：skill、work log，还有让它能干真活的判断力。我们交付的 PR 能过 CI，并不是因为模型是天才。它们能过，是因为我们搭了一个不允许它出错的 harness。其中每一道闸门，都是由人来拍板的。这就是 harness 工程。要规模化的是判断力，而不是模型。所以，我想请大家做这么一件事。

[18:20] Andrew

Don't just go write a skill. That's only that's only one implementation. Find the judgment your team always asks the same person about and make it explicit somewhere that an agent can reach it. It could be a persona, a lint check or a commit hook. That's the job. Now less product engineering and more harness engineering and the method demonstrates itself. The flag agent produces PRs that pass CI. This talk resumed from a worklog. We started here. Humans absorb judgment implicitly while agents require it explicitly. Self-driving code bases will exist the moment we get that judgment out of our heads and into the codebase. Then we point the agents at the gate and let them spin until it's green. Thank you.

别只是跑去写个 skill，那只是其中一种实现方式。去找出你们团队总要去问同一个人的那种判断，把它显式地写到 agent 够得着的地方。可以是一个 persona、一条 lint 检查，或者一个 commit hook。这就是我们的工作。如今，这份工作里产品工程少了，harness 工程多了，而这套方法自己就能证明自己：清理 flag 的 agent 产出的 PR 能通过 CI；这场演讲是靠一份 work log 接着做完的。我们从这句话开始：人类隐式地吸收判断力，而 agent 要求判断力是显式的。一旦我们把判断力从脑子里搬进代码库，自动驾驶式的代码库就会成为现实。然后我们把 agent 对准闸门，让它们反复打转，直到变绿。谢谢大家。

[19:31]

[music]

（音乐）