Context engineering with Dex Horthy

频道: The Pragmatic Engineer
视频: https://newsletter.pragmaticengineer.com/p/context-engineering-with-dex-horthy
原文语言: en
统计: 共 102 轮 · Gergely Orosz 20 · Dex Horthy 82

[0:00] Gergely Orosz

What happens when you let AI agents ship code for months and no developer reads a single line?Today's guest tried exactly that. He built a lights-off software factory and four months laterhe had no choice but to shut it down as things just stopped working. Dex Horthy is the founderof HumanLayer and the person who coined the term context engineering days before Andrzej Karpathyand Tobi Luzka made it famous. He spent the last two years talking to hundreds of AI engineers aboutwhat actually works when you build with LLMs and is testing the most extreme ideas with his own team.In today's conversation we discuss context engineering, what it is and the physics of context windowsincluding what the dump zone is. Loop engineering, from the Ralph Wiggum technique to the slow loopthat Dex's team runs every night to wake up to code cleanup PRs. The rise of software factories,from a NATO conference in 1968 through DevOps to today's agentic factories. Spec driven developmentand why specs always drift from the code itself. And many more. If you want to understand increasinglyimportant concepts like concept engineering and harness engineering or want to know how far you can pushthe let agents build everything idea from someone who pushed it further than almost anyone, then this

当你放手让 AI agent 连续几个月自己发代码、而没有任何一个开发者读过其中哪怕一行，会发生什么？今天的嘉宾真的这么试了。他造了一座「关灯软件工厂」（lights-off software factory），四个月后不得不把它关停——因为东西就是不转了。Dex Horthy 是 HumanLayer 的创始人，也是「上下文工程」（context engineering）这个词的提出者——比 Andrej Karpathy 和 Tobi Lütke 把它带火早了几天。过去两年，他跟几百位 AI 工程师聊过「用大模型（LLM）做东西时到底什么才真的管用」，同时在自己团队里试验最极端的想法。今天这期我们聊：上下文工程是什么、上下文窗口的「物理学」，包括什么是「笨区」（dumb zone）；循环工程（loop engineering），从 Ralph Wiggum 技法讲到 Dex 团队每晚跑、第二天一早醒来就能收到一堆代码清理 PR 的「慢循环」；软件工厂的崛起，从 1968 年一场北约（NATO）会议、经 DevOps 一路讲到今天的 agent 化工厂；规格驱动开发（spec-driven development），以及为什么规格总会和代码本身漂移。还有很多。如果你想搞懂上下文工程、harness 工程这类越来越重要的概念，或者想从一个「把这件事推得比几乎所有人都远」的人那里，知道「让 agent 造一切」这个想法到底能推到多远，那这期

[1:03] Gergely Orosz

episode is for you. This episode is presented by Antisys. If you work with agents, your job is nolonger just writing code, it's specifying and testing it. And Antisys is the most effective methodof verifying agentic code today. Today's episode is brought to you by BuildKite, the CI orchestrationplatform trusted by OpenAI, Entropic, Cursor, NVIDIA, Uber, Canva and more. Today we're talking aboutpushing the right context into models so that they write better code. Right after that starts working,your agents will write more code. A lot more. Trusting that code avalanche is where many teamsface a challenge today. Every change that an agent makes still has to be built, tested and proven safebefore it ships. Worked on my machine is not enough, so you obviously need CI. But when agents are pushing5, 10 or 50 times the commit volume to your pipelines, faster CI runners won't save you. Shaving 30 secondsoff a single build is meaningless when a queue has 100 plus jobs deep. What you really want is a CI systemthat gets faster as the volume grows, and CI that offers instant parallelization to give you unlimitedconcurrency and to intelligently route changes at runtime. This is what BuildKite does and why global

就是给你听的。本期由 Antithesis 呈现。如果你在和 agent 一起工作，你的工作就不再只是写代码了，而是「把它讲清楚 + 把它测出来」。而 Antithesis 是今天验证 agent 所写代码最有效的方法。今天这期还由 BuildKite 赞助——OpenAI、Anthropic、Cursor、NVIDIA、Uber、Canva 等公司都在用的 CI 编排平台。今天我们聊的是「把对的上下文推进模型里，好让它写出更好的代码」。这件事一旦开始奏效，你的 agent 就会写出更多代码——多得多。而如何信任这场「代码雪崩」，正是很多团队今天面临的挑战。agent 做的每一个改动，上线前仍然要被构建、测试、证明安全。「在我机器上是好的」不够用，所以你显然需要 CI。但当 agent 往你的流水线里推 5 倍、10 倍甚至 50 倍的提交量时，更快的 CI 机器救不了你。队列里堆着 100 多个任务的时候，单次构建省下 30 秒毫无意义。你真正想要的是一套「量越大反而越快」的 CI 系统：提供即时并行、无限并发，并在运行时智能地路由变更。这正是 BuildKite 在做的事，也是为什么全球的

[2:07] Gergely Orosz

software leaders continue to rely on it. The same architecture that observed the scale of Shopify and Ubera decade ago now runs about 1.4 billion job minutes a week across Cursor, Meta, Reddit, and Snowflake.While the rest of the CI world are cracking under the weight of re-architecting their platform,BuildKite continues to reliably grow. Agents running on your infrastructure are BuildKite. Any cloud,any chip, your secrets, your skill. Every artifact and log is captured, so when something fails,either you or your agents have immediate insight for why. As you're entering the context you'll give toyour agents, think about how you'll verify what they hand back. If your system is buckling under theincreased volume, head to buildkite.com slash pragmatic. 30-day all-access trial, no credit card,and an actual human engineer on standby. His name's Ola, and he's very helpful.So Dex, welcome to the podcast.Super stoked to be here, dude.Before we get into some of the context engineering and some of the more spicy stuff as well,how did you get into tech? How did you fall in love with computers?

软件领军公司持续依赖它。十年前扛住 Shopify 和 Uber 规模的同一套架构，如今每周要跑约 14 亿「任务分钟」，服务 Cursor、Meta、Reddit、Snowflake。当 CI 世界的其他玩家还在重构平台、被压得喘不过气时，BuildKite 依然稳稳增长。agent 跑在你自己的基础设施上——任何云、任何芯片、你的密钥、你的技能。每一个产物和日志都被记录下来，所以出问题时，你或你的 agent 都能立刻看清原因。当你在琢磨「要给 agent 什么上下文」时，也顺便想想「怎么验证它交回来的东西」。如果你的系统正被暴涨的量压垮，去 buildkite.com/pragmatic 看看。30 天全功能试用、不用信用卡，还有一位真人工程师随时待命——他叫 Ola，非常热心。

那么，Dex，欢迎来到播客。

太兴奋了，哥们。

在聊上下文工程和一些更「辣」的话题之前——你是怎么进入科技行业的？你是怎么爱上计算机的？

[3:01] Dex Horthy

Oh man, so I was doing undergrad as a physics major, and I realized that I didn't like academia.And there's basically two or three paths out of physics. It's basically you go get a PhD,or you go into finance, or you go do programming. At that time, this was 2012, 2011, when I was in themiddle of undergrad and deciding what to do. And I had done an internship when I was in high school.I was working with NASA researchers to a jet propulsion lab in California. They had justgotten this really high fidelity, like the most, you know, fine-grained data set of altitudes,like the heights of very, like topographical map of the South Pole of the Moon. And the South Poleof the Moon is really interesting because some of the craters there are so deep. Because of the angleit has, it got hit by meteor storms like no other part of the Moon. So there's very deep craters thathave never seen sunlight. And so there's frozen liquid water in there from the formation of theMoon. And so scientists were really interested in getting down there and exploring. And so we hadthis really fine-grained map. And it was like, okay, cool, let's build software so that I havepoint A to point B. I know the limitations of my rover. It can, you know, max incline up is this,

哦天。我本科读的是物理，然后我发现自己不喜欢学术圈。从物理专业出来基本上只有两三条路：要么读博，要么进金融，要么去做编程。那会儿是 2012、2011 年，我本科读到一半、正在决定要干什么。我高中时做过一次实习，跟 NASA 的研究员一起在加州的喷气推进实验室（JPL）干活。他们刚拿到一份保真度极高的数据集——那种粒度极细的高程数据，也就是月球南极的地形图。月球南极特别有意思，因为那儿有些环形山特别深。由于它所处的角度，它被陨石雨砸得比月球上任何其他部位都狠，所以那里有些极深的环形山从来没见过阳光，里面就留着月球形成时期的冻结液态水。科学家们特别想下到那儿去探索。于是我们手上就有这份超精细的地图。当时的任务就是：好，我们来写软件，我有一个 A 点到 B 点，我知道我这台月球车的限制，比如它最大上坡角度是多少、

[4:14] Dex Horthy

max incline down is that. Find a path from point A to point B that doesn't like break those rulesof the incline. So I was, you know, 17. I had never cracked a CS textbook. So I wrote, I basically likewrote a really naive, bad version of Dijkstra's algorithm for pathfinding. So I was in college. Iwas like, I don't know if I want to do the academics thing, but I really enjoyed programmingback in the day. And so, so I decided to go, I got like half of a CS minor and then started workingon a API platform team at a software company in Chicago and Sprout Social, right? Yes. And basically neverwent back. Yeah. And then, and then where did you go from there? Where did you pick up like the partsof the trade? Cause very early on your first job, that's not really common. You were doing platformengineering back in, you know, more than a decade ago. From that point, it took me about two or threemonths to notice that like the most valuable work that was being done in the company was being doneby like, of course, it's obvious like the first couple engineers who know everything and understandwhere everything was. And like you spend a day on a support ticket from a customer and they solve it infive minutes, but like you have to solve it. So you learn and whatever. And I realized like the most

最大下坡角度是多少。那就找一条从 A 到 B、不违反这些坡度规则的路径。当时我 17 岁，从没翻过一本计算机科学（CS）教材，所以我基本上是写了一个非常朴素、很烂的 Dijkstra 最短路算法来做寻路。后来上了大学，我心想：我不确定自己想不想走学术那条路，但我当年是真的很享受编程。于是我修了大概半个 CS 辅修的学分，然后就去了芝加哥一家软件公司的 API 平台团队上班。

是 Sprout Social 吧？

对。然后基本上就再没回头。

那之后你去哪儿了？你是在哪儿把这门手艺的各个部分学到手的？因为你第一份工作就是这样其实挺少见的——十多年前你就在做平台工程了。

从那时候起，大概花了两三个月我就注意到：公司里最有价值的活儿是被谁干的？当然，答案很明显——是最早那几个什么都懂、知道每样东西在哪儿的工程师。你花一整天在一张客户支持工单上，他们五分钟就解决了，但你必须自己去解，才能学会，等等。然后我意识到，最

[5:20] Dex Horthy

valuable people in the company were the people that were building the developer platform,CICD, sandbox environments, preview stuff. And so I kind of like, that was my first step into thejourney. And I've basically been obsessed with software factories since that, like three or sixmonths into my first job. We talk about software factories now, but you're, you're talking aboutsoftware factories back then. So like you were, you're, you were starting to already think that this ishow we can produce better software inside. This is a pre AI world, right?

有价值的人，是那些在建开发者平台的人：CI/CD、沙盒环境、预览环境这些东西。所以那算是我这段旅程的第一步。基本上从我第一份工作的第三到第六个月起，我就对「软件工厂」着了魔。

我们现在都在谈软件工厂，但你那时候就已经在谈软件工厂了。所以你当时就已经开始想「这就是我们在内部造出更好软件的方式」了？那可是个前 AI 的世界，对吧？

[5:49] Dex Horthy

Well, and I'm always surprised, like there's a huge class of developers that say, I don't want to workon CICD. I hate CICD. I'm like, really? Because building the thing that builds the thing and buildingthe thing that builds the thing that builds the thing is like, as software engineers, we're lazy.We want to do the most high leverage thing that makes our job easier. So how do we, if we can build athing that helps us build a thing that helps us move faster, then that's the best use of my time as a lazyengineer. And then you went to another startup, uh, as aspiration aspiration, yeah. Aspiration alsoplatform engineering. Yeah. I was brought in and then like three, three months into the job, the VPof engineering who hired me quit or got fired. I don't know. There was some drama about it. Iprobably shouldn't talk about it. And then I was there for about a year, uh, and was kind of likeacting CTO for a while, like hired a couple of people helped hire the new VP of engineering, but I wasout of there. I don't think I'll ever do consumer again. I think I'm actually a B2B guy.Good to know. And then you went to replicate it where you spent like a good, like, like solid,like four years and went from engineer for the deployed engineer to product manager.

是啊。而且我一直很惊讶：有一大类开发者会说「我不想搞 CI/CD，我讨厌 CI/CD」。我就想：真的吗？因为「造那个造东西的东西」，以及「造那个造出『造东西的东西』的东西」——作为软件工程师，我们是懒的。我们想干杠杆最高、能让自己活儿更轻松的事。所以如果我们能造一个帮我们造东西、让我们跑得更快的东西，那对一个懒工程师来说就是最好的时间投入。

然后你去了另一家创业公司，Aspiration。

Aspiration，对，也是做平台工程。我被招进去，结果入职三个月，招我进来的工程副总裁（VP of Engineering）要么辞职要么被开了，我也不知道，中间有点狗血，我大概不该讲。之后我在那儿待了大概一年，有一段时间算是代理 CTO，招了几个人、也帮着招了新的工程副总裁，但我还是走了。我觉得我这辈子不会再做 to C 了，我发现自己其实是个 B2B 的人。

知道了。然后你去了 Replicated，在那儿待了挺久，扎扎实实四年左右，从工程师做到前线部署工程师（forward deployed engineer）、再做到产品经理。

[6:47] Dex Horthy

Yeah. I did core engineering for like two years that we were building a container orchestrator,like before Kubernetes, before Docker swarm was really a thing. We built our own orchestrator.The founders had this vision that like, Oh, Docker is going to make it much easier to ship on-premsoftware. And when I say on-prem, I don't mean literally like a rack in a colo. It's more like, Hey,look, bring the app to where the data is rather than sending the data up to some cloud vendor.And Docker makes it much more, much easier to package up apps and, and, and, and move themaround. And so they had this thesis that like, basically you could build a platform that theexperience that you get when you use GitHub enterprise, which is like you install it andhas this admin panel, but then you just get GitHub running in your data center and your codenever has to leave your, your data center. Suddenly you could build a generic SAS where everybodycould have that. So I did two years in engineer there. And then our head of sales, we partedour ways with our head of sales. And, uh, honestly, I was having a lot of arguments about the softwarefactory with our CTO. And it's kind of like a, almost like a too many cooks in the kitchen

对。我做了大概两年核心工程，我们当时在造一个容器编排器（container orchestrator）——那是在 Kubernetes 之前、在 Docker Swarm 真正成气候之前，我们自己造了个编排器。创始人们有个愿景：Docker 会让「交付本地部署（on-prem）软件」容易得多。我说 on-prem，不是字面意义上机房里的一个机柜，更像是「把应用送到数据所在的地方，而不是把数据传到某个云厂商那儿」。而 Docker 让打包应用、搬运应用容易得多。所以他们的判断是：你基本上可以造一个平台，把「用 GitHub Enterprise 时的那种体验」——你装上它，它有个管理面板，然后 GitHub 就跑在你自己的数据中心里、你的代码永远不用离开你的数据中心——变成一个通用 SaaS，让人人都能有。所以我在那儿做了两年工程。然后我们跟销售负责人分道扬镳了。而且老实说，我当时跟我们 CTO 在「软件工厂」这件事上吵了很多架。有点像「厨房里厨子太多」

[7:44] Dex Horthy

kind of thing. I'm sure many listeners, listeners have had this experience of like, well, yeah,I know I had these tickets to build, but like CI sucks. I got to fix CI because it's too slow.Or it's like, there's too many different bills and it's always breaking. I'm like, I want to fix that.And then I'm going to do the end is just like Dex, I need you to stop fixing the build pipeline andlike do the tickets I gave you. I'm sure you've had this experience perhaps.Yeah. And then was this what led you to either forward deploy engineering?

那种情况。我相信很多听众都有过这种体验：是，我知道我手上有这些要做的工单，但 CI 太烂了，我得去修 CI，因为它太慢了；或者构建种类太多、老是坏。我说我想去修那个。然后那头的回应就是「Dex，我要你别再修构建流水线了，去做我给你的工单」。我猜你大概也有过这种体验。

有。那这是不是就把你推向了前线部署工程（forward deployed engineering）？

[8:09] Dex Horthy

Yeah. So I like, I really loved our customers, our customer, our customers are Hashi Corp,data stacks, puppet, all these really cool engineering brands, Travis CI, circle CI. I was like,yeah, I actually love working with our customers. Our customers are awesome. And, uh, it was a greatway to like get in the trenches. A lot of really good engineers who were solving the hardestproblem at the company, which is like, how do we take this three to five year old SAS platformand package it all up so that someone who knows nothing about our architecture can run it reliablyin their own AWS VPC, in their own on-prem data center, whatever it was. And so I spent,I was our first kind of customer facing engineer and it was about three months. I, we closed, I met withlike every company customer that was like kind of in the pipeline, but wasn't moving sales wise.And we closed like 12 deals in three months. And the CEO was like, holy crap, Dex, like the,the investors are taking my calls again. Like, I don't, I know you want to get back to coding,but like, I need you to go hire three people and like build this team out. Cause I think you mighthave been like born for this. Wow. Yeah. So I did that for about four years, built that org to like

对。我是真的很爱我们的客户——我们的客户是 HashiCorp、DataStax、Puppet，全是这些特别酷的工程品牌，还有 Travis CI、CircleCI。我当时想：我其实很喜欢跟客户打交道，我们的客户太棒了。而且那是个特别好的「下战壕」的方式：一群非常优秀的工程师，在解公司里最难的问题——怎么把这个已经做了三到五年的 SaaS 平台整个打包起来，让一个对我们架构一无所知的人，能在他自己的 AWS VPC 里、在他自己的本地机房里，稳定地把它跑起来。所以我成了公司第一个面向客户的工程师，干了大概三个月。我把销售管道里那些「卡住不动」的公司客户几乎见了个遍，三个月内成了 12 单。CEO 就说：「我的天，Dex，投资人又开始接我电话了。我知道你想回去写代码，但我需要你去招三个人、把这个团队搭起来，因为我觉得你可能就是天生干这个的。」

哇。

对。所以我干了大概四年，把那个组织带到了

[9:10] Dex Horthy

25 people. And then Zerp happened and, uh, it got a lot smaller and we kind of realized like,Hey, we have a product that's like pretty good. Uh, and we've been solving what lots of earlystartups do is like, okay, there's some usability issues. We'll throw, we'll get a bunch of smartpeople, throw them in the trenches with our customers. Great for sales, great for retention,all this stuff. And it was like, Oh, we actually like the margins on that aren't, aren't good enough.And so we basically were like, cool. We actually just need to make the product way more usable,do a more PLG shaped thing.PLG meaning product-led growth?

25 个人。然后 ZIRP（零利率时代）结束了，团队缩小了不少。我们也慢慢意识到：嘿，我们的产品其实挺不错的。我们一直在解的问题，跟很多早期创业公司一样——好，产品有一些易用性问题，那我们就找一堆聪明人扔进客户的战壕里。这对销售好、对留存好，什么都好。然后发现：哦，其实这么干的毛利不够好。所以我们基本上就说：行，我们真正需要的是把产品做得好用得多，做成更 PLG 形态的东西。

PLG 是指产品驱动增长（product-led growth）？

[9:37] Dex Horthy

Product-led growth. Make a little more self-service so you don't need an expert to teach you how to use it.And I was like, cool. If that's the most important thing that I want to go be a productmanager. Cause I have tons of opinions. I've now spent four years in the trenches with our customers.I have a laundry list of roadmap things that I think would make the product way easier to useand adopt and implement and deploy.And now you went to the full, you went towards a dark side.Exactly. Yeah, I did. I was like, this is going to kill my street cred, isn't it? But, uh, I wasreally glad, you know, I think a lot of engineers are afraid that if they go do a customer facingthing, they lose all their credibility. And like, yes, I wasn't coding for 10 hours a day. I wascoding for like three or four hours on a Saturday for fun. Not, uh, but I mean, we were helpingpeople build YAML. We were building CLIs. We owned a lot of the tooling that customers use.But it was like the last mile delivery side of it, not the core platform. And like ona more personal note, I had spent the last, like most of my twenties feeling like, okay,a little bit introverted, a little bit like socially awkward. What I, what a lot of engineers

产品驱动增长。做得更自助一点，不需要一个专家来教你怎么用。我当时想：好啊，如果这是最重要的事，那我要去当产品经理。因为我有一大堆看法——我已经在客户的战壕里泡了四年，我攒了一长串路线图上的东西，我觉得那些能让产品好用得多、更容易被采纳、实施和部署。

所以你走向了黑暗面。

没错，我确实走了。我当时想：这会毁了我的「街头信誉」（street cred）吧？但我其实挺庆幸的。我觉得很多工程师害怕，一旦去做面向客户的事，就会把所有技术公信力丢光。确实，我不再一天写十小时代码了，我变成周六为了好玩写三四个小时。不过，我们当时是在帮人写 YAML、在做命令行工具（CLI），客户用的很多工具链都是我们负责的。只是那属于「最后一公里交付」那一侧，不是核心平台。还有更私人的一面：我二十几岁大部分时间都觉得自己有点内向、有点社交笨拙——我相信很多工程师

[10:31] Dex Horthy

I'm sure experience. And, uh, I had talked to my uncle's a music producer. So he used towork with like Randy Newman and a bunch of like really famous musicians.Oh wow.Yeah. This guy Mitchell Froome. And he, he, I was sitting with dinner with him at some point.And when I was, I think it was when I was still in undergrad, but he gave me this lecture. He'sbasically like, if you want to be really good at something, you have to make it the only thing youdo. The guy playing guitar nights and weekends, trying to get his band off, off the ground willprobably never achieve greatness. The people who become great are the people who basically make itlike if I don't play guitar, I don't eat. And you go and you sit on the street all day and you playfor 14 hours a day or whatever it is. That's the only way to become great. And so I said, okay,instead of trying to like read self hub books about how to be less introverted and less sociallyawkward, like what if I just made it my fricking job to just talk to people and make friends andlike help people and solve their problems. And, uh, I think it worked out. I recommend it. I thinkeveryone should spend a year or two at least doing something really like customer facing.

都有这种体验。我叔叔是个音乐制作人，他跟 Randy Newman 还有一堆很有名的音乐人合作过。

哇。

对，这人叫 Mitchell Froom。有一次我跟他吃饭——我想那时我还在读本科——他给我上了一课。他大意是说：如果你想在某件事上做到非常好，你就必须让它成为你唯一做的事。那种晚上和周末弹吉他、想把自己乐队搞起来的人，大概率永远达不到卓越。真正变得伟大的人，基本上是把它变成「我不弹吉他就没饭吃」的人——你坐在街上弹一整天，一天弹 14 个小时之类的。那是变伟大的唯一途径。所以我说：好，那与其去读那些「怎么变得不那么内向、不那么社交笨拙」的自助书，不如我干脆把「跟人说话、交朋友、帮人解决问题」变成我他妈的工作本身。我觉得这招奏效了，我推荐。我觉得每个人至少都该花一两年去做真正面向客户的事。

[11:30] Dex Horthy

Did you do this because you felt that it was holding you back being introverted or like what,what, what, and I know you got the motivation from the whole musician motivation. I get it on onepart, but what was it that you said like, all right, is the customer facing thing that I'm going to bedoing it? Cause clearly you were pretty great at like writing code by that point. You could argueyou were doing it night and day. So where did you find that? Like, I actually, I think like customerfacing or like getting this introvert off of me, did you feel that I was holding you back or you justwanted to be good at it? It was just kind of a thing that was like interfering with my like generallife satisfaction. And it was also like, I'm not a very type a person. I'm very disorganized. I don't knowpeople call it like, okay, I'm like ADHD now. That's why I can run 30 clods in parallel orwhatever it is. But it was like, I was really bad at email and calendars and spreadsheets.I was like, didn't care about these, didn't understand them. And so like another side effectof this was like, it just forced me to be organized and keep a lot of things going. And so like,I don't know, there's like weird benefits you get from like stepping outside your comfort zone and

你这么做，是因为你觉得内向拖了你后腿吗？还是说……我知道你从那位音乐人那儿拿到了动机，这我懂。但到底是什么让你说「行，就是这件面向客户的事，我要去做」？因为很明显那时候你写代码已经相当厉害了，你甚至可以说你一直在没日没夜地写。所以你是从哪儿找到那个点的？是你觉得「面向客户」或者「把内向这层壳去掉」在拖你后腿，还是你只是想在这件事上做得好？

它就是那种会干扰我整体生活满意度的东西。而且还有一点：我不是那种典型的 A 型人格，我很没条理。我不知道，现在人们会说「哦这是 ADHD（注意力缺陷多动）」，所以我才能同时跑 30 个 Claude 之类的。但当时就是——我特别不擅长邮件、日历、表格，我不在乎这些，也搞不懂它们。所以这件事的另一个副作用是：它逼着我变得有条理、同时推着一堆事往前走。所以我觉得，走出舒适区、去学一门

[12:25] Dex Horthy

learning like industrial disciplines that are separate from what you've been doing. And so theopportunity presented itself and I was like, oh, I like working. I'll try this for a little bit.Started going really well. I'm like, cool. Let's keep, let's see, let's see how far this thread goes.And then afterwards you're now in your second startup, you, you became a founder and you alsogot involved in, in AI pretty early as I, as it was even before it was so obvious that it wouldchange how it would change how we develop software. Right. Well, I would say I was, I was later thanI could have been because we started the company, uh, me and a buddy in Chicago started a company inthe data engineering space in about 2020, November, 2020, we decided in like August of 2020.This is metalytics.Metalytics. Um, technically still the same company as human layer.We just like pivoted the, the, the mission, but, uh, yeah, the, the, the advice I got fromevery angel investor that, you know, people who just knew CTOs I'd worked for before and stuff,they were just like, look, hitting a lot of heads wins. And I don't know if you know, likethe whole DBT data engineering, five trend, that whole arc where it was like this huge party and

跟你原本做的事完全不同的行业纪律，会带来一些很奇怪的好处。所以机会摆在那儿的时候我就想：哦，我喜欢工作，那试试看吧。结果跑得挺好，我就想：行，接着走，看看这条线能拉多远。

再往后你就到了第二家创业公司，你成了创始人，而且你很早就卷进了 AI——那还是在「AI 会怎么改变我们写软件」这件事变得如此显而易见之前。

嗯，我得说我其实比我本可以的时间要晚。我和一个芝加哥的哥们儿开了这家公司，做数据工程方向，大概 2020 年 11 月开的，2020 年 8 月左右决定的。

这是 Metalytics。

Metalytics。严格说它跟 HumanLayer 还是同一家公司，我们只是把使命转了向。不过，我从每一位天使投资人那儿拿到的建议——都是些认识我以前老板 CTO 的人——他们就说：你看，往人多势头大的地方撞就是能赢。不知道你了不了解 dbt 那波数据工程的趋势，整个那条弧线：一开始是场巨大的派对、

[13:24] Dex Horthy

tons of investor money going into all these different companies. And then within by like 2021,2022, there was kind of the Zerp thing. And just this general realization that the TAM forthose sorts of tools is not as big as everyone thought it was.Total addressable market.Yes. The total addressable market for those sorts of tools was, was not as quite as big as, uh,as we all thought it was. Um, so it was a, it was a hard place to raise money. It was a hard placeto get customers.Yeah. And then I met you at, well, while you were at human layer NSF at an event, we,you actually talked and we chatted afterwards, but by, by that, this was about a year ago,you were already, you, you started to have some really strong opinions on using AI. And one ofthem was this now famous 12 factor agents manifesto.Is it, are we calling it a manifesto now?

海量投资人的钱涌进各种各样的公司。然后到了 2021、2022 年，一方面是零利率时代（ZIRP）结束，另一方面大家普遍意识到：这类工具的 TAM 没有所有人以为的那么大。

总可寻址市场（total addressable market）。

对。这类工具的总可寻址市场，并没有我们所有人以为的那么大。所以那是个很难融资的地方，也是个很难拿客户的地方。

是。后来我在旧金山的一个活动上见到你，那时你已经在做 HumanLayer 了。我们当时聊过，之后也接着聊。但那大概是一年前，你那时候对「怎么用 AI」已经有一些非常鲜明的看法了。其中一条就是现在很有名的「12-factor agents」宣言。

我们现在管它叫宣言了吗？

[14:09] Dex Horthy

I'm calling it a manifesto. It's a manifesto. I'm calling it. Let's talk about this. This was 12engineering principles to build a reliable production ready apps. Uh, how did you comeup with this? And maybe we can also talk about some of them.Yeah. So, um, I'll, I'll kind of like go to like around August, the co-founder I was working with,the kind of burned out and left, and it was very, we're on good terms. It was very mutual. Um,and I decided to just start messing with AI stuff and I was building AI agents. And what was reallyin vogue right then was like the Lang chain, the crew AI, these like agent frameworks. Um, and itseemed like there was a ton of, you go on, you go on the crew AI discord, there's 10,000 people.It's like, okay, this feels like the right shape. And this, there's clearly this eco you're goingto every single one of those projects. They have a chroma DB plugin. They have like a composio plugin.There's like, clearly like, this is the, this is the shared interface that everybody is buildingfor. I said, okay, what's missing from all of this? The agents can call tools, but it's reallyhard to like control which tools they call. And if it's a chat bot, obviously you can show approved

我管它叫宣言。它就是宣言，我说了算。我们来聊聊这个。它是 12 条「造出可靠、可上生产的 agent 应用」的工程原则。你是怎么想出来的？也许我们还可以聊聊其中几条。

好。我先把时间拉回到大概八月：跟我一起干的联合创始人有点烧干了、离开了——我们关系很好，是很和平的分手。然后我决定就开始瞎折腾 AI 的东西，我在造 AI agent。那会儿最流行的是 LangChain、CrewAI 这类 agent 框架。看上去有巨大的势能——你上 CrewAI 的 Discord，里面有一万人。我想：好，这个形状看着是对的。而且很明显有个生态：你去看那里面每一个项目，它们都有一个 Chroma DB 插件、都有一个 Composio 插件。很明显，这就是所有人都在为之构建的共享接口。我就问：这一整套里缺了什么？agent 能调工具，但要控制它调哪些工具真的很难。如果是聊天机器人，你当然可以在应用界面里显示「批准 /

[15:07] Dex Horthy

deny in the UI of your application. But I kind of was obsessed with what I would call like outer loopagents or proactive agents that would run in the background, get triggered by events. I mean,open clause, basically like the biggest manifestation of this, of like, you have a heartbeat, it wakesup, it sees if there's any work to do. It tries to do stuff. And my thought was like, I'm not goingto trust that agent to do anything meaningful. If I can't get like a Slack message or an I messageor something when it wants to do something and kind of guaranteed deterministically that I can approveor deny that or deny it with feedback and say, actually, no, do it like this. So we played inthat space for a while and talked to a lot of founders and founding engineers and builders.We came and did YC in the fall of 2024 with this idea. We're building out this API platform and itwas sort of like pager duty, but like, it wasn't who's on call to fix the servers. It was like,who's on call to this like routing mechanism for like, who needs to approve this agent? And can theylike escalate it or delegate it or defer it? All this stuff. And we built it for this ecosystem.Great. I like chain five. There's so many grip tape. There was so many in that, in that time.

拒绝」。但我当时着魔于我称之为「外循环 agent」（outer loop agents）或者说「主动型 agent」的东西——它们在后台跑、被事件触发。像 OpenClaw 基本上就是这件事最大的体现：你有一个心跳，它醒来，看看有没有活要干，然后试着去干。而我的想法是：如果我不能在它想做某件事的时候收到一条 Slack 消息或者 iMessage，并且能确定性地保证我可以批准、拒绝、或者带着反馈拒绝它说「不，其实应该这么干」，那我不会信任这个 agent 去做任何有分量的事。所以我们在那个空间里玩了一阵，跟很多创始人、创始工程师和构建者聊过。2024 年秋天我们带着这个想法去了 YC。我们在造一个 API 平台，它有点像 PagerDuty，但不是「谁值班去修服务器」，而是「谁值班来批准这个 agent」——这样一套路由机制：谁需要批准这件事、他能不能升级、转派或延后，等等。我们是为那个生态造的。挺好。我喜欢 LangChain，还有 Griptape，那阵子这类框架特别多。

[16:10] Dex Horthy

And then I talked to tons of AI engineers who were actually building really interesting thingsand like actually making money doing six figure contracts, shipping AI to the enterprise.And all of them had tried that stuff for like a month or two. And then they had thrown it outand they were just writing all the API calls by hand. And they were building more things that lookmore like pipelines and workflows than these sort of like hands-off call tools in a loop kind of thing.And so I talked to a hundred people and I spent a lot of time, a lot, a lot of time hanging out withone of my best friends, uh, vibe of from a boundary. So they built a programming, they builtlike this, like proto buffs for AI thing. And they're, I think they're about to launch theirlike full fat, like programming language Turing complete thing. But he had this way of thinkingabout agents and building with models and building with inference where it was a lot more aboutunderstanding what structured output really is under the hood.And every single step in your AI workflow is just tokens in tokens out. And your job as an engineeris figure out, okay, what tokens do I need to put in to maximize the chance that the tokens out are

然后我跟一大堆 AI 工程师聊过，他们是真的在做很有意思的东西、真的在赚钱、签六位数的合同、把 AI 交付给企业。而所有这些人，都试过那套框架一两个月，然后就把它扔了，改成自己手写所有的 API 调用。他们造出来的东西，更像流水线（pipeline）和工作流（workflow），而不是那种撒手不管的「在循环里调工具」。所以我跟一百个人聊过，还花了非常非常多的时间，跟我最好的朋友之一——Boundary 的 Vibhav——泡在一起。他们造了一个东西，本质上是「AI 界的 Protocol Buffers」。我想他们快要发布完整版的、图灵完备的编程语言了。他有一套思考 agent、思考「用模型构建」「用推理构建」的方式，重点更多放在「结构化输出（structured output）在底层到底是什么」。你 AI 工作流里的每一步，本质上都只是 token 进、token 出。而你作为工程师的工作，就是搞清楚：我要放什么 token 进去，才能最大化「输出的 token 是好的」

[17:09] Gergely Orosz

going to be good. And kind of distilled all these ideas into about 12 principles and wrote about iton GitHub, posted just like this, like 12 page GitHub repo, threw it on Hacker News, got like 500,it was on the front page for like two days. And it, I think it really resonated with a lot of people.Yeah. So I'll just quickly read the 12 principles and then let's talk about like one or two thatresonate. So the 12 are natural language of tool calls, own your prompts, own your context window,tools are just structured outputs, unify execution state and business state, launch, pause, resumewith simple APIs, contact humans with tool calls, own your control flow, compact errors into contextwindow, small focused agents, trigger from anywhere, meet user where they are, make your agent astateless reducer. Haha. The stateless, yeah. The stateless reducer one was a little, actually,someone hit me up on Twitter and corrected me. It's actually, it's actually a transducer becausethere's technically multiple steps in the workflow, but there we go. But, but, but of this one, this,this was a year ago. So like, which is like forever and, and, and how the tooling is, is evolving,which ones still stick with you or if you're like, all right, these were good that, that still

这个概率。我把这些想法蒸馏成了大约 12 条原则，写在 GitHub 上，就发了这么一个 12 页的 GitHub 仓库，扔到 Hacker News 上，拿了 500 多分，在首页挂了两天。我觉得它确实戳中了很多人。

好，我快速把这 12 条原则念一遍，然后我们挑一两条你还有共鸣的聊。这 12 条是：自然语言到工具调用、拥有你的提示词、拥有你的上下文窗口、工具只是结构化输出、统一执行状态与业务状态、用简单 API 启动 / 暂停 / 恢复、用工具调用联系人类、拥有你的控制流、把错误压缩进上下文窗口、小而专注的 agent、可从任何地方触发、在用户所在之处与他相遇、把你的 agent 做成无状态归约器（stateless reducer）。哈哈，无状态归约器。

对，无状态归约器那条其实……有人在 Twitter 上找我纠正说：严格讲那应该叫 transducer（转换器），因为工作流里技术上有多个步骤。不过就这样吧。

但在这些里面——这可是一年前了，一年在这行简直是一辈子，工具链一直在变——哪几条你现在还留着？或者说哪几条

[18:17] Dex Horthy

seemed to hold off. Yeah. I think I spent most of March writing it, published this in April. Uh,and then Swix hit me up from AI.engineer and he said, Hey, can you come to, you want to come talkabout this? So I gave this talk, 12 factor agents in like June 6th, I think. And, uh, small room,maybe like it was packed, but it was like maybe a hundred people. That was the year at AI engineerwhere like the lower physically, like on the, on the second basement floor was all the super corporatestuff. And you go up a level, it's a little bit more. And then like on the top floor is all the likeweird cutting edge, like startup stuff that like, you probably shouldn't care about yet kind of thing.So we were up there on the top of this like weird way of thinking about agents. Uh, and then about aweek later, two weeks later, uh, Toby looky from Shopify. He says, I really liked this idea of likecontext engineering. And I'm like, I, I wrote about this two months ago. This is great. Toby gets it.And then a week later, Andre Karpathy is like, well, I really like, I think what we should think aboutis not prompt engineering, but context engineering. I was like, yes, that's my, anyways, I don't know.If you ask Gemini depends what day it is. They will tell you either me or Toby or Andre came up

看起来还站得住？

对。我记得我大部分是三月在写，四月发的。然后 AI.Engineer 的 Swyx 找我，说「嘿，你要不要来讲讲这个？」于是我在大概 6 月 6 日做了那场「12-factor agents」的演讲。房间很小，虽然挤满了，但大概也就一百人。那一年的 AI Engineer 大会是这样布局的：物理上最下面、地下二层全是特别企业范儿的东西；往上一层就稍微前沿一点；再到顶楼，全是那些怪异的、最前沿的创业公司的东西，那种「你现在大概还不用管」的东西。我们就在顶楼，讲这套怪异的思考 agent 的方式。然后大概一周、两周之后，Shopify 的 Tobi Lütke 说他非常喜欢「上下文工程」这个想法。我心想：我两个月前就写过这个，太好了，Tobi 懂。再过一周，Andrej Karpathy 说：我觉得我们该想的不是提示词工程（prompt engineering），而是上下文工程。我心想：对，那是我的……唉，算了。你要是去问 Gemini，看你哪天问，它会告诉你上下文工程是我、是 Tobi、或者是 Andrej 提出的。

[19:19] Dex Horthy

with context engineering. You can't really own a word. Like I don't, no one remembers who inventedthe word prompt engineering, but of all the factors, factor three of own your context window.And basically the only way you can, whether it's agentic or a single step at a pipeline,the only way you can impact the quality of your output from AI is by caring a lot about what theinputs and crafting them. So let's talk about context engineering, which I am going to credit youthat you coined it. I did some research and like, I think you were earlier, but a few days. So therewe go. You, you coined it. We're adding, we're adding to the, we're adding to SEO juice. We'llhave it in transcript decks coin context engineering. And, and like a asterisk on that is basically like,I learned about context engineering from talking to these hundred engineers and founders. I justkind of like, what was the same about what they were all doing? And I put a name on it. So like,I didn't invent doing it. I was just like, I think, I think there's this thing and likevocabulary and names are really important and having like clean ways to talk about the problem,especially when like a lot of the content about AI right now is so much hype and jargon that is

你没法真的「拥有」一个词。我是说，没人记得「提示词工程」这个词是谁发明的。不过在这 12 条里，第三条「拥有你的上下文窗口」——本质上是说：不管你是 agent 化的，还是流水线里的单步调用，你唯一能影响 AI 输出质量的方式，就是极其在意输入、并且精心打磨输入。

那我们就来聊上下文工程。我要把这个词记在你名下——我查过资料，我觉得你确实更早，虽然只早了几天。所以就是你提出的。我们这是在给 SEO 加权：我们会在转录稿里写上「Dex 提出了上下文工程」。

这件事得加个星号：我是从跟这一百位工程师和创始人聊天里学到上下文工程的。我基本上就是看「他们所有人做的事里，哪一部分是相同的」，然后给它取了个名字。所以我不是发明了这件事的做法，我只是觉得「有这么个东西」，而且词汇和名字真的很重要，有干净的方式来谈论问题很重要——尤其是现在关于 AI 的内容里，有那么多炒作和黑话是

[20:17] Dex Horthy

like meaningless. I was like, okay, I think there's a word here that is useful to builders that explainshow they should be thinking about building their software. So what is context engineering?It's kind of like de-abstracting a lot of the abstractions that have been layered on top. So youhave a rag, you have memory, you have agentic history, you have structured output, you have all thesethings that are like different ideas in the frame of agentic programming. And at the end of the day,they're all like different ways to pass tokens into a model and ask it to produce usually somestructured output. And understanding that is a lot more powerful than trying to learn memory and tryingto pick some agent framework off the shelf and some memory framework off the shelf. I mean, those are,these things are all really good. If you want to get to like 80%, you want to get a really good demo,but when you have to go from 80% to 95% or 99%, you need to go down a level and think about what'severything we're putting into the context window. What order is it going in depending on which modelwe're doing and all of this stuff matters. You have all of these levers that you can pulland it just felt like the right abstraction for thinking about how do I get AI to do the thing I

毫无意义的。我当时想：好，我觉得这里有一个对构建者有用的词，它能解释他们该怎么思考「造自己的软件」这件事。

那么，什么是上下文工程？

它有点像是把一层层叠上去的抽象重新「去抽象化」。你有 RAG（检索增强生成），你有记忆（memory），你有 agent 历史，你有结构化输出，你有这一堆在 agent 编程框架里各不相同的概念。可到最后，它们全都只是「把 token 传进模型、让它产出通常是某种结构化输出」的不同方式。理解这一点，比去学「记忆」、比去货架上挑一个 agent 框架再挑一个记忆框架，要有力得多。我是说，这些东西都挺好——如果你想做到 80%、想做个很不错的 demo。但当你必须从 80% 走到 95% 或 99% 时，你就得往下沉一层，去想：我们放进上下文窗口的每一样东西到底是什么？根据用的是哪个模型，它们进去的顺序是什么？所有这些都有影响。你手上有一堆可以拉的杠杆。它就是感觉是那个对的抽象层——用来思考「我怎么让 AI 尽可能准确地做我想要

[21:25] Gergely Orosz

want as accurately as possible.Why is context engineering started to become more, more talked about? It was about a year ago. Was it,did it have to do with the context, the context window that we could pass onto LLMs pretty much?

的事」。

为什么上下文工程开始被越来越多地讨论？那大概是一年前。这跟我们能传给大模型的上下文窗口有关系吗？

[21:38] Dex Horthy

Did it start to expand or did we just start to realize that we can do a lot more by passing onfrom, you know, the easiest one is of course system prompts, but of course, whenever you build an LLMbehind the scenes, you will pass additional context as well. Not just to prompt the user,you will add a bunch of subdesk, I guess, a dirty secret of any LLM. But why do you think the focusis moving on to like, all right, context is important? I think it always was important.I think what had to happen is a ton of smart people, again, like all these builders I talkedto, a ton of smart people had to like focus really hard on producing. Like I want to makesoftware that I can sell. I want to make something that's accurate enough that I'm proud of and I cansell to an enterprise and they're going to be happy with it. And there's just like the easiest way toget to really high quality AI applications is by thinking at that token level, thinking abouta string of different LLM calls, like rather than just tools in a loop and it's kind of open-endedand very flexible, but not that reliable. Thinking of agents as workflows, as pipelines,as some mix between maybe a couple tools in a loop versus just, hey, I have my tools and I have my model

是上下文窗口开始变大了，还是我们只是开始意识到「多传一些东西进去能做的事多得多」？最简单的当然是系统提示词（system prompt），但显然，你造任何一个基于大模型的东西，幕后都会传额外的上下文进去——不只是用户的提示，你还会加一堆子任务，我猜这算是所有大模型应用的一个「肮脏的小秘密」。但你觉得为什么焦点现在移到了「好吧，上下文很重要」上？

我觉得它一直都很重要。我觉得必须先发生的事情是：一大批聪明人——还是我说的那些构建者——必须非常用力地聚焦在「产出」上。就是「我想做出我能卖的软件，我想做出准确到让我自己骄傲、能卖给企业并且让他们满意的东西」。而做出真正高质量 AI 应用最简单的路径，就是在 token 这一层去想：想清楚一串不同的大模型调用，而不是只有「工具在循环里跑」——那种很开放、很灵活但没那么可靠的形态。把 agent 当成工作流、当成流水线，或者是某种混合体：也许有几个工具在循环里，而不是只有「我有我的工具、我有我的模型

[22:46] Dex Horthy

and I have my system prompt and these are the only levers I have. And it's actually, no, you have waymore levers. It's going to take more work and you're going to have to like understand the LLMwith a deeper intuition. But it was a thing that we always needed and it just took time for peopleto build with this technology to figure out that like this is the layer of abstraction that allowsyou to break through the quality ceiling. And how are cost and context engineering connected?

、我有我的系统提示词，我就这三根杠杆」。其实不是的，你手上的杠杆多得多。这要花更多功夫，你也得对大模型建立更深的直觉。但这是我们一直都需要的东西，只是需要时间让人们用这项技术造东西，才能想明白：这就是那个让你突破质量天花板的抽象层。

那成本和上下文工程之间是什么关系？

[23:11] Dex Horthy

Yeah. I don't know. I was talking about this with someone this morning about like when you'reworking with LLMs, one of the things I like to say is kind of like make it run, make it right,make it fast. See if the world's best LLM at the time, I think we did a podcast episode that at thetime it was like O3. See if O3 can solve your problem and then give it to people and see ifthey want that. And then if people want it and you use it a lot, then go do a bunch of contextengineering because your engineering time is always the bottleneck. Like humans trying to figure outand solve problems and build evals and improve and try different dimensions or set up JEPA or whatever it is,is always going to be more expensive than just using a smarter model until you have millions ofrequests a day. And then it's like, okay, we're going to do a bunch of context engineering, breakthis up into three calls and get it to work on GPT-40. And then we're going to take two of thoseand make those two work on GPT-40 and I'm using old model names. But the point is like for a certaintask in your workflow, can you get GPT-OSS-120b, which is like one one thousandth of the cost of Opus?

是。我今天早上还跟人聊这个。跟大模型打交道时，我喜欢说的一句话是「先让它跑起来，再让它跑对，最后让它跑快」（make it run, make it right, make it fast）。先看看当时世界上最强的模型——我记得我们做过一期播客，那时候是 o3——看看 o3 能不能解你的问题，然后把它交给用户，看看他们要不要。如果人们要、而且用得很多，那你再去做一堆上下文工程。因为你的工程时间永远是瓶颈：人去琢磨、去解决问题、去建评测（evals）、去改进、去尝试不同维度、去搭 JEPA 什么的，永远比「直接用一个更聪明的模型」贵——直到你每天有几百万次请求。到那时候才是：好，我们来做一堆上下文工程，把这个拆成三次调用，让它能在便宜些的模型上跑；然后我们再把其中两次拿出来，让它们在更小的模型上跑。我在用老的模型名字了，但意思是：对你工作流里的某个特定任务，你能不能用 GPT-OSS-120B——成本大概是 Opus 的千分之一——

[24:10] Dex Horthy

Can you get it to solve parts of the problem so that the tokens and the things you're using thesmartest frontier models for are just the things that you really need that level of intelligence?But you shouldn't go build all of that and over-engineer it until you've proved that youneed it, that it's valuable, that it's like, okay, this is now, I'm going to get to Eli Goldrattand like what is the, he had this book, The Goal, right? It was about how to model your factory.And I'm sure we'll get to that when we talk about software factories. It was like, what is thebottleneck in your system? And one day it will be latency and cost, but it's probably not that when youfirst start out. And context engineering is how you move from the, you add human effort to theequation to improve the efficiency, the speed, the price, the cost efficiency of your system.Interesting. And then one thing that came up more recently and a lot later,as soon as recently, is harness engineering. What is harness engineering?

来解掉问题的一部分，好让你花在最聪明的前沿模型上的 token 和事情，只剩下那些你真正需要那个智能水平的部分？但你不该在证明「你需要它、它有价值」之前，就把这一整套造出来、过度工程化。这就要说到 Eli Goldratt（艾利·高德拉特）了——他有本书叫《目标》（The Goal），讲的是怎么给你的工厂建模。我相信我们聊软件工厂的时候还会回到这本书。它问的是：你系统里的瓶颈在哪儿？总有一天瓶颈会是延迟和成本，但你刚开始的时候多半不是。而上下文工程，就是你往这个等式里加入「人的努力」，去提升系统的效率、速度、价格和成本效益的方式。

有意思。那还有一个最近才冒出来的词，叫 harness 工程（harness engineering）。什么是 harness 工程？

[24:58] Dex Horthy

So I made a post in like October, I think about, or maybe November of like, hey, there's this newthing that I see is like, I'm calling it harness engineering. My definition that I had at the timeis not what actually this guy Viv, who's at Langchain now does a lot of really good writing onagents and how to think about harness. And he had written something called harness engineering, likea couple of weeks before me, but I hadn't read it at that point. And my take was basically like, okay,when you build an agent, you use context engineering. When you use an agent, because we gave this talk inAugust of 2025 about like how to apply context engineering to how you use coding agents.And that kind of evolved into this idea of like, how do you take a harness like CloudCode, like Codex?

我大概在十月、也可能是十一月发过一个帖子，说：嘿，我看到有个新东西，我管它叫 harness 工程。我当时给的定义，跟这个词现在的实际含义不太一样。有个叫 Viv 的人，现在在 LangChain，他写了很多关于 agent 和「怎么理解 harness」的好文章。他在我之前几周就写过一篇叫 harness engineering 的东西，只是我当时还没读到。我的看法基本上是：当你「造」一个 agent 时，你用的是上下文工程；而当你「用」一个 agent 时——因为我们 2025 年 8 月做过一场讲「怎么把上下文工程应用到你使用编码 agent 的方式上」的演讲——那件事后来演化成了这个想法：你怎么对待像 Claude Code、像 Codex 这样的 harness（agent 的外壳 / 脚手架）？

[25:39] Dex Horthy

How do you engineer against the integration points of that harness? So commands, MCPs, skills,how you organize your code base? How do you kind of optimize the environment that the coding agent runsin to like get the best results? The same way with context engineer, how do you optimize the inputs toevery single prompt? Well, harness engineering just is like, how do I raise the floor so that every singleturn of this thing, the results are as good as possible? And the term got super blurry. And somepeople think harness engineering means building a harness. And some people think harness engineeringmeans building around a harness. I actually like what Martin Fowler came up with. As usual, he's verygood at naming things. And he kind of defined the, you have the LLM, and then you have the inner harness,which is like the thing that the tool definitions and the integration points that like, say, like aCloudCode or a Codex or a AMP actually exposes, that's your inner harness. And then you have theouter harness, which is the stuff that you, the human do to customize that for your specific needs,your code base, your languages, et cetera. That's the best definition I think we have for harnessengineering. It's interesting how naming is still so important, isn't it?

你怎么针对这个 harness 的集成点去做工程？也就是命令（commands）、MCP、技能（skills）、你怎么组织你的代码库？你怎么优化编码 agent 运行所在的那个环境，好拿到最好的结果？这跟上下文工程里「怎么优化每一个提示词的输入」是一回事。而 harness 工程就是：我怎么把地板抬高，让这东西的每一轮结果都尽可能好？

这个词后来变得特别模糊。有些人认为 harness 工程是「造一个 harness」，有些人认为是「围着一个 harness 造东西」。我其实很喜欢 Martin Fowler 给的说法——他一如既往地特别会取名字。他的定义是：你有大模型；然后你有「内层 harness」，也就是像 Claude Code、Codex 或 AMP 实际暴露出来的那些工具定义和集成点，那是你的内层 harness；再往外你有「外层 harness」，也就是你这个人为了自己的具体需求、你的代码库、你用的语言等等去做的那些定制。我觉得那是我们目前对 harness 工程最好的定义。

有意思，取名字到现在还是这么重要，是吧？

[26:44] Dex Horthy

Well, it's like, and as soon as you name anything, people are, most people are, I'm actually surprisedthat context engineering still means the same thing to most people that it did a year ago. And thatit's even still relevant. Like that's honestly the craziest thing to me is like you wrote,how many things that were written about AI 15 months ago still matter or are still interestingor are still like have good advice baked into them. Stuff changes a lot. I think context engineeringhas been so long lived because it's, it's grounded in the fundamentals of how transformer attentionworks. And until we have post transformer models or linear attention or whatever it is,which who knows when that's going to happen, context engineering will be interesting andimportant to anyone building on AI.Can we talk about the physics of, of context? Uh, you, you, you had a, you had a tweet, uh, this,this one, the, the context reality check. This is a graph of, uh, as you get to 1 million context,just the, the quality just drops, it goes down. What do we need to know about like the context?

嗯，而且只要你给任何东西取了名字，大多数人就会……说实话我挺惊讶的：上下文工程对大多数人来说，到今天含义还跟一年前一样，而且它居然还有意义。这对我来说才是最疯狂的事——你想想，15 个月前写的关于 AI 的东西，还有多少今天还重要、还有意思、里面的建议还站得住？东西变得太快了。我觉得上下文工程之所以这么长寿，是因为它扎根在「Transformer 注意力机制怎么工作」这个基本面上。除非我们有了后 Transformer 模型、线性注意力之类的东西——天知道那什么时候会发生——否则上下文工程对任何在 AI 上做东西的人，都会一直有意思、一直重要。

我们能聊聊上下文的「物理学」吗？你发过一条推，叫「上下文现实检查」（the context reality check）。那是一张图：当你逼近 100 万 token 上下文时，质量就一路往下掉。关于上下文，我们需要知道些什么？

[27:45] Dex Horthy

Again, we, we now have models that do have a 1 million context window. Maybe we'll have evenlonger ones, but when you start to just put in more stuff into the context, it starts to becomeless efficient. Like what, what do we know so far in terms of, from the practical perspective of likesomeone who is using the context window to add on a bunch of stuff, may that be MCP, may that betools, may that be skills, may that be all of these things? Yeah. I mean, so the longer contextwindows are good. You can talk to it for longer, like they're doing a good job, but at the end ofthe day, like, especially when you had like Opus, it was like Opus 4.5 and then Opus 4.5,4.5, 1 mil or 4.6 and 4.6, 1 mil. You're not actually getting a like smarter model. Like theintelligence of the model is, is what drives its ability to attend to all of the tokens in thecontext window to figure out on the next turn, which parts of this a hundred K or 200 K contextwindow are the most relevant to making the decision of like, what is the next tool we calland doing that over and over again in a loop. So I don't know. There was some study that came outin 2025, which found that, and again, these are old models. So like inflate your numbers,

我们现在确实有 100 万上下文窗口的模型了，也许以后还会有更长的。但当你开始往上下文里塞更多东西，它就开始变得没那么高效。从实用角度看，对一个正在往上下文窗口里加一堆东西的人——不管加的是 MCP、是工具、是技能还是这一切——我们目前知道些什么？

我是说，更长的上下文窗口是好事，你能跟它聊更久，它们干得不错。但说到底，尤其是当你看到 Opus 4.5，然后是 Opus 4.5 的 100 万版，或者 4.6 和 4.6 的 100 万版——你并没有拿到一个更聪明的模型。是模型的智能水平，决定了它有没有能力去关注上下文窗口里的所有 token，从而在下一轮判断出：这十万或二十万 token 的上下文里，哪一部分对「下一个工具调什么」这个决定最相关——然后在循环里一遍又一遍地这么做。所以，2025 年有个研究出来，发现——这些又都是老模型了，所以你得把数字往上调——

[28:48] Dex Horthy

but it was like frontier LLMs can follow about 150 to 250 instructions before it starts to drop off.Their ability to follow all the instructions just like drops off pretty quickly. And I thinkLori Voss at Arise, I haven't actually looked at the data, but they did a study with like the nextgeneration models a year later. And it looks like it's like much better the number of instructionsyou can get in. But in any case you have like, I split context engineering into like two categories.You have like the, the most people think about like the information budget of like, okay, I can dorag and I can pull out chunks of this document rather than putting the entire book into my contextwindow. I can just go grab the pages that matter, but it's also your instruction budget is like,if you give the model too many instructions and especially too many conflicting instructions,and that's in your initial prompt. And also like, if you have a conversation,you start going down a path and then you change your mind and you start going down a different,you're like, actually, I don't want to do any of that. I want to do this.It's like, it's a lot of computation. The model has to do to notice that it has to ignore that whole

但当时的结论是：前沿大模型大概能遵循 150 到 250 条指令，超过之后就开始掉，它遵循全部指令的能力会掉得挺快。我记得 Arize 的 Lori Voss——数据我其实没细看——他们用一年后的新一代模型又做了一次研究，看起来好多了，能塞进去的指令数上升了不少。

但不管怎样，我把上下文工程分成两类。大多数人想的是「信息预算」：好，我可以做 RAG，可以从这份文档里抽出相关的片段，而不是把整本书塞进上下文窗口，我只去取那些真正重要的页。但你还有一个「指令预算」：如果你给模型太多指令，尤其是太多互相冲突的指令——这既指你最初的提示词，也包括对话过程中：你开始往一个方向走，然后改主意，开始走另一个方向，你说「其实我不要那些了，我要这个」。模型要花很大的计算量，才能注意到它必须忽略掉之前那一整

[29:49] Gergely Orosz

thing. And when both of those things are kind of far back enough in the context window that they'reonly half getting attended to your likelihood that it's like actually going to like, rememberthe exact instructions you gave it a hundred thousand tokens ago is like, it goes down quitesignificantly. This is all very interesting because as engineers there, we are expected when,you know, when we're AI engineers, which now a lot of software engineers are, meaning you just likeuse LLMs to build software, like underneath there's an LLM layer somewhere you're an AI engineer.Congratulations. But it sounds like the expectation is to be, you know, to be a good,to be a good software engineer pre AI, you need to understand, you know, how to write good code.And it helps when you understand a little bit of the underlying, we didn't need to do that thatmuch over time, but it never hurts. But sounds like right now we're in this phasethat to be an engineer who can write an efficient AI system that use LLMs, you need to understandthe dynamics of the context. You need to understand why stuffing your context one way or the othercan be compute, can introduce latency. And all of these, it sounds like it's kind of more of an

块。而当这两样东西都往上下文窗口后面退得足够远、只被「半注意」到时，它真正记住你十万 token 之前给它的确切指令的概率，就下降得相当明显。

这一切都很有意思。因为作为工程师，我们现在被期待——你知道，当我们是 AI 工程师时，而现在很多软件工程师都是了，意思是你只要用大模型来造软件、底层某处有一层大模型，你就是 AI 工程师，恭喜——但听起来这个期待是：在前 AI 时代要当个好软件工程师，你需要懂怎么写好代码，懂一点底层原理会有帮助，虽然后来我们不太需要懂那么多了，但懂了从不吃亏。可听起来现在我们处在这样一个阶段：要成为一个能写出高效、用大模型的 AI 系统的工程师，你必须理解上下文的动力学。你得理解为什么用这种方式还是那种方式塞上下文，会带来计算量、会引入延迟。这一切听起来更像是一种

[30:55] Dex Horthy

intuition. And of course there's some understanding, but from talking to you, you're like, well, it doesthis computation. Like I know, you know, cause you've tried it out, right? Like I'm not, I'm not a PhD inmachine learning. Like I couldn't actually go like draw a mathematical proof of how this works,but we know attention is quadratic. And the more stuff you put in, the more it has to spread thisattention out over everything. This just feels like an absolute new area and like a little bit verydifferent to like what we're used to like software engineering, which is like pretty kind of likeblack and white, right? It compiles or doesn't compile. That's true. I mean, there's a different kind ofintuition. I was talking about this earlier as well. I was like, there's a different kind of intuitionthat you, that you develop over years as a software engineer. And, uh, there's many categories of it,but the one I'll, I'll call attention to that is like a thing that you cannot teach. You cannot do,you cannot learn in a textbook. The only way to learn it is like, I know bad patterns in softwarebecause I have debugged them at three in the morning. This is my buddy, Jake from Netflix said thisin his talk at AI engineer code. It's just like, there's no better way to learn what is good and what is bad

直觉。当然也有一些理解成分，但从跟你聊天来看，你会说「它就是要做这个计算」——你知道，因为你试过，对吧？

对，我不是机器学习的博士，我没法真的去画一个数学证明来说明这是怎么工作的。但我们知道注意力是二次方复杂度的，你塞进去的东西越多，它就得把这份注意力摊到越多东西上。

这感觉完全是一个新领域，而且跟我们习惯的软件工程很不一样——软件工程是相当非黑即白的，对吧？要么编译通过，要么不通过。

是这样。我是说，有另外一种直觉——我今天早些时候也在聊这个——是你作为软件工程师在多年里养成的那种直觉。它有很多类，但我想特别点出的那一类，是你教不会的：你没法从教科书上学。学它的唯一方式是——我知道软件里哪些是坏模式，因为我在凌晨三点调试过它们。这是我哥们儿、Netflix 的 Jake 在 AI Engineer Code 大会演讲里说的。就是说，要学会什么是好、什么是坏、

[31:54] Gergely Orosz

and what works and what doesn't than suffering through the thing that doesn't work.Well, speaking of suffering through the things that doesn't work, uh, a new paradigm, uh, that isspreading up is loops, loop engineering. The idea that instead of writing prompts, just write loops,set up your loops. And this all started with the Ralph Wiggum technique where it will, it will just,well, I guess that's an early version of loops that were just loops around. And now we're,we're, we're hearing what some of the big, biggest labs talking about that. They're actuallyjust doing loop venturing. What is your take on, have, have you done some loop venturing yourself?

什么管用、什么不管用，没有比「亲自熬过那个不管用的东西」更好的办法了。

说到熬过那些不管用的东西——有一个正在蔓延的新范式，叫循环（loops）、循环工程（loop engineering）。这个想法是：与其写提示词，不如写循环、把你的循环搭起来。这一切都始于 Ralph Wiggum 技法——我猜那算是循环的早期版本，就是一直绕圈跑。而现在我们听到一些最大的实验室也在谈这个，说他们其实就在做循环工程。你怎么看？你自己做过循环工程吗？

[32:28] Dex Horthy

Have you set up some loops and what do you think is good about it? And what do you think is bad aboutit? Yeah. So I think of loops as, I mean, this could, I could ramble on this for 10 minutes.This is an entire talk, but I'll, I'll try to, I'll try to lay out some high level stuff and then wecan dig in wherever you think is most interesting. We had Ralph Wiggum as I see a year and four days agowas the first time I saw the Ralph Wiggum demo and like Jeff Hundley was just like visiting SF and hejust like came through and like dropped everybody's jaws with his like, yeah, I just ran Sonnet aroundthe clock and spent six grand in six weeks. And like, I built an entire Gen Z programming languageand look at it compiles and it has a stage two compiler where the compiler for the language iswritten in the language itself and all that insane. And the core lesson from all of that,I think was the idea of back pressure, which is basically, and I think a lot of people were doingthis for a very, have been doing this for a long time, which is how do I let the model check its ownwork? How do I automate the process of getting feedback into the model? And there's lots andlots of different flavors of this. You can have deterministic linters, you can have unit tests,

你搭过一些循环吗？你觉得它好在哪儿、坏在哪儿？

对。我把循环理解成……这个我能唠十分钟，它本身就是一整场演讲了。不过我尽量先讲些高层的，然后我们再挑你觉得最有意思的往下挖。Ralph Wiggum 是——据我看是一年零四天前——我第一次看到 Ralph Wiggum 的演示。Jeff Huntley 当时正好来旧金山，就顺路过来，把所有人的下巴都惊掉了。他说：对，我就让 Sonnet 全天候不停地跑，六周烧了六千美元，然后我造出了一整门 Z 世代风格的编程语言，你看它能编译，还有一个「二阶编译器」——这门语言的编译器是用这门语言自己写的，全都疯了。

而这整件事的核心教训，我觉得是「反压」（back pressure）这个概念。基本上——而且我觉得很多人已经这么干很久了——就是：我怎么让模型检查它自己的工作？我怎么把「拿到反馈并喂回模型」这件事自动化？这有非常非常多的花样：你可以有确定性的 linter（代码检查器）、可以有单元测试、

[33:35] Dex Horthy

like part of what made the programming language easy to build with Ralph is a programming language can beinfinitely verified. You write, you write the code in the language, you compile it. If the compiler fails,you go fix the compiler. You run the program. If the program fails, you go fix the compiler. Likeit's like, it's very, very verifiable. And I think the lesson in loops engineering is like,if you can make a problem very verifiable, you can kind of like treat it like a black box.And then have it loop because it will keep improving itself because of the verificationloop is already there. Exactly. And so like, you can do this with CI CD is like, I do this everytime I do a release. I'm like, I'm tired. The CI CD is slow. Cool. Go research the code base,make a change, make a pull request, run the test, see if it's faster. Try again, run the pull, run,run the test, push, push to the branch, check again, see if it's faster. And so it's like,if it can verify its own work in a loop, instead of design, instead of saying, let's try this approach,or let's try that approach or success and being really back and forth, you just say like,my goal is to make the CI faster. And you tell the model, here's the steps. Here's the five,

……Ralph 之所以能轻松造出一门编程语言，部分原因就是编程语言可以被无限验证：你用这门语言写代码，你编译它；编译器挂了，你去修编译器；你运行程序，程序挂了，你去修编译器。它非常非常可验证。我觉得循环工程的教训是：如果你能把一个问题变得很可验证，你就基本上可以把它当黑箱来对待。

然后让它循环——因为验证回路本来就在那儿，它会不断自我改进。

没错。你也可以对 CI/CD 这么干——我每次发版都这么干。我说：我累了，CI/CD 太慢。行，去研究代码库，做个改动，开个 PR，跑测试，看看是不是变快了；再试一次，推到分支，再检查一遍，看看是不是更快了。所以就是：如果它能在循环里验证自己的工作，那就不用「我们试试这个方案、试试那个方案」地来回折腾，你只要说「我的目标是让 CI 更快」，然后你告诉模型：这是步骤，这五步——

[34:38] Dex Horthy

here's the five steps. You're going to write some code. You're going to commit it. You're going topush it. You're going to launch a sub agent to watch the job until it's finished. It's going to tell youwhat happened. Then you're going to decide what to do next. And so that's what, that's like the verysimplest example I have of like designing loops. And you just said the goal, which iscloud code. And I think codecs have both chip slash goal, which is you just set the goal and ititerates until it reaches it or as long as it makes progress towards it.Exactly. And so it's like, if it's verifiable, if you can measure, this is auto research too.Auto research is like, Hey, go make this model twice as fast. And like, it's just a prompt thattells the model, like go to it over and over again and try things until it actually has good results.So that's what I think of loops engineering. I don't know. We, we do a very interesting kindof loops engineering where like the, the challenge is like, I think it's very easy to get very excitedabout building the thing that builds the thing or building the thing that builds the thing thatbuilds the thing we talked about. Uh, and so people say, Oh, we need to like redo everything

这五步。你要写点代码，你要提交，你要推上去，你要起一个子 agent 去盯着这个任务直到它跑完，它会告诉你发生了什么，然后你再决定下一步做什么。这就是我能给的、关于「设计循环」最简单的例子。

你刚才说了「目标」（goal）——Claude Code 和我想 Codex 都有 /goal 这样的东西：你只要设定目标，它就一直迭代，直到达成目标，或者只要它还在朝目标推进就一直做下去。

没错。所以就是：如果它可验证、如果你能度量。自动化研究（auto research）也是这样：嘿，去把这个模型跑快一倍。那就是一段提示词，告诉模型一遍遍地去试，直到真的拿到好结果。这就是我理解的循环工程。

我们还做一种很有意思的循环工程。挑战在于——我觉得人很容易对「造那个造东西的东西」，或者「造那个造出『造东西的东西』的东西」过度兴奋，我们前面聊过。于是有人说：哦，我们得把一切都重做成

[35:34] Dex Horthy

as this big, like agentic first factory, maybe even a dark factory. And they're like redesigningtheir entire thing to be their infrastructure for the next five years. And I'm sure one thing we knowof, uh, in engineering, uh, and especially, uh, pragmatic engineering is, uh, how can you make thismore incremental? How can you make it more continuous? Uh, and a lot of people don't have the optionto just, Hey, I ran a Ralph loop for three days and it fixed every linter error at our code base.Here's a 60,000 line PR who wants to review it and who wants to sign off on merging and deploying it.And, uh, that there's not going to be any bugs, nobody. So I think that the thing I'm most excitedis actually like what we call like iterated loops or like slow loops, where we basically have a cron job.We have the loop. The, the, the structure of the loop is really easy. It's like run this linter,fix one thing, commit and push. And then we run that every night in our GitHub actions.And we wake up every morning to one PR that makes the code base a little bit better.I like the stall loops. Yeah. And it has two dimensions. So you can add,now we have a blueprint for it. And actually Kyle just shipped a skill so that you can build

一个宏大的、agent 优先的工厂，甚至是黑灯工厂。他们在把自己未来五年的整套基础设施重新设计一遍。而我们在工程里、尤其是在「务实工程」里知道的一件事是：你怎么能把这件事做得更增量？怎么能做得更连续？而且很多人根本没有这个选项——「嘿，我跑了一个 Ralph 循环三天，它把我们代码库里所有的 linter 报错都修了，这儿有个 6 万行的 PR，谁来评审？谁来签字同意合并和部署？而且保证不会有 bug」——没人会接。

所以我最兴奋的，其实是我们叫作「迭代循环」或者「慢循环」（slow loops）的东西。我们基本上就是搞一个定时任务（cron job），循环的结构非常简单：跑这个 linter，修一个问题，提交并推送。然后我们每天晚上在 GitHub Actions 里跑一遍，每天早上醒来就有一个 PR，让代码库好那么一点点。

我喜欢这个慢循环。

对。而且它有两个可以加码的维度。现在我们有一份蓝图了，Kyle 刚做了一个技能（skill），你可以自己造

[36:35] Dex Horthy

these yourself. You can add more like feedback mechanisms. So we have react doctor for the frontend. We have another anti pattern that has no deterministic tooling, but Kyle's just like,here's what good looks like. Here's what bad looks like. Go fix one thing and bring it up. It waslike prop narrowing. Basically we have a bunch of optional props and most of them don't need to beoptional. It's like, here's how to make the prop not optional so that you know that the code justis like cleaner and easier to reason about. And so you can add more conditions, more things of like,fix one thing. I want to wake up to a PR. So now we wake up to like four PRs because there's fourseparate things. And then the other dimension you can do here is as you gain confidence,you can increase the scope instead of fixing one thing, fix four things. And so these are likeother ways to think about loops where it's like something that's not a human triggers it to start,whether it's, you know, an alert from Sentry, whether it's a user feedback, like support ticket,whether it's PM writes a ticket, whether it's a test is failing, any of, or it's a cron,it runs on a schedule, but it's like the trigger should be something that you don't have to like

这些东西。第一个维度是加更多反馈机制：前端我们有 react doctor；还有另一种反模式，它没有确定性的工具能查，但 Kyle 就直接写清楚「好长什么样、坏长什么样，去修一个，然后提上来」。那个反模式叫「prop 收窄」（prop narrowing）：我们有一堆可选的 props（属性），其实大多数不需要是可选的。那就写清楚「怎么把这个 prop 变成非可选的」，这样代码就更干净、更好推理。所以你可以加更多条件、更多「修一个」的项目。我原本想每天早上醒来收到一个 PR，现在我们醒来会收到四个 PR，因为有四件独立的事。

第二个维度是：随着你信心增加，你可以扩大范围——从「修一个」变成「修四个」。这些都是理解循环的另一些方式：触发它的不是人，可能是 Sentry 的一条告警，可能是用户反馈、一张支持工单，可能是产品经理写了一张工单，可能是某个测试挂了——或者就是一个定时任务，按计划跑。总之触发器应该是那种你不需要去

[37:33] Gergely Orosz

press a button on. And there's a defined workflow and it makes everything a little bit better.That's just described letting agents fix things without a human pressing a button.But what if a bug is too difficult, not just for an agent, but also for human to reproduce,let alone fix. This is where presenting sponsor Antithesis comes in. I was recently pairing withthe Antithesis team where we did a walkthrough of how they helped fix a nasty bug in etcd,the open source key value store used by Kubernetes. This is a bug that actually happened in etcd.The team noticed that the linearization validation assertion failed during the regular Antithesis runs.This is not good because the linearization guarantees strong consistency. So this needs to be fixed.So what the etcd team did was run a casualty analysis inside Antithesis.This generates this graph, which is a bug probability graph. Here, the x-axis is virtual time,and the y-axis is probability. Now we see that something happened just before virtual time 24that caused a huge jump in the probability that the bug would occur. Going deeper,we can look at the entire set of timelines. Vertical lines going down represent events branching off fromthe same state. And the purple dots are where the bug happens. If we look closely enough,

按按钮的东西。然后有一条定义好的工作流，它让一切都变好一点点。

这说的就是：让 agent 去修东西，不需要人按按钮。

但如果一个 bug 难到不只是 agent、连人都很难复现，更别说修呢？这就轮到本期呈现赞助商 Antithesis 出场了。我最近跟 Antithesis 团队结对，他们带我走了一遍：他们是怎么帮忙修掉 etcd 里一个很讨厌的 bug 的——etcd 是 Kubernetes 用的那个开源键值存储。这是一个在 etcd 里真实发生过的 bug。团队注意到，在常规的 Antithesis 运行中，「线性化校验」断言失败了。这很糟糕，因为线性化保证的是强一致性，所以必须修。于是 etcd 团队在 Antithesis 里跑了一次因果分析。它会生成这样一张图，叫 bug 概率图：横轴是虚拟时间，纵轴是概率。现在我们看到，就在虚拟时间 24 之前发生了某件事，导致这个 bug 出现的概率猛涨。再往深看，我们可以看到全部的时间线：向下延伸的竖线代表从同一状态分叉出去的事件，紫色的点就是 bug 发生的地方。如果你看得够仔细，

[38:43] Gergely Orosz

we see that all of the failures come from one parent branch. Gotcha. This is such a useful debugging tool.In the end, the team was able to figure out that process pauses were causing the bug using all theseAntithesis debugging tools. This non-deterministic bug was diagnosed in a deterministic way. How cool is that?

会发现所有的失败都来自同一个父分支。明白了。这真是个特别有用的调试工具。最后，团队用这一整套 Antithesis 调试工具，搞清楚了是「进程暂停」导致了这个 bug。一个非确定性的 bug，被用确定性的方式诊断了出来。这多酷啊？

[39:00] Gergely Orosz

Oh, and this is an actual bug that then got fixed in etcd. You can see the bug and the fix in etcd'sGitHub repo. Honestly, the tools that Antithesis built for debugging feel pretty darn futuristic,but they are also really powerful. Head over to antithesis.com slash pragmatic to learn more.I'd also like to talk about our season sponsor, Sentry. Sentry is a tool I use for applicationmonitoring on all of my projects, including their pragmatic engineering backend. I've used it for 10years now, starting with when I worked at Uber. An 8th Sentry feature I'm liking is their Sear AI agent,which helps investigate production errors. For example, here's an actual error I had in myapplication. I can just ask Sear what might be the root cause and it brings context and it can alsomake a plan to fix it right from the web interface. And the nice thing is how Sear also works great fromSlack as well, not just from the web. One place I find even more handy to use Sentry is from codex orclock code using Sentry MCP. Also, you can set up neat automations like when I resolve Sentry issueresurfaces, you can kick off a cursor agent or GitHub copilot agent to investigate the regression,read the relevant code and open a PR with a suggested fix. I'm not a fan for using AI tools just for the

而且这是一个真实的 bug，后来在 etcd 里被修掉了，你可以在 etcd 的 GitHub 仓库里看到这个 bug 和它的修复。老实说，Antithesis 为调试造的这些工具，感觉相当有未来感，同时也真的很强。去 antithesis.com/pragmatic 了解更多。

我也想聊聊我们的赛季赞助商 Sentry。Sentry 是我在所有项目上做应用监控都在用的工具，包括 The Pragmatic Engineer 的后端。我用它已经十年了，最早是在 Uber 工作时开始的。我最近喜欢的一个 Sentry 功能是他们的 Seer AI agent，它能帮你排查生产环境的错误。比如，这是我应用里真实出过的一个错误，我可以直接问 Seer 根因可能是什么，它会把上下文带出来，还能直接在网页界面里给出修复计划。好的地方是 Seer 在 Slack 里也很好用，不只是网页端。我觉得更顺手的一个用法，是通过 Sentry MCP 从 Codex 或 Claude Code 里用 Sentry。你还可以设置一些漂亮的自动化，比如当我已经解决的 Sentry 问题再次出现时，就触发一个 Cursor agent 或 GitHub Copilot agent 去排查这次回归、读相关代码、并开一个带修复建议的 PR。我不喜欢为了用 AI 而用 AI，

[40:02] Dex Horthy

sake of it, but I really like the practical integrations where I can fix errors faster and with morecontext. Check out Sentry at sentry.io slash pragmatic and start monitoring and fixing regressions today.And with this, let's get back to Dex and to agentic loops that trigger themselves.Now, you said we can get more ambitious and we can add more things to it, but I'm going to quote youwith one of your tweets, which says, this may surprise you that this is coming from me,but I think we're in for a one to three year period where stuff might break at 3am and you're relyingon loops to fix it and nobody understands what's under the hood and you're looking at it as anexistential threat to your company.Yes. Yeah, that one was great. That one did a lot of numbers.It resonated.Here's the other side of it is like, I think that the today with today's models, today's programminglanguages, today's infrastructure, you might get away with not reading the code.Problem with loops is like at a certain point, you're going to generate so much code that youcan't read it anymore. This is the strong DM dark factory. This is like Ryan La Popolo'slike harness engineering, just spend as many tokens as possible. We tried this. We built a

但我真的很喜欢这种实用的集成——让我能更快、带着更多上下文去修错误。去 sentry.io/pragmatic 看看，今天就开始监控并修复回归。

好，我们回到 Dex，回到那些「自己触发自己」的 agent 循环。

你刚说我们可以更有野心、往里面加更多东西。但我要引用你的一条推文：这话从我嘴里说出来可能会让你意外，但我认为我们将进入一个一到三年的时期——东西会在凌晨三点坏掉，你却只能靠循环去修，没人搞得懂底下是怎么回事，而你会把这看成公司的生存威胁。

对，那条挺好的，数据跑得很不错。

很有共鸣。

另一面是这样：我觉得以今天的模型、今天的编程语言、今天的基础设施，你也许还能侥幸不读代码。循环的问题在于，到某个点，你会生成多到你再也读不完的代码。这就是 StrongDM 的黑灯工厂，这就是 Ryan La Popolo 那种 harness 工程——尽可能多地烧 token。我们试过。我们在

[41:09] Dex Horthy

Lightsoft software factory in July of 2025. And by November we had shut it down. I think it takesabout three to six months of you shipping all the time with nobody reading the code before yourealize like, wow, this is getting way worse and it's easier to start over than it is to fix it.Like the models have made the code base so bad that it is actually going to be easier to justlike rethink this from scratch. And maybe that's okay because we have AI and it's easier to rebuildthings from nothing. And like usually when engineers say like, oh, we can't fix this. We have to rebuildit. The feedback is like, no, just refactor in place. Just constantly keep the code base gettingbetter. You mentioned what I said. You'll notice what I said was not use loops to ship the features thatusers want. We use loops to actually improve the code base quality. And we read all the code because we careabout how it's architected and we care not just about the system architecture, but what I wouldcall the program design, which I think is something that people are going to, where are the interfaces?

2025 年 7 月造了一座关灯软件工厂，到 11 月我们就把它关了。我觉得大概需要三到六个月：你一直在发布、没人读代码，然后你就会意识到——哇，这东西越来越糟，重来一遍比修它还容易。模型已经把代码库搞得那么差，以至于从头重想一遍反而更省事。也许这也没关系，因为我们有 AI，从零重建东西更容易了。而且通常当工程师说「哦，这个修不了了，我们得重写」时，得到的反馈都是「不，就地重构，让代码库持续变好」。

你注意到我刚才说的不是「用循环去发用户想要的功能」。我们用循环来提升代码库的质量。而且我们读所有的代码，因为我们在意它是怎么架构的——不只是系统架构，还有我称之为「程序设计」（program design）的东西，我觉得这是大家接下来要开始重视的：接口在哪儿？

[41:57] Dex Horthy

Where are the seams? How are we doing dependency injection? All of these things that like make yourcode base more maintainable over time and keep you from falling into this trap of like, okay, well nowif I change something over here, it broke something over here. This is the classic problem software engineeringthat like software engineering was invented in the 1970s because we realized we needed techniquesfor avoiding that problem of like this giant ball of spaghetti. And I don't think the models aresmart enough. And I don't think we actually have the training and the benchmarking and the evaltechniques to get models to write code that is more maintainable over time versus they're all trainedon Sweebench and Sweebench looking things, right? All of the benchmarks are basically like, here's acommit in Django. Here's an issue that was filed around that time. See if you can create the fix thatthe human created and it's Django and it's Apache and it's, there's a hundred repos in Go and C++ andTypeScript and Java and all these different languages, but they're all, it's like the problemwith training models on maintainability is like the cost function of bad architecture and bad programdesign can't be evaluated by running the unit test because it hits you three to six months later

缝在哪儿？我们怎么做依赖注入？所有这些让你的代码库随时间更可维护、并且不掉进「我在这边改一下、那边就坏了」这个陷阱的东西。这是软件工程的经典问题——软件工程之所以在 1970 年代被发明出来，就是因为我们意识到需要一些技术，来避免「一大坨意大利面」这个问题。

而我不认为模型现在够聪明。我也不认为我们真的有那种训练、基准和评测技术，能让模型写出「随时间更可维护」的代码。它们全都是在 SWE-bench 以及类似 SWE-bench 的东西上训练的，对吧？所有基准基本上都是：这里有一个 Django 的提交，这里有一个大约同期提的 issue，看看你能不能造出人类当时做的那个修复。而且是 Django、是 Apache，有一百个仓库，涵盖 Go、C++、TypeScript、Java 各种语言。但问题在于：用「可维护性」来训练模型的难点是，糟糕架构和糟糕程序设计的代价函数，没法靠跑单元测试来评估——因为它是在三到六个月之后才砸到你头上，

[43:05] Gergely Orosz

when you're like, holy crap, like no one can make it's the software has become so hard to change.Is this not similar to how senior software engineers, why it took years for someone to become a senior?

那时候你才会说「我的天，这软件已经难改到没人动得了了」。

这跟资深软件工程师的养成不是很像吗？为什么一个人要好几年才能成为资深？

[43:17] Gergely Orosz

Because typically, and in some environments you became a, you can become a senior faster,typically fast moving where there's a bunch of issues and you have to keep fixing it.Sometimes, you know, some people are working in the same place for 10 years and they're still notthat level. The point was it just takes time for you to understand the small mistake that you makeright now that snowballs into like something disastrous later and you get hit by it and yourealize like, okay, things like, you know, like testing matters, architecture matters,tech depth can actually be a killer. You know, we don't talk about it anymore, but we used to talkabout how tech depth kills or slows down companies so badly pre AI that their competitors can overtakethem or they're just like stuck with a two-year refactor and not shipping any new features and thecompetition, you know, ships a bunch, a bunch of other stuff and now they're ahead.And I will say like, it is possible that GPT-7 will fix this, but if you are turning the lightsoff in your software factory and you're saying like, hey, you know what? Like we're not goingto read the code. It's fine. The models are smart enough. If we give it the right feedback and just

因为通常——在某些环境里你能更快变成资深，通常是那种节奏快、问题一堆、你必须不停去修的环境；而有些人在同一个地方待了十年，还是达不到那个水平。重点是：你需要时间，才能理解「你现在犯的这个小错误，怎么滚雪球滚成后面某个灾难」，然后你被它砸中，你才意识到：好吧，测试是重要的、架构是重要的、技术债真的会要命。我们现在不太谈这个了，但在前 AI 时代我们常说，技术债会把公司拖慢到竞争对手可以超车的地步，或者他们被困在一场两年的重构里、发不出任何新功能，而竞争对手一直在发一堆东西，现在人家领先了。

我得说，有可能 GPT-7 会解决这个问题。但如果你正在把你的软件工厂关灯，说「嘿，你知道吗，我们不读代码了，没关系，模型已经够聪明了。只要我们给它对的反馈、

[44:15] Dex Horthy

throw enough tokens at the problem, it will keep getting better. This is what led to this tweet.Like that might work, but if nobody read the code in three months and you replace all of your,all of your like code review with loops of like, hey, if a user complains, we give it to an agent.If something crashes, we give it to an agent. If a, if a PM writes a ticket, we give it to an agent.If a CEO writes an obnoxious essay about what we should be building in Slack, we give it to an agent.And then you stop reading the code because that's going to produce way too much. Like no one can readit. And like the, the, the PR reviews become the bottleneck. So you replace that with agentictesting and agentic, uh, agentic code review. Uh, but none of these things have intuition forsoftware architecture because we haven't trained it in yet. And so you're going to wake up one day andyou're going to have an issue with this happened to us. And like, we got through it. And at the timeof like, it was still worth it. It was like spent three weeks onboarding back into the code base thatwe had stopped reading three months ago, because no matter how much sophisticated expert prompting,we could not get Opus. I think it was Opus 4.1 at the time. We could not get Opus 4.1 to actually

往问题上砸足够多的 token，它就会一直变好」——这就是我发那条推的由来。那也许行得通。但如果三个月没人读过代码，而且你把所有的代码评审都换成了循环：嘿，用户投诉了，交给 agent；有东西崩了，交给 agent；产品经理写了张工单，交给 agent；CEO 在 Slack 里写了篇讨人厌的长文说我们该造什么，交给 agent。然后你不再读代码了，因为产出量太大了，没人读得完。于是 PR 评审成了瓶颈，你就把它换成 agent 化测试和 agent 化代码评审。但这些东西没有一个对软件架构有直觉，因为我们还没把这件事训练进去。

所以某天你会醒来遇到一个问题——这事真发生在我们身上，我们也挺过来了，而且当时来看还是值的。当时是花了三周重新「入职」回一个我们三个月前就不再读的代码库——因为无论用多么复杂、多么专家级的提示词，我们都没法让 Opus（我记得当时是 Opus 4.1）

[45:13] Dex Horthy

find the root cause. We had to go spend several days digging through the code and figuring out like,oh, there's just actually a primary key that's being routed through this whole thing that needs to bechanged to a different type of object. And it needs its own table.This actually happened to you.This happened to us. Yeah. And when it happened, I was like, you know what? That sucked. That wasterrible, but we did it. We solved it. And, uh, it's still worth, it's still worth not reading thecode for most of the time at the cost of every once in a while, I'm going to have to spend twoweeks fixing an issue by hand. And I don't believe that anymore because I think the amount of code we'reable to write now is actually like 10 X or a hundred X. And I think the problem's just getting worse.So let's talk about software factories. Yeah. In your mind, cause I feel it's an overloaded word,but what do you think of a software factory before AI and now post AI?

找到根因。我们只能自己花好几天翻代码，最后发现：哦，其实是有一个主键被一路传穿了整条链路，它需要被换成另一种类型的对象，而且它需要有自己的表。

这真的发生在你们身上。

真发生了，对。当时它发生的时候我想：你知道吗，那确实很糟、很痛苦，但我们做到了、我们解决了。而且这仍然是值的——大部分时间不读代码，代价是每隔一阵子我得手工花两周去修一个问题。但我现在不再相信这个结论了。因为我觉得我们现在能写出的代码量其实是 10 倍甚至 100 倍，我觉得这个问题只会更糟。

那我们来聊软件工厂。

好。

在你心里——因为我觉得这是个被用滥的词——AI 之前的软件工厂是什么？AI 之后又是什么？

[45:57] Dex Horthy

Do you know what the first definition of software factory, the first time it was used?No.It was a NATO conference in 1968.Oh, Grady, which would know about this.Yeah, exactly. Yeah. Great. You should ask Grady about it. They talked about the idea of like,okay, you actually need to build a system of steps and like, just like a factory floor. You have likethe coding part and the testing part and the validation part and the integration part. We hadno CICD. We barely had version control, like, but you needed a factory. And then it was adopted bylike, um, Toshiba and a bunch of companies. And then the, the next moment was like DevOps. And youhave like this idea of like, okay, we're going to do CICD. We're going to automate. We're going toChef and Ansible, Puppet, whatever. All these technologies is like, instead of having dudesrunning around data centers, like resizing disks and stuff or clicking around the AWS console.Yeah, exactly. It was like, cool. We build loops. The server hits 90% disk space that sends analert to Nagios. Nagios triggers a Chef run. Chef makes the disk, the disk bigger. Feedback loops,right? This has been around for a while. And in 2018, I want to say this guy, Nick Chalain,

你知道「软件工厂」这个词第一次被定义、第一次被使用是什么时候吗？

不知道。

1968 年的一场北约（NATO）会议。

哦，Grady（Booch）应该知道这个。

对，正是。你该去问问 Grady。他们当时谈的想法是：好，你其实需要造一套步骤系统，就像工厂车间一样——你有编码环节、测试环节、验证环节、集成环节。当时我们没有 CI/CD，连版本控制都勉强，但你需要一座工厂。后来这套被东芝之类的一堆公司采用了。

再下一个时刻就是 DevOps：好，我们要做 CI/CD，我们要自动化，我们上 Chef、Ansible、Puppet 这些技术——而不是让一群人在数据中心里跑来跑去扩磁盘什么的。

或者在 AWS 控制台里点来点去。

对，正是。那时候就是：行，我们造循环。服务器磁盘占用到 90%，它给 Nagios 发一条告警，Nagios 触发一次 Chef 运行，Chef 把磁盘扩大。反馈循环，对吧？这东西已经存在很久了。然后在 2018 年，我想是一个叫 Nick Chaillan 的人——

[46:57] Dex Horthy

who was, uh, he was like the CTO or chief software officer of the air force. He wrote this a hundredpage essay of, Hey, the DOD needs a software factory. The department of defense. Yeah. The departmentdefense and the air force. And he called it DevSecOps factory. And he said, we need all the thingsthat all of the good enterprises are using. We need Jenkins. We need like code quality scanning.We need security scanning. We need CICD. We need to be able to ship. We're shipping once every threemonths or once a year. We need to be able to ship every day, like all these other companies.And the way we do that is we actually embrace all these automations and technologies so that engineersare, are the 90% of the issues are caught by automations instead of people actually likemanually checking it or manually reading the code or manually integrating modules together.Wow. Talk about forward thinking in the government.I know. No, as I was surprised, like, Oh, nice. Like this is, I mean, and that was a part of it islike, Hey, look, we're falling behind in like, you know, I don't know exactly all the reason,but I imagine also about like attracting really good talent is like, Hey, look, if we have like

他当时是美国空军的首席软件官（Chief Software Officer）。他写了一篇一百页的文章说：嘿，国防部需要一座软件工厂。

国防部。

对，国防部和空军。他管它叫 DevSecOps 工厂。他说，我们需要所有优秀企业都在用的东西：我们需要 Jenkins，需要代码质量扫描，需要安全扫描，需要 CI/CD，需要能持续交付。我们现在是三个月发一次或一年发一次，我们得像其他公司那样每天都能发布。而做到这一点的方式，就是真正拥抱这些自动化和技术，让 90% 的问题被自动化抓住，而不是靠人去手工检查、手工读代码、手工把模块拼在一起。

哇，政府里居然有这么前瞻的。

我知道，我当时也挺意外的：哦，不错。我是说，这也是其中一部分原因——嘿，我们在某些方面落后了。我不知道全部原因，但我猜也跟吸引优秀人才有关：你看，如果我们有

[47:59] Dex Horthy

the modern software stack and we're building things fast and we care about efficiency and we careabout people's using people's time. Well, we care about them spending time on the hard parts ofthe job, not manually looking for SQL injections. Like you could automate that.So this was software factories, pre AI, pre AI. Now I've heard the term a lot more because of AI.Yeah. Is it the same? Is it different?

现代的软件栈、我们造东西很快、我们在意效率、我们在意怎么用好人的时间——我们希望他们把时间花在这份工作最难的部分上，而不是手工去找 SQL 注入。那种事你可以自动化掉。

所以这是 AI 之前的软件工厂。

AI 之前的。

现在因为 AI，我听到这个词的频率高多了。

是。

它是一样的东西吗？还是不一样？

[48:19] Dex Horthy

So this is really hard to say without a drawing, but I'll try to draw it out. At the core of asoftware factory, you have like a source of work. Most you can imagine a linear, a JIRA, a source oftruth, your object, whether it's a spreadsheet or whatever it is, you have like, what stages is thework in? Yep. And pre AI, you would take, you know, you would maybe do some architecture reviewplanning. You would maybe do some sprint planning and then people would take tickets off the queueand they would go build them. And then you would make a pull request and people would review itand you would run CI checks and then you would send it to prod and then it would make contact withyour users and your users would complain about stuff. And that would go to your support team and backinto your work tracker and it would crash and you would have issues and that would go into yourmonitoring stack and that would go into your tracker. And that was your loop. And then peoplewould take stuff off the tracker based on priorities, product managers, engineering managers, engineers,prioritizing work, and they would go and do that. And the first change is like this long wind,lots of phases. And this is also why when like a developer shifts a bug, but by the time it comes

这个不画图真的很难讲，但我试着画一下。软件工厂的核心，你有一个「工作的来源」。你可以想象成 Linear、Jira，一个事实源，你的对象——不管是电子表格还是别的什么——你有「这项工作处在什么阶段」。

嗯。

AI 之前，你会先做一些架构评审、规划，也许做一些迭代计划（sprint planning），然后人们从队列里领走工单去做。做完你开一个 PR，别人评审，你跑 CI 检查，然后送到生产环境，然后它接触到用户，用户开始抱怨，抱怨进到你的支持团队、再回到你的工作跟踪系统；它会崩溃、会出问题，那些进到你的监控栈、再进到跟踪系统。这就是你的循环。然后人们根据优先级从跟踪系统里领活儿——产品经理、工程经理、工程师排优先级，然后去做。

第一个变化是：这条链路特别长、阶段特别多。这也是为什么当一个开发者交付了一个 bug，等它转回到你手上时

[49:16] Dex Horthy

back to you, it might be two or three months or even longer. And by the time it gets fixed, it mightbe a year or two. And you know, this is why when you're using a piece of software, it's like thatannoying bug and you talk with customer support, but it's just a very like long latencies at eachpart of the factory, if you will. Yeah. And the step where someone pulls a work item off the queueand starts working on it is, you know, a couple hours to a couple days before it actually getsintegrated into everything else and touches user. And that's in a great world, right? Sometimes you gobuild it and then you merge it and then it actually gets released three months later. But we're going toassume we're in a fairly modern, like we're somewhere like the Netflix or a meta where engineers arecapable of shipping a hundred times a day or a thousand times a day, but it still takes two, threehours to do the work. And now with an agentic factory, what you do is you take out that personbuilding the thing and you replace it with an agent building the thing. And so you have orchestrationto trigger things. You have a sandbox, you have an LLM, you have an inner harness, you have an outerharness, which is like the dev environment you build for the agent. And maybe you give it a browser,

可能已经过了两三个月甚至更久；等它被修好，可能已经一两年了。你知道，这就是为什么你在用一个软件时会碰到那个烦人的 bug，你去跟客服聊——本质上就是工厂每一个环节的延迟都特别长。

对。而「有人从队列里领走一个工作项、开始做」这一步，到它真正被集成进其他一切、接触到用户，中间是几个小时到几天。而且这还是在理想世界里，对吧？有时候你造完、合并了，三个月后才真正发布。不过我们就假设我们处在一个相当现代的环境，比如 Netflix 或者 Meta 这种，工程师有能力一天发一百次甚至一千次，但做这份活儿本身仍然要两三个小时。

而现在，在 agent 化工厂里，你做的事是把「那个造东西的人」拿掉，换成「一个造东西的 agent」。于是你有编排（orchestration）去触发任务，你有沙盒，你有大模型，你有内层 harness，你有外层 harness——也就是你为 agent 搭的那套开发环境。也许你给它一个浏览器、

[50:15] Dex Horthy

you give it a video recorder. If you use like things like cursor background agents, they've kind ofbuilt this outer harness around the inner harness that is the coding agent. And then you make PRs with that.Problem there is that like, okay, now, now it takes 10 minutes to do a build instead of two hours ortwo days. And so now the bottleneck is code review. So, okay, let's throw a bunch of AI agents at codereview and let's do agentic testing so that like we can basically catch a lot of the easy stuff. Andhumans are only focused on the most like important critical core parts of the code base. And then thenext level up of your agentic factory is you do the top is like, okay, and then it gets deployed.It goes to prod and a user complains. You just hook your support queue right up to the agent.Someone complains about something, agent tries to fix it. And instead of looking at a ticket andthen saying, okay, go send, you just close that loop. And instead, every time something goes wrong,you just get a PR. And then every time something crashes in century or data dog or whatever,it goes into the tracker. It gets picked up by an agent and you get a PR. This is the ramp inspectthing. This is the, the only difference is like, then you have so much code to review and people say,

给它一个录屏工具。如果你用 Cursor 的后台 agent 之类的东西，他们其实已经在「作为编码 agent 的内层 harness」外面搭了这么一层外层 harness。然后你用它开 PR。

问题来了：现在做一次构建从两小时或两天变成了十分钟，所以瓶颈变成了代码评审。行，那我们往代码评审上砸一堆 AI agent，再做 agent 化测试，这样简单的问题基本能被抓住，人只聚焦在代码库里最重要、最关键的核心部分。

再往上一层的 agent 化工厂是：好，然后它被部署了，上了生产，用户投诉了。你直接把支持队列接到 agent 上。有人抱怨了，agent 就去试着修。你不再是「看一张工单，然后说好去发一个」，你把那个循环闭上了。于是每次出问题，你就直接收到一个 PR。每次有东西在 Sentry 或 Datadog 之类的地方崩了，它进到跟踪系统、被 agent 领走，你就收到一个 PR。这就是 Ramp 的 inspect 那套东西。唯一的区别是：接下来你要评审的代码多到有人会说

[51:14] Gergely Orosz

well, let's try turning the lights on. Let's just take all the human testing and review steps out.And we'll say, okay, cool. If users complain that it's broken. And if users don't complain,then it's working. And we're not going to read the code. We're going to use, we're going to treatthe whole system as a black box. So you said you tried this out when it was like Opus 4.1,you built the software factory. It was running beautifully until it just blew up on your faces.How do you think of this model? Cause I can see an ideal world where it works, but clearlywe're not an ideal work. Like where do you think we are? Like, and could some of this actuallywork at some point or, you know, like, like what, what progress are you seeing right now?

「那我们试试关灯吧」。我们干脆把所有人工测试和评审步骤拿掉，然后说：好，如果用户抱怨它坏了，那就是坏了；如果用户不抱怨，那就是好的。我们不读代码，我们把整个系统当黑箱。

所以你说你在 Opus 4.1 那会儿试过这个，你造了那座软件工厂，它跑得很漂亮，直到某天在你们脸上炸了。

你怎么看这个模式？因为我能想象一个理想世界里它是行得通的，但显然我们不在理想世界。你觉得我们现在在哪儿？这里面有多少东西在某个时点是真的能跑通的？你现在看到了哪些进展？

[51:50] Dex Horthy

And, and what is the, the, today, the situation, like how much of this do you believe we can automateor should we automate? Yep. So if you know me and you follow my stuff, you know, I stand for threethings. Number one is like cutting through the hype and the jargon and going, trying things and talkingto people who are using things and figuring out which parts of this actually work and are valuable.Number two, we talked about words. I try to find and protect useful bits of languagebecause I think it helps us all move forward. And when you take a useful word like agents,or you take a useful word like software factory, and then you semantically diffuse it, this isanother Martin Fowler word. You make it mean everybody likes the word and it's all becomeshype and everyone starts agents means nothing anymore. Agents could be a chat bot. It couldbe a Slack bot. It could be a coding agent. It could be tools in a loop, whatever it is.So I like to protect important, useful words and like help, help us all like elevate theconversation out of that hype and jargon. And then I care a lot about going one level down beneathwhere I'm generally working. I think there's always, this is the same thing with context

以及今天的状况是什么？我们能自动化多少、又应该自动化多少？

好。如果你了解我、关注我的东西，你会知道我坚持三件事。第一，戳破炒作和黑话，去试、去跟真正在用这些东西的人聊，搞清楚这里面哪些部分是真的管用、真的有价值。第二，我们前面聊过词：我努力去发现并保护有用的语言片段，因为我觉得这能帮我们所有人往前走。当你拿一个有用的词，比如 agent，或者比如「软件工厂」，然后把它「语义扩散」掉——这是 Martin Fowler 的另一个说法——你让它变成人人都喜欢的词、变成纯炒作，于是 agent 什么都不是了：agent 可以是聊天机器人，可以是 Slack 机器人，可以是编码 agent，可以是「工具在循环里」，什么都行。所以我喜欢保护重要、有用的词，帮我们所有人把讨论从炒作和黑话里拔出来。第三，我很在意「往我通常工作的那一层再下探一层」。我觉得总是这样，跟上下文

[52:50] Dex Horthy

engineering. It's like I was rarely actually going and like building LLMs or understandingor training LLMs, but knowing how they're trained and how Transformers works informs howyou build at one layer up. And for the software factory, my version of that is I spent the lastcouple of weeks going really deep on reinforcement learning with verifiable rewards. RLVR, which islike this very productionized, like it's not like RLH, RLHF is still like fairly academic and pure.RLVR is this like, it's a machine in these labs of how we train these models. And I'm studying likethe benchmarks for coding agents and the techniques for training them and how we like give it a smallproblem, have it solve it, delete the test changes it made, revert them, apply a test patch,see if it passed. And then even the frontier this year, we have like, we can get into this later,but like frontier code and SWE marathon, these new benchmarks that are supposed to be likebetter at evaluating models ability to maintain a code base over time and write maintainable code.And they are better, but I don't think they're sufficient. But it's basically this idea thatlike the only thing that made cloud code good was reinforcement learning. And the dimension along

工程是一回事：我很少真的去造大模型、去理解或训练大模型，但知道它们是怎么训练的、Transformer 怎么工作，会指导你在上一层怎么造东西。对于软件工厂，我的对应做法是：过去几周我在深挖「带可验证奖励的强化学习」（RLVR，reinforcement learning with verifiable rewards）。这是非常产品化的东西——不像 RLHF 还相当学术和纯粹，RLVR 是这些实验室里训练这些模型的那台机器。我在研究编码 agent 的基准、训练它们的技术：比如我们怎么给它一个小问题、让它去解、删掉它做的测试改动、回滚、再打上一份测试补丁、看看是不是通过了。甚至今年的前沿——我们待会儿可以细聊——比如 frontier code 和 SWE marathon 这些新基准，它们本应更擅长评估「模型随时间维护一个代码库、写出可维护代码」的能力。它们确实更好了，但我觉得还不够。不过核心想法是：让 Claude Code 变好的唯一原因就是强化学习。而它变好的那个维度

[53:56] Dex Horthy

which it got good was like, we made a model, we trained the model and the harness together.And so the model got really good at calling the specific tools in that harness, really good atreading files, writing files, searching for files, all this stuff through doing these problems.And that was what made it feel so much better than all the other CLI coding agents that came before it.And so people are like, okay, that was so much better. And they're just going to keep gettingbetter. But it's like, it got really good in one dimension and the dimension that they're notgetting better in because it's hard, expensive. Maybe we need to like get a lot more creative with how wedesign these, these verifiers and benchmarks is in how do I make code that in three months is going tolike improve the productivity of humans and agents, mostly agents, but humans and agents in the codebase instead of making it worse over time. And so you think that part is just missing? We haven't seentoo much improvement. I haven't seen, obviously no one knows what the labs are doing internallybecause it's all very secret, but I think if we looking at where the bench, the benchmarks tend toreflect where the labs are, right? If there is no benchmark that can convey to me, did this model

，是我们把模型和 harness 一起训练。所以模型变得非常擅长调用那个 harness 里的特定工具，非常擅长读文件、写文件、搜文件这些事——都是通过做这些题练出来的。这就是为什么它感觉比之前所有命令行编码 agent 都好那么多。

于是大家说：好，那它已经好了这么多，它会一直变好下去的。但问题是：它是在「一个」维度上变得非常好。而它没有变好的那个维度——因为那个维度很难、很贵，也许我们需要在设计验证器（verifier）和基准上创造性得多——那个维度是：我怎么写出这样的代码，让它三个月后能提升人和 agent（主要是 agent，但也包括人）在这个代码库里的生产力，而不是让情况随时间变糟。

所以你觉得那一块就是缺失的？我们还没看到太多进步？

我没看到。当然没人知道实验室内部在做什么，因为都是高度保密的。但我觉得，基准往往反映了实验室在往哪儿走，对吧？如果没有任何一个基准能告诉我「这个模型

[55:03] Dex Horthy

write code that is going to make my code base better or worse? The best we have is I think frontier codefrom the cognition team is really interesting. They have like, did the test pass and then they'd havelike two layers of model review. So they have a judge model that checks, okay, is the patch,the model made similar to the patch that is like the golden answer set. So even if the model didn'twrite the exact code that the benchmark was expecting, did, was it functionally equivalent?

写出的代码，会让我的代码库变好还是变坏」——我们目前最好的东西，我觉得是 Cognition 团队的 frontier code，挺有意思。他们的做法是：先看测试过不过，然后加两层模型评审。他们有一个「裁判模型」去检查：模型做出的这个补丁，跟那个「标准答案」补丁像不像？也就是说，即使模型写的不是基准期待的那份代码，它在功能上是不是等价的？

[55:27] Dex Horthy

And the next one is like a like code quality review from another judge model. And like, that's better,but it's not, it's not sufficient. And this is why I also think agentic code review is like,yes, it will catch things and it will raise your floor. But I don't believe like the model writing thecode is the same model reading the code. And if you ask a model, Hey, is this code good?

下一层是另一个裁判模型做代码质量评审。这更好了，但还是不够。这也是为什么我觉得 agent 化代码评审——是，它会抓到一些东西，它会把地板抬高。但我不相信「写代码的模型和读代码的模型是同一个模型」这件事。而且如果你问一个模型「嘿，这段代码好吗？」

[55:46] Dex Horthy

And it's going to be like, Oh yeah, it's great. Comprehensive. It's got unit tests. You've triedthis. I'm sure. And you say, okay, review this PR that my coworker wrote and tell me everythingthat's wrong with it. And I was like, Oh, it has this problem and this problem and this,but this is sycophantic. And they want to tell you what you want to hear. And so like,it's really hard for me to trust a model to evaluate the quality of, of, of code that's written.And so I have some ideas on like, okay, can you build a benchmark where the model builds 20features in a row and maintains the code base the whole time? And it doesn't know what features arecoming. You treat it like a real product team where you don't know what you're going to buildnext week until you get there and you find out what's most important. And then can we try toevaluate, like, can we build a problem like that? That's hard enough that most frontier models failby issue six or seven.Is it fair to say that, you know, like we've had the software factory, like before AI, it was justlike lots of loop. It was like the PM giving a ticket to the dev, the dev building it, deployingto production user customers, using it, customer support, getting tickets, and then creating

它会说：哦当然好，很全面，还带单元测试。你肯定试过这个。然后你说：好，评审一下我同事写的这个 PR，把它所有的问题都告诉我。它就会说：哦，它有这个问题、这个问题、还有这个。这就是谄媚（sycophantic），它们想说你想听的话。所以我很难信任一个模型去评估「写出来的代码」的质量。

所以我有一些想法，比如：你能不能造一个基准，让模型连续造 20 个功能，并且全程维护这个代码库，而且它不知道后面会来什么功能？你把它当成一个真实的产品团队——你不知道下周要造什么，直到那时候你才发现什么最重要。然后我们能不能去评估……我们能不能造出这样一个问题，难到让大多数前沿模型在第六、第七个 issue 就挂掉？

这么说公平吗：AI 之前我们就已经有软件工厂了，它就是一堆循环——产品经理给开发一张工单，开发造出来、部署到生产，客户在用，客户支持收到工单，然后

[56:47] Gergely Orosz

PM triaging. And it just kind of goes around by in this loop. Is it fair to say that thesoftware factory of how a company, a team builds and maintains software that is changing becausenow everyone's replacing some parts of it, you know, maybe the, the least advanced teamswill just be, devs are starting to use cloud code or codex to write faster. They're not spendingas much time on there. Some others are also having the deployment, the feedback. Some,some actually have the agents already one-shotting bucks. So like, is it fair to say that the softwarefactory is just, it's just changing everywhere, maybe at different speeds, but everyone, I thinkevery team who is building production software, they're like, they're frantically experimenting,trying, and everyone's at a different pace. You'll have the AI native starters where most of thiswill have agents in them and you'll have the, the laggers who are, or more, more cautiousones. They have agents in a few places, but not on the others.Well, and I think that's the key is like, if you want to do loops engineering, you shouldbuild one loop at a time and you should keep them small and contained. Basically, I thinkeverything except stop reading the code is really good advice. Take support tickets and turn them

产品经理分诊，就这样在这个循环里转。那么这么说公不公平：一家公司、一个团队「造和维护软件」的方式——也就是软件工厂——正在改变，因为现在每个人都在替换其中的某些部分？最不先进的团队可能就是开发开始用 Claude Code 或 Codex 写得更快，不再在那上面花那么多时间；另一些团队还改了部署和反馈环节；还有一些人的 agent 已经能一枪修掉 bug 了。所以这么说公平吗：软件工厂就是在到处都在变，也许速度不同，但每个在做生产软件的团队都在疯狂地试验、尝试，每个人节奏不一样？会有 AI 原生的新公司，大部分环节里都有 agent；也会有落后的、或者说更谨慎的，只在少数几个地方有 agent、其他地方没有。

对。我觉得关键就在这儿：如果你想做循环工程，你应该一次只造一个循环，并且保持它们小而受控。基本上，除了「别读代码了」这一条，其他建议都很好。把支持工单变成你系统里的工单，

[57:52] Dex Horthy

into tickets in your system. And then maybe turn those into PRs. Great. The advice that I haveand like what we kind of like are chasing at human layer is like, how can I add another checkpointin that factory? So instead of having one human viewpoint where you're reviewing PRs and sometimesthere are a hundred lines and sometimes there are a thousand lines, but it's quite a lot of effortfor, especially if it's bad, especially if it needs a rework, it's quite a lot of effort for a human tobe like, okay, this is wrong. Go change it in this way. And then you loop back to the agent and thenyou come with another one and like doing a lot of loops on there. Once, once the direction has beencommitted to, it's really hard to steer off. Like you're better off just kind of restarting fromscratch. How do you build like controls and mechanisms around that? And then my take islike, if you do a little bit of human agent planning and like discussion before you handit to the implementer, whether it's, I mean, planning and specs, whatever you want to callit again, this is spectrum development is another word that has become kind of very like muddled asfar as what it means. But basically how can we spend an hour before we start building so that the PR,

然后也许再把那些变成 PR，很好。我的建议、以及我们在 HumanLayer 追的东西是：我怎么在这座工厂里再加一个检查点？现在你只有一个「人的观察点」，就是评审 PR——有时是一百行，有时是一千行。这对人来说是相当大的工作量，尤其如果它是差的、尤其如果它需要返工：人要说清楚「这里不对，按这个方式改」，然后绕回给 agent，然后又来一版，在那儿做很多轮循环。而一旦方向已经定死了，就很难再掰回来，你还不如从头重来。

那你怎么围绕这件事造出控制和机制？我的看法是：如果你在把活儿交给「实现者」之前，先做一点人和 agent 的共同规划、共同讨论——不管你管它叫规划还是叫规格，「规格驱动开发」这个词又是一个已经被搅得很浑浊的词——但基本上就是：我们怎么在开始造之前花一个小时，好让那个 PR

[58:55] Dex Horthy

when we read it only takes 20 minutes because the code is perfect instead of not touching it,just literally saying every user reported issue becomes a PR through the loop. And then we readthat PR and it takes six hours because it's back and forth with making changes and things. It's all,I'm all about like, let's find leverage. And so you basically, you have three optionsin the software factory world. If you're going to go all in on agentic software factories,you can turn the lights off and just let everything flow and pray that you don't create too muchslop and pray that the next generation of models comes fast enough before you create a giant pileof ash. You can slow way down and read every PR and read every line of code. Uh, and then you'reonly going to really get modest benefits from AI because that becomes, I think you should expectmaybe 30 to 50% lift in productivity is kind of what I see when we go into teams. Or you can findthe right leverage points where humans can actually, an hour spent over here in planning can save youfour hours in implementation in terms of fixing and going back and getting the design right.And that's what I call like seeking leverage. You can find the right leverage points for the agents

在我们读它的时候只花 20 分钟，因为代码是完美的？而不是完全不碰这一步，字面意义上「每一个用户报告的问题都通过循环变成一个 PR」，然后我们读那个 PR 要花六个小时，因为要来回改。我一直讲的就是：我们去找杠杆。

所以在软件工厂这个世界里，如果你要在 agent 化软件工厂上全押，你基本上有三个选项。第一，你可以关灯，让一切自己流动，然后祈祷你别造出太多垃圾（slop）、祈祷下一代模型来得够快，赶在你堆出一座灰烬山之前。第二，你可以大幅放慢，读每一个 PR、读每一行代码——那你从 AI 拿到的收益就只是温和的：我们进到团队里看到的大概是 30% 到 50% 的生产力提升。第三，你可以找到正确的杠杆点：在这里花一个小时做规划，能在实现阶段替你省下四个小时的返工和「把设计搞对」的来回。

这就是我说的「寻找杠杆」。你可以给 agent 找到正确的杠杆点

[1:00:01] Dex Horthy

to guide the work. Then you can actually move like two to three times faster while maintaining a like99% like accuracy to like, if the humans were carefully writing this code by hand, how would it come out?

去引导工作。那样你其实可以快两到三倍，同时保持大约 99% 的准确度——也就是「如果由人小心翼翼地手写这段代码，它会长成什么样」。

[1:00:14] Dex Horthy

Now, jumping a little bit back to ideas, I will come back to this. This was earlier,maybe it was last year, but you had the research plan implement. Can we talk about the originalresearch plan implement framework? And then also what you've learned about this approach? What,what you got wrong about it? Yeah, sure. Yeah. So, um, I mean, the first time we talked about RPIwas in August of 2025. Um, and it was basically like the research was this thing of like, Hey,before you go build anything, go read lots and lots of code, use a bunch of sub agent and sub agents inparallel, understand all the code. It was this technique that like worked really well for hard problems and complexcode bases. You just ask Claude, uh, to do a thing that that's, it would read three files and make achange. It would have no context. So you start the research. You don't even tell it what you'reworking on. You just tell it, Hey, can you tell me how this system works and this system and how theyconnect together? And then you get a markdown doc out. And this was the context engineering part islike, that would take a hundred thousand tokens of context, but you would get a 10 K token doc out ofit that summarized it. Then you would start a new context window and you would do planning.

现在稍微往回跳一点，我等会儿还会回来。这个更早，也许是去年——你提出过「研究—规划—实现」（research, plan, implement，RPI）。我们能聊聊最初的 RPI 框架吗？以及你后来学到了什么、你在哪儿搞错了？

当然。我们第一次谈 RPI 是 2025 年 8 月。「研究」这一步基本上是：嘿，在你造任何东西之前，先去读大量大量的代码，并行用一堆子 agent，把所有代码都搞懂。这个技法对难题和复杂代码库特别管用。你直接叫 Claude 做一件事，它会读三个文件就动手改，它没有上下文。所以你先做研究，甚至不告诉它你要干什么，你只说：嘿，能不能告诉我这个系统怎么工作、那个系统怎么工作、它们怎么连起来的？然后你会拿到一份 Markdown 文档。而这就是上下文工程的部分：那件事会烧掉十万 token 的上下文，但你会拿到一份一万 token 的文档，把它总结出来。然后你开一个新的上下文窗口，去做规划。

[1:01:14] Dex Horthy

And the planning would be, and actually realized like the plans that we were building last summerwere actually terrible, but it would basically be this long, you'd say, okay, now here's whatwe're building. Here's the research doc, build a plan to implement it. And, uh, in retrospect,now that we see like everyone is obsessed with how do I get agents to work for longer? I think thereason why in like May, June, July, August of 2025, that a lot of people became really interested inplanning was, it was a very powerful lever to get agents to work for longer. If you said, build me aB2B SaaS for a burrito delivery, you'd get like a homepage and that's it. But if you said, build mea plan, it would build out this big plan. And then in the next context window, you'd say, Hey, here'sthe plan. Here's all the changes we're going to make go implement. It would actually keep going untilthe plan was done. So the plan was a really good way to anchor an agent and remind it that like,Hey, you're not done until this is all finished. So that was the original RPI. And the plan doc,what was bad about it is it didn't give you leverage. The plan was every single line ofcode that was going to change like in diff blocks and like all the new stuff to write.

规划这一步是——其实我们后来意识到，我们去年夏天做的那些计划真的很糟糕——但它基本上是这样一大段：好，这是我们要造的东西，这是研究文档，做一份实现计划。

回头看，现在大家都痴迷于「我怎么让 agent 干得更久」。我觉得 2025 年五六七八月很多人对「规划」产生浓厚兴趣的原因是：它是一根特别强的杠杆，能让 agent 干得更久。如果你说「给我造一个卷饼配送的 B2B SaaS」，你会拿到一个首页，然后就没了。但如果你说「给我造一份计划」，它会写出一份大计划。然后在下一个上下文窗口里你说：嘿，这是计划，这是我们要做的所有改动，去实现。它就真的会一直干到计划做完。所以计划是一个特别好的「锚」，能不停提醒 agent：嘿，这些没全做完你就没完事。

这就是最初的 RPI。而这份计划文档的问题是：它没给你杠杆。计划里写的是每一行将要改动的代码，以 diff 块的形式，还有所有要新写的东西。

[1:02:15] Dex Horthy

And so like people would review these plans. We recommended this. We told people to read theplans. We read all our plans. And then eventually I found myself like, I just kind of skimmed the plans.And so you're not really using it as a way to re-steer the agent. It's just kind of there.And then you go write the code and there's a crap. Some people would review the plans and the code.And it's like, okay, well, the plan was took you 20 minutes to read. And then the pull request takesyou 20 minutes to read. And they're different. And so you actually doubled the amount of timeyou're spending reading code instead of like doing less of it. You've been anti-leverage.And hang on, was spec different development not related to this? The one that Amazon Kiro,for example, and GitHub workflows again a year ago did, which was it also, it first generated a planand it had the human review it. And then it started to, and you could edit it as well. And then it wentoff and implement this part and it looked beautifully on the surface. It should have worked great,but it's tossed into the garbage outside of some maintenance projects. I think it just didn'twork. Like all the feedback I got, people just stopped using it because it just didn't really

于是人们会去评审这些计划。我们推荐过这么做，我们让别人读计划，我们自己也读所有的计划。但后来我发现自己只是随便扫一眼。所以你其实并没有把它当成「重新引导 agent」的方式，它就只是摆在那儿。然后你去写代码，麻烦来了：有些人计划也评审、代码也评审。结果是：计划花你 20 分钟读，PR 又花你 20 分钟读，而且两者还不一样。所以你其实把读代码的时间翻倍了，而不是变少。这是「反杠杆」。

等一下，规格驱动开发（spec-driven development）跟这个不是有关系吗？比如亚马逊的 Kiro，还有一年前 GitHub 的那套工作流——它也是先生成一份计划，让人评审，你还能编辑，然后它再去实现。表面上看起来很漂亮，本来应该效果很好，但除了少数维护型项目之外，它基本被扔进垃圾桶了。我觉得它就是不管用。我收到的所有反馈都是：人们就是不再用了，因为它真的没那么

[1:03:14] Dex Horthy

work that well. It just rhymes to the RPI framework a little bit, the original one, right?Well, so our thing too, like the biggest difference between RPI and spec-driven development,and some people refer to RPI as spec-driven dev, because for some people, SDD, all it means isI use a bunch of markdown files while I'm coding and forget what's in them. I just spec-driven dev,those are my specs and I'm using them to drive development. There was this open AI researcherwho talked about spec-driven dev and like, hey, stop reading the code, just write the specs and treatlike the coding part is compiling specs into code. That part never really materialized,maybe with GPT-7, you know? But the challenge, I'm on a GitHub issue in spec kit that has beenopen for a year and every couple of weeks, I get it. There's a new email on the thread of peoplecomplaining about this problem, like, okay, I edit my specs and then I edit the code and the codedrifts and the specs. How do I keep the specs up to date as the code is changing? And it's basicallylike, you now have two sources of truth and it stops being useful. And so like, that's why when RPI,the idea of the docs is they're all, for a while we kept them around, but after two or three months,

好使。它跟最初那版 RPI 框架有点像，对吧？

嗯。我们这套跟规格驱动开发最大的区别是——有些人把 RPI 也叫作规格驱动开发，因为对有些人来说，SDD 的全部含义就是「我写代码时用了一堆 Markdown 文件，然后忘了里面写了啥」，那就是我的规格，我在用它们驱动开发。曾经有一位 OpenAI 的研究员谈规格驱动开发，说：嘿，别读代码了，只写规格，把编码这一步当成「把规格编译成代码」。那部分从来没真的落地，也许 GPT-7 能做到，谁知道呢。

但挑战在于：我在 spec kit 的一个 GitHub issue 上挂了一年，每隔几周我就收到一封新邮件，是有人在那个帖子里抱怨同一个问题：好，我编辑我的规格，然后我编辑代码，然后代码和规格就漂移了。代码在变，我怎么让规格保持最新？本质上你现在有了两个事实源，它就不再有用了。

所以在 RPI 里，这些文档的定位是——有一阵子我们也把它们留着，但两三个月后

[1:04:17] Dex Horthy

we're like, oh, these are actually like tactical execution docs. I do the research, I do the plan,I do the implementation, I throw the docs out. And the next time I need research, I just do it fromscratch because tokens are cheap and my time is expensive. And the amount of time I might wasteif I reuse a research that is no longer in sync with the real state of the code base. So we justcreate it live every time. This is why it's like context engineering still matters. Creating artifactsthat compress the state of the code base and compress the intent of the builder into small things thatcan be reused in the future for the scope of a task is like a very powerful, like tactical approach.But it's not a thing like I have very few opinions on like, what sorts of docs that you should leavelying around your code base that are like evergreen. I've seen people try to maintain parity betweendocumentation or specs and the code itself. And I don't think anyone actually like found it very useful.Like you can do it and it works, but it's like the ratio of the effort it takes to keep them up to date.And trivially, you could do this with AI probably, but I've never known anyone who was like, yeah,this is great and we're glad we have it. Like you could do it and it might help, but I don't think

我们就想：哦，这些其实是「战术执行文档」。我做研究，我做计划，我做实现，然后我把文档扔掉。下次我需要研究，我就从头再做一遍——因为 token 便宜，我的时间贵。而如果我复用一份已经跟代码库真实状态不同步的研究，我可能浪费的时间更多。所以我们每次都现做。

这也是为什么上下文工程仍然重要：造出「把代码库状态压缩、把构建者意图压缩」成小东西的产物，让它能在一个任务的范围内被复用——这是一个非常强的战术手段。但对于「你该在代码库里留下什么样的常青文档」，我几乎没有什么看法。我见过有人试着维持文档或规格与代码本身的一致性，我不觉得有谁真的觉得那很有用。你可以做，它也能跑，但那个「维持更新所要花的力气」的比例……当然，用 AI 你大概能轻松做到，但我从没见过谁说「对，这太棒了，我们很庆幸有这个」。你可以做，它也可能有帮助，但我不觉得

[1:05:23] Dex Horthy

anyone found it useful enough to like maintain a system to keep the specs in the code in sync versusjust using the code as the source of truth always. You mentioned something interesting, which is withcontext engineering, you need to sometimes compact. And you've previously talked about intentionalcompaction that when context is noisy, deliberately compress the useful part into a clear,like mark that artifact, verify it, and then start a fresh conversation. Can we talk about this kindof compaction and why it's important? And, and it sounds like it's going to be a building block whereit already is for context engineering, right? Yeah. No, frequent intentional compaction isthe building block. It is, it is completely comes from context engineering is context engineering islike, how do we get the most out of today's models? How do we change what we're putting into the model,into the context window, into the agentic chat? How do we control that in such a way that we get thebest results possible, which means doing as much work as possible in the smart zone, the, you know,first hundred thousand tokens of the context window. And, uh, this intentional, frequent,intentional compaction is basically like, okay, the research step, we're going to go read a bunch

有人觉得它有用到值得去维护一套「让规格和代码保持同步」的系统——相比之下，永远把代码当作唯一事实源就够了。

你刚提到一个很有意思的点：做上下文工程时，你有时需要压缩（compact）。你以前谈过「有意压缩」（intentional compaction）：当上下文变嘈杂时，刻意把有用的部分压缩成一份清晰的、标记好的产物，验证它，然后开一段全新的对话。我们能聊聊这种压缩、以及为什么它重要吗？听起来它已经、并且会继续是上下文工程的一块基石，对吧？

对。频繁的、有意的压缩就是那块基石。它完全是从上下文工程里长出来的：上下文工程问的是——我们怎么从今天的模型里榨出最多？我们怎么改变放进模型、放进上下文窗口、放进 agent 对话里的东西？我们怎么控制它，好拿到尽可能好的结果？这意味着尽可能多的工作要发生在「聪明区」（smart zone），也就是上下文窗口的前十万 token 左右。

而这种有意的、频繁的压缩基本上是：好，研究这一步，我们要去读一堆

[1:06:26] Dex Horthy

of code and turn it into a doc. That's our compaction. We take that forward in the next session.We're going to read, we're going to read the ticket and the intent and turn that into a designdocument that we call it. It was like, okay, here's the high level spec of what we want to do.Here's the high level, like current state desired end state. And then a bunch of design questionsthe model has kind of like a very thorough, maybe even over-engineered like plan mode.And then you take the research and the design and you do a new session, new context when you're like,cool, you've compressed the intent and you've compressed the state of the code base so thatyou can then do your planning of like, okay, we know what the end state looks like. We know wherewe're going. Now let's break down how we're going to get there. All of these different steps of theprocess exist because models have shortcomings in each of these phases. So the research is prettyhands-off. I don't read the research docs. It's just like, go read a bunch of code and then likemake a doc out of it. Models are pretty damn good at that. If you ask it to find a bug and haveopinions about the code base, that's different. But if you just ask it, what is the intent and how

代码，把它变成一份文档，那就是我们的压缩，我们把它带进下一段会话。然后我们读工单和意图，把它变成一份我们叫「设计文档」的东西：好，这是我们要做的事的高层规格，这是高层的「当前状态 → 期望的最终状态」，再加上一堆模型提出的设计问题——有点像一个非常彻底、甚至可能过度工程化的 plan mode（规划模式）。

然后你拿着研究文档和设计文档，开一段新会话、新的上下文窗口：好，你已经把「意图」压缩了，也把「代码库状态」压缩了，那你现在可以做规划了——我们知道终点长什么样，我们知道要去哪儿，现在来拆解怎么走到那儿。

这个流程里的每一步之所以存在，都是因为模型在那个阶段有短板。研究这一步是相当撒手的，我不读研究文档，就是「去读一堆代码，然后写成一份文档」，模型在这件事上真的挺强。如果你叫它找 bug、对代码库发表看法，那是另一回事。但如果你只是问它：意图是什么、

[1:07:21] Dex Horthy

this stuff fit together? That's usually pretty straightforward, but designing the end stateof the software, the architecture and the program design models are not great at. They make a lot oflike, they make decisions and sometimes they're right and sometimes they're wrong. So we want to have ahuman in the loop there. And then the steps to get there, we talked about this before,but models love making what I call like horizontal plans. If you ask a model, like build a plan ofsteps to go build this app, it's like, cool, we're going to do the database and then we'regoing to do the services layer. Then we're going to do the API and then we're going to do the frontend. And it's like, well, that actually kind of sucks because we're going to be on the other sideof 2000 lines of code. And let's imagine this is an existing code base, right? We're going to makechanges to all these different parts of the system. I can't test it till the end. And so what I woulddo is like, okay, how would I have built this if I were building my hand? Well, okay, I would probably create a mock APIendpoint with fake data. And then I would go kind of get the front end, kind of how Iwant it to look. And then I would actually go like build a services layer and actually wire the data

这些东西怎么拼在一起？那通常挺直接的。

但「设计软件的最终状态」——架构和程序设计——模型不擅长。它们会做很多决定，有时对、有时错。所以我们希望那一步有人在环（human in the loop）。

然后是「走到那儿的步骤」，我们之前聊过：模型特别爱做我称之为「横向计划」的东西。你要是让模型「做一份造这个应用的步骤计划」，它会说：好，我们先做数据库，然后做服务层，然后做 API，然后做前端。可这其实挺糟糕的，因为我们要在 2000 行代码之后才走到另一头。而且想象这是一个已有的代码库，我们要改动系统里所有这些不同的部分，我到最后才能测试。

所以我会做的是：好，如果这是我自己手写，我会怎么造？我大概会先做一个带假数据的 mock API 端点，然后把前端弄成我想要的样子，然后再去造服务层、把真实数据接通，

[1:08:14] Dex Horthy

through. And then I would make a database migration and make my new table. And then I would actuallyadd a lot of business logic. And then I would add a bunch of error handling. And it's completelyorthogonal to how model like models will write the database layer and all the error handling withoutever like anyone's ever touched or seen the code or whatever it is. And so this is another place wherewe like to have humans involved because humans have really good taste and judgment. And like,I would rather read five separate little mini diffs of like things that I can manually verify andexplore than read 2000 lines of code. And be like, well, it's not working. I don't know where youdon't know where, cause you wrote the code. You were supposed to get it right. We talk aboutcompaction context engineers. Like how can you stay in the smart zone of the context window, which isagain, the dumb zone. I will say disclaimer, it's really good training wheels. If you don't haveintuition about this. So let's just define these things. What is the smart zone and what is thedumb zone? So it's, it's, it's a little bit blurrier than like, I would like, I would like it to be.I think in November we, we talked and said, Oh, that's about the first 40% of the context window.

然后做一个数据库迁移、建我的新表，然后再加大量业务逻辑，然后再加一堆错误处理。这跟模型的做法完全正交——模型会在没有任何人碰过或看过代码的情况下，就把数据库层和所有错误处理都写完。

所以这是另一个我们喜欢让人参与的地方，因为人有非常好的品味和判断力。而且比起读 2000 行代码然后说「它不工作，我不知道哪儿出问题」——你也不知道哪儿出问题，因为代码是你写的、你本该写对——我宁愿读五个独立的小 diff，每个我都能手工验证和探索。

我们聊压缩、聊上下文工程，说的都是：你怎么待在上下文窗口的「聪明区」，与之相对的还有「笨区」。我要加个免责声明：如果你对这件事还没有直觉，这套说法是很好的辅助轮。

那我们就把这两个词定义一下：什么是聪明区，什么是笨区？

它比我希望的要模糊一些。我记得十一月我们聊的时候说过：大概是上下文窗口的前 40%。

[1:09:13] Dex Horthy

But then we had million smart zone. Yeah. Then we had million token context windows. So then Ichanged it to like the first hundred thousand tokens. If it's a really like 4.8, I usually will go upto like 200 K. But basically the thing Jeff Huntley had and Ralph Wiggum was like the less contextwindow you use the better outcomes you'll get. And basically the smart, the smart zone mean,meaning if you have context in that first part, it should work a lot better. And then like the dumbzone is like, once you have stuff there is kind of forget about it. Like it'll be confused. It'snot going to do much. Like it'll degrade. Yeah. And there were times, and this is an intuition thing.Like I will often go up to three, 400 K token for is rare, but I will go up to 250, 300 K tokensfor certain types of work where my intuition tells me that I can keep working without,without degrading the performance. But if you don't have good LLM intuition, like a hundredK for smaller models, 200 K for these like really beefy, like codecs and Opus 4.8 modelsis usually a good like training wheel guideline of like, if you pass there, your quality of resultsmay be degrading. The biggest tell I see for this is often the, uh, models trying to get the test to

但后来我们有了百万 token 的上下文窗口。所以我把它改成了「前十万 token」。如果是像 4.8 那种很强的模型，我通常会放到 20 万。但基本上，Jeff Huntley 和 Ralph Wiggum 那套的核心就是：你用掉的上下文窗口越少，你拿到的结果越好。

所谓「聪明区」是说：如果你的上下文在前面那一段，效果会好很多。而「笨区」是：东西一旦到了那儿，基本就可以忘了它——它会犯迷糊，做不了什么，性能会退化。

对。而且有时候——这也是个直觉活儿——我偶尔会上到三四十万 token，这很少见，但对某些类型的工作我会上到 25 万、30 万 token，因为我的直觉告诉我我可以继续干而不掉性能。但如果你没有好的大模型直觉，那么小一点的模型十万、像 Codex 和 Opus 4.8 这种很壮的模型二十万，通常是个不错的辅助轮准则：过了那条线，你的结果质量可能就在退化了。

我看到最明显的信号，往往是模型在试着让测试

[1:10:18] Dex Horthy

pass and your 200 K token. Well, let me try this. Okay. Let me try that. And it's like trying a bunchof stuff and it's getting more and more extreme and it's like thing. Oh, let me delete your dotend file and try again. Like this is where things get really, really weird. And so it's like, if youstart to see certain types of, if I'm like, Oh, we're at 300 K tokens and I need to like fix theunit test. I'm like, cool. Right. Everything we did do a file or even I'll just do like a built-incompaction depending on the model. And then I'm starting a new session at 30 K or 50 K tokens. I'm like,cool. We're going to do a hard thing, which is you're going to get this fricking test to passand you're not going to be stupid about it. By the way, one thing that you said like about,about the, the, the model being dumb is you said that if the model ever tells you, you are absolutelyright, you should start over. And we've all had that when it tells me like, Oh, you know, you didn't,you're absolutely right. And I'm like, we just get annoyed, but why should we start over? What'shappening there in your observations? Yeah, that's great. Yeah. And the new, the new,you're absolutely right. I think is a, you're right to push back on that. Right. Yes. That's Opus,

通过，而你已经在 20 万 token 了。「我来试试这个。好，我来试试那个。」它在乱试一堆东西，而且越来越极端，最后变成「哦，让我把你的 .env 文件删了再试一次」。这时候事情就变得非常非常怪了。

所以就是：如果我开始看到某些类型的信号，比如「哦，我们已经 30 万 token 了，而我还得修这个单元测试」，那我就说：行，把我们做过的一切写进一个文件，或者干脆用模型内置的压缩功能，然后我在 3 万或 5 万 token 处开一段新会话，说：好，我们来做一件难事，你要把这个该死的测试跑通，而且别犯傻。

对了，你还说过一件关于「模型变笨」的事：如果模型对你说「你说得完全对」（You're absolutely right），你就该重开。我们都遇到过这种情况，它说「哦，你说得完全对」，我们只是觉得烦。但为什么该重开？在你的观察里，那时候发生了什么？

对，问得好。新版的那句是「你这么反驳是对的」（You're right to push back on that）。对，那是 Opus，

[1:11:17] Dex Horthy

right. Yeah. Opus is like, you didn't run the test. Did you, you're right to push back on that.I totally did it. But no, for me, you're absolutely right. It was always what the model would respond.If you were like, that's totally wrong. You did it. Like you, if you were, if you said somethingwhere you were angry or frustrated, or just wanted to point out that it's done something wrong,it would respond with, you're absolutely right. And most of us have had the experience of,it says that, and then it continues to do the wrong thing. So it's like, once it starts doing dumbthings, because there's, there's four things in your context window that matter. There's likethe size of it, how many tokens there's like the quality of the information is like,is there any incorrect information? Like if the model had some thinking trace where it decidedthe wrong thing was true, is there missing information? Does this is like have contextmissing that it should have. And then there's the trajectory and the trajectory is very subtle,but you may have had sessions. The trajectory, meaning you're promptingthe actual history of everything. I call it trajectory is like the actual history of like what the agent has

对。Opus 会说：你没跑测试，是吧？你这么反驳是对的，我确实没跑。而对我来说，「你说得完全对」永远是模型的那句回应：如果你说「这完全错了，你搞砸了」——如果你说话带着生气、沮丧，或者只是想指出它做错了什么——它就会回「你说得完全对」。而我们大多数人都经历过：它这么说了，然后继续做错的事。

所以一旦它开始做蠢事……你的上下文窗口里有四样东西重要：一是它的大小，有多少 token；二是信息的质量——里面有没有不正确的信息？比如模型有一段思考痕迹，在里面认定了某个错误的东西是真的；三是有没有缺失的信息——它是不是缺了本该有的上下文；四是「轨迹」（trajectory），这个非常微妙，但你可能遇到过这样的会话。

轨迹是指你的提示词吗？

我说的轨迹，是指 agent 过去实际做过什么的完整历史。

[1:12:12] Dex Horthy

done in the past. And so if I say, Hey, make this change. And the agent makes the change. And then it runsthe test and they're broken. And then it fixes the test. I have very high confidence. The next changeI asked it to make, it's going to follow that path again, because it's like, okay, here's aconversation. And the last time the user asked me to do a thing, I made the change. I ran the test,test broken, fix the test. And then I told the user, but if I say make a change and it makes thechange, it doesn't run the tests. Then I'm on a different trajectory. And if I say, okay, makeanother change. It's like, basically the they're auto-regressive. So they're, they're predicting theneck. What's the next message in this conversation. And so the example we talked about in, uh,no vibes allowed was of course they're like, Hey, the model makes a mistake. And then you yelled atit. And then it made another mistake. And then you yelled at it. And then it's like, cool. What's thenext message in this conversation? Well, look, if I read the history, I should probably make anothermistake so the human can yell at me. So I was like, okay, that's a great, that is a great example oflike a time to start over. Let's talk about some observations on how software engineering is

所以如果我说「嘿，做这个改动」，agent 做了改动，然后它跑测试，测试挂了，然后它修好了测试——我有很高的信心：我要它做的下一个改动，它还会走同一条路。因为它会想：好，这是一段对话，上一次用户让我做一件事，我做了改动，我跑了测试，测试挂了，我修了测试，然后我告诉了用户。但如果我说「做个改动」，它做了改动，它没跑测试，那我就在一条不同的轨迹上了。然后我说「好，再做一个改动」——本质上它们是自回归的，它们在预测「这段对话里的下一条消息是什么」。

所以我们在《不许 vibe》（No Vibes Allowed）里讲过的那个例子是：嘿，模型犯了个错，然后你冲它吼；它又犯了个错，你又冲它吼。然后它就想：好，这段对话的下一条消息是什么？看，如果我读一遍历史，我大概应该再犯一个错，好让人类再吼我一次。所以我觉得，这是一个特别好的「该重开」的例子。

我们来聊聊你对软件工程正在如何变化的一些观察。

[1:13:05] Dex Horthy

changing. One thing you talked about recently on the evolution of the coding meta is going from tokenharder to token smarter. Can we talk about what you mean by token harder and token smarter?Yeah. So token harder is, I mean, I'm in a, I'm in a group chat called hyper engineering and it's alllike people trying to max out their cloud subs. Oh, wow. Okay. It's just like, okay. That soundslike a fun, fun place. It's a fun place, but it's like all token harder. It's like, look at all theside projects I built. It's look at everything that, uh, I I I've gotten my cloud token. I've got six,six cloud code accounts. I've gotten all of them maxed out every five hour period. I've timed it out.So I always use all the tokens and it starts up immediately when the limit resets. And so it'slike, I mean, getting into Eli gold rat. And the goal is like optimizing for utilization andefficiency of one node in your factory, rather than the end to end goal of like, how do we ship valueand things that people like that are stable and like will last a long time. But that's my idea oftoken harder. And it's the same thing with the dark factory thing is like, Hey, if you, if you,if you remove humans from code review, you can push more tokens through the system.

你最近谈过「编码 meta（主流打法）的演化」：从「更用力地烧 token」（token harder）到「更聪明地烧 token」（token smarter）。我们能聊聊这两个词是什么意思吗？

对。token harder 是——我在一个叫「hyper engineering」的群聊里，里面全是想把自己 Claude 订阅额度用满的人。

哇，好吧。

听起来是个好玩的地方。

是挺好玩的，但那全是 token harder：看我造了这么多副业项目、看我把我的 Claude token 全用上了、我有六个 Claude Code 账号、我把它们每一个的每五小时窗口都跑满了、我算好了时间，所以我永远把 token 用光，而且额度一重置就立刻开跑。

这就是——又回到 Eli Goldratt 和《目标》了——这是在优化你工厂里「某一个节点」的利用率和效率，而不是端到端的目标：我们怎么交付价值、交付人们喜欢、稳定、能长久的东西。但那就是我说的 token harder。黑灯工厂也是一回事：嘿，如果你把人从代码评审里拿掉，你就能往系统里推更多 token。

[1:14:09] Dex Horthy

So we talk about software factories, but what is the dark factory?Ah, so the dark factory is this comes from this idea of like, there are factories where, uh,everything is automated by robotics. So you can imagine like a car factory where it's all robotsbuilding the cars and they don't have lights because there's no humans. Oh, so that's whereit comes from dark factory. Yeah. You walk in there, there's no lights. There's not even lightswitches. So it will be the fully automated software factory where it will be like,no human input, basically no human input, raw materials go in cars come out.Yep. And I think in a micro, like you can have many loops that are dark in your, in your thingof like, Hey, if, uh, if the code review agent comes back with a problem, you loop that back tothe builder agent, it fixes it and comes back and that's dark. You don't need a human loop for that.But the full dark factory where you don't read any code. Yeah. It's a good way to maximize yourtoken utilization. It's like, if your belief is like, my job is to extract as much intelligence out ofthe machine God as I can, because that's how I get the most value and the most leverage on my timethan token harder. Um, and my take is basically what we talked about before token smarter is like,

我们一直在说软件工厂，那什么是「黑灯工厂」（dark factory）？

啊，黑灯工厂这个说法来自：有些工厂里一切都由机器人自动化完成。你可以想象一个汽车工厂，全是机器人在造车，而且它们不开灯，因为里面没有人。

哦，原来「黑灯工厂」是这么来的。

对。你走进去，没有灯，甚至连电灯开关都没有。

所以那会是完全自动化的软件工厂：没有人的输入，原材料进去，汽车出来。

对。而且我觉得在微观层面，你的系统里可以有很多个「黑」的循环：嘿，如果代码评审 agent 发现一个问题，你把它回环给构建 agent，它修好再回来——那就是黑的，那个环不需要人。但完整的黑灯工厂，也就是你完全不读任何代码……对，那是最大化你 token 利用率的好办法。如果你的信念是「我的工作就是从机器之神那里榨取尽可能多的智能，因为那才是我获得最大价值和最大时间杠杆的方式」，那就是 token harder。

而我的看法，就是我们之前说的 token smarter：

[1:15:14] Dex Horthy

okay, how do I move faster? How do I get as much value out of as AI as I can without having to turnthe lights off while still maintaining control and taste and judgment and understanding the systemarchitecture and having a lot of like applying my hard one opinions through 10 years of softwareengineering to the design of the program so that I can feel confident that the code is going to getbetter and more maintainable over time. It's the same thing of like, you look at like the SRE teaminside Google, they brought out this book SRE site reliability engineering. And the whole take waslike, Hey, we're going to go from one data center to five data centers. And we need the same six personteam to be able to manage five data centers. And we need the same six person team to be able to manage50 data centers next year. And it's basically, how do we apply software to this problem? So that insteadof scaling linearly of like, okay, every data center needs five DevOps people. So we need to scale thepeople with this things. How do we continually automate the parts that we don't need? So a little bitorthogonal and maybe even like contradictory to what I just said, but this idea of like, how do you find

好，我怎么跑得更快？我怎么从 AI 里拿到尽可能多的价值，同时不必关灯，还能保住控制力、品味和判断力，还能理解系统架构，还能把我十年软件工程里得到的那些来之不易的看法应用到程序设计上，让我有信心「代码会随时间变得更好、更可维护」？

这就像 Google 内部的 SRE 团队一样——他们出了那本《SRE：Google 运维解密》（Site Reliability Engineering）。整个思路是：嘿，我们要从一个数据中心变成五个数据中心，我们需要同样六个人的团队能管五个数据中心；明年我们需要同样六个人的团队能管 50 个数据中心。本质上是：我们怎么把软件用到这个问题上？这样就不用线性扩张——不是「每个数据中心需要五个 DevOps，所以人要跟着规模涨」——而是：我们怎么持续把不需要人的部分自动化掉？这跟我刚才说的有点正交、甚至有点矛盾，但这个想法就是：你怎么找到

[1:16:13] Gergely Orosz

leverage and the way the way? Well, I think what you were saying there is like, Google did never seekto remove those SREs from the process at all. They just said, like, look, can we think ahead and scaleyourselves? And they actually grew the team. It wasn't actually six people. It was more like, I thinkGoogle specifically said, okay, we have five data centers. Next year, we'll have 50. You're six of you.We do not want to have 60 people. We don't want and then management leader and all that is like,how can we do it with like 12 or like, or like 10. And then when we'll have 500 and now actuallytheir SRE has grown, but, but of course, yeah, but, but they never, you know, I think as engineers,like we feel pretty threatened when someone says like, all right, we just want to have zero engineers.Like, I mean, that's not a fun place to work at, but it sounds like it's not a possible place towork at. If they have zero engineers, neither of us can work there. Right. But don't understand thetoken smarter is like, let's keep humans in the loop. Let's keep adding value and figure out what are theparts, which are not as relevant, boring where we don't need it. And so like one developer canprobably do more than before, but you are built to like be part of this whole thing and the lights

杠杆，以及怎么……

嗯，我觉得你刚才说的是：Google 从来没打算把这些 SRE 从流程里拿掉。他们只是说：我们能不能提前想清楚、把自己扩起来？而且他们其实是把团队做大了，并不是只有六个人。我想 Google 具体说的是：好，我们现在有五个数据中心，明年会有 50 个，你们有六个人。我们不希望变成 60 个人——我们不想要那样，还有管理层什么的——我们怎么用 12 个人、或者 10 个人做到？然后我们会有 500 个数据中心，实际上现在他们的 SRE 团队确实变大了。但他们从来没有……我觉得作为工程师，当有人说「行，我们希望零工程师」时，我们会感到相当受威胁。我是说，那不是个好玩的工作场所，但听起来那甚至不是一个「可能存在」的工作场所——如果他们有零个工程师，你我都不能在那儿工作，对吧？

但要理解 token smarter 就是：让我们把人留在环里，让我们持续创造价值，然后找出哪些部分是不那么相关的、无聊的、不需要人的。所以一个开发者大概能比以前做得更多，但你是被设计成这整件事的一部分，而且工厂里的灯

[1:17:19] Dex Horthy

are on in the factory. Yeah. And it's like, basically, I think, I think what I'm trying toget to is like the connection here is like SRE built a thing where like headcount scales at likea square root function or a logarithmic function, whereas their output scales like linearly. And youwant to say that the way you do that is with good architecture and good program design.And so in order to like avoid this problem where you have to throw more people or more tokens atthe problem, if you design good software in such a way that it gets more maintainable and morescalable over time and like just today, it doesn't feel like like basically you need humans in theloop to be able to do that. Let's talk about AI slop. At one point you wrote, yeah, AI can write yourcode, but it can also write your specs and PRDs. But the same rule is always slop in, slop out. If yououtsource your thinking, you're going to get garbage. Yep. So yeah, that's basically the ideais like the way we think about like getting high quality outputs is like, yeah, you could write thecode by hand or you could sit with a model and work back and forth and go maybe a little bit faster.And you have control. And every time it makes a change, you go read the change. And if it's bad,

是亮着的。

对。本质上，我觉得我想说的连接点是：SRE 造出了一种东西，让人头数按平方根或者对数函数增长，而产出按线性增长。而你想说的是，做到这一点的方式是好架构和好程序设计。

所以，为了避免「必须往问题上砸更多人或更多 token」这个困境，你要把软件设计成随时间更可维护、更可扩展的样子。而就今天的感觉来说，你需要人在环里才能做到这件事。

我们来聊 AI 垃圾（AI slop）。你写过：是，AI 能写你的代码，但它也能写你的规格和 PRD。但同一条规则永远成立——垃圾进，垃圾出。如果你把思考外包出去，你得到的就是垃圾。

对。基本想法就是：我们怎么拿到高质量的输出？是，你可以手写代码；或者你可以跟模型坐在一起来回，也许快一点，而且你有控制权，它每做一个改动你就去读那个改动，如果不好

[1:18:25] Dex Horthy

you tell it, nope, we want it like this. And you kind of incrementally slowly. This is likekind of the stage two or stage three version of working with agents where like the agents writingall your code, but you're kind of very much in the loop. And this will make you go faster,but it won't make you go that much faster. It won't make you go anywhere near there's like,there's like that level. And then there's like the maximum speed you can go while still caringabout the code. And then there's like the maximum speed you can go if you turn the lights off.And so we always think about it as like, in terms of leverage is like, okay, let me take everythingstarts with like a sentence or a voice note ramble. Like I want to build this thing and it's going to worklike this, whatever it is. Let's say like on average, like two sentences, I got to fix thisthing. Or there's a support ticket. I got to fix this thing. If you can turn that with AI into aone pager and then turn that one page and make sure that's correct. And then turn that one pagerinto a three pager and make sure that's correct. And then turn that three pager into a 10 page,like detailed outline. Then you can write a hundred pages worth of code. And it's maybe not perfect.

你就告诉它「不对，我们要这样」，你就这样一点点、慢慢地推进。这算是「跟 agent 一起工作」的第二或第三阶段版本：agent 写你所有的代码，但你非常深地在环里。这会让你更快，但不会快太多，远远达不到那个上限。

然后还有一档：在仍然在意代码的前提下，你能跑到的最高速度。再然后是：如果你关灯，你能跑到的最高速度。

所以我们总是从「杠杆」的角度想这件事：一切都始于一句话，或者一段语音随口说的东西——「我想造这个东西，它大概这么工作」，随便什么。平均来说就两句话：「我得修这个东西」，或者有一张支持工单，「我得修这个东西」。如果你能用 AI 把它变成一页纸，然后确认那一页是对的；再把那一页变成三页，确认三页是对的；再把三页变成十页的详细提纲——那你就能写出一百页份量的代码。它也许不完美，

[1:19:18] Dex Horthy

You shouldn't like sweat over these documents and make sure they're perfect,but you're increasing the chance that like you're decreasing the uncertainty of the outputs. It'slike, you can think of like, you have like a line of like where it's going. And then you have likethe probabilities of where like it might go in that range. If you are kind of reviewing along the way,as you get more and more detailed into how, what you're building and how you want it to be built,you kind of collapse the uncertainty and the set of end states that you could land in.That's me doing the physics thing of like, you got to superimpose all these probabilities. And like,I don't know, I have this thing that like, I think people who really like playing real-time strategygames, uh, are probably going to be really good with AI because you kind of have to like,I don't know, Matt Pocock was just talking about fog of war and like things that are at the frontierof like, there's stuff we don't know about this problem yet. How can we find that out? And how can Imake the best decision now knowing what I have seen? There's a, I've seen a couple pieces of information.And so there's a 30% chance it's this, and there's a 40% chance that it's this. How can I get more

你不该在这些文档上死磕、非要它们完美。但你在提高「输出符合预期」的概率，在降低输出的不确定性。你可以想象成：你有一条「它要往哪儿走」的线，然后有「它可能在这个范围里走到哪儿」的一堆概率。如果你在过程中一路评审，随着你对「要造什么、要怎么造」越来越细，你就把不确定性、把它可能落到的终态集合收敛掉了。

这是我在做「物理那一套」——你得把所有这些概率叠加起来。我还有个想法：我觉得那些真的很喜欢玩即时战略游戏的人，大概会很擅长用 AI。因为你必须——我不知道，Matt Pocock 刚才还在聊「战争迷雾」，聊那些处在前沿的东西：关于这个问题还有我们不知道的部分，我们怎么把它弄清楚？我怎么基于我已经看到的东西，做出当下最好的决定？我看到了几条信息，所以有 30% 的概率是这样，有 40% 的概率是那样。我怎么拿到更多

[1:20:17] Dex Horthy

information? So in my head, I can like recalculate those probabilities and decide what's the mostlikely path that's going to lead us to success. Speaking of the most likely path that leads you tosuccess, let's talk about your company. That's you've, you've just come out of stealth, human layer.What is human layer? And what is the probability that you're setting up for success?

信息，好让我在脑子里重新计算这些概率、判断哪条路最可能通向成功？

说到最可能通向成功的路——我们来聊聊你的公司。你们刚刚从隐身状态出来，HumanLayer。HumanLayer 是什么？你把成功的概率设成了多少？

[1:20:37] Dex Horthy

That's a good question. A hundred percent, a hundred percent probability, uh, maybe 110,but, uh, no. Uh, so human layer is, uh, it's an AI IDE. It's a collaboration platform and it isbuilding blocks for your software factory. And the basic pitch is like engineers solving hardproblems and complex code bases. Basically there's two categories of builders. There's likevibe coders building side projects. And then there's people building production software where thestakes are high. And if something breaks, we're going to get fine millions of dollars or, you know,we're going to lose millions of dollars of money for the company. And there's a whole spectrum inbetween there, but it's like, if you are kind of in the left half of that spectrum, you're buildingsoftware that matters and it has to last and be around for a while, then we are helping peoplelike that solve problems two to three times faster without descending into slop. It was like, how do youmaintain that near human level of quality and move two to three times faster?

好问题。百分之百，百分之百的概率，也许 110%。开玩笑的。

HumanLayer 是一个 AI IDE，是一个协作平台，也是你软件工厂的积木。基本的说法是：面向「在复杂代码库里解难题」的工程师。基本上构建者有两类：一类是造副业项目的 vibe coder；另一类是造生产软件的人，赌注很高——如果东西坏了，我们会被罚几百万美元，或者会让公司损失几百万美元。中间当然有一整条光谱，但如果你处在这条光谱偏左的那一半，你造的是「要紧的、必须活得久」的软件，那我们就是在帮这样的人：把问题解得快两到三倍，同时不掉进垃圾堆里。也就是：你怎么维持接近人类水平的质量，同时快两到三倍？

[1:21:27] Dex Horthy

And what were the ideas that you, you built and that you came with?One idea that we're really excited about right now. I mean, it all comes from thisRPI and this like using specs to like, I mean, I've kind of been hinting at it this whole time,right? Of like, okay, cool. Like start really high level and zoom in layer by layer and re-steer andlike find, find that leverage that helps you move faster and increase the chance that your agent'sgoing to build exactly what you want or something that's really high quality. The other thing I thinkthat's really interesting that where I just posted yesterday, I said, Hey chat, should we kill thepull request? And that's something I can't talk too much about, but basically the idea is likethe IDE of the future needs to be rethought from the ground up for agents. And it might not even be a,like, I don't know, a lot of editors kind of started with the text field and bolted on anagent's tab. And then eventually you've seen like cursor three, I can't even find the text editor.I know it exists. People have told me you can get to a text view of files, but it's also very agentfirst. And so we started from the ground up of like, what is an IDE designed for helping a developer

那你们造了哪些想法？你们带着什么来的？

我们现在特别兴奋的一个想法——它其实全都来自 RPI 和「用规格」这一套。我这一路其实一直在暗示，对吧：好，从非常高层开始，一层层往里放大，一路重新引导，找到那个能让你跑得更快、并且提高「agent 会造出你真正想要的东西 / 造出高质量东西」概率的杠杆。

另一件我觉得非常有意思的事——我昨天刚发了个帖子，说「各位，我们是不是该干掉 pull request？」这个我不能讲太多，但基本想法是：未来的 IDE 需要为 agent 从头重新设计。它甚至可能不是一个……我不知道，很多编辑器都是从一个文本框开始、然后往上焊了一个 agent 标签页。而后来你看到 Cursor 3，我甚至都找不到文本编辑器在哪儿了——我知道它存在，有人告诉我你能进到文件的文本视图，但它也非常 agent 优先。所以我们从零开始想：什么样的 IDE 是为「帮助开发者

[1:22:27] Dex Horthy

interact with and manage the work of agents. And then we zoomed out and said, how do we make thiscollaborative and build in a sync engine and durable streams and all of these like pieces oftech that enable me to get human input and feedback on what I'm doing with agents in real time,rather than waiting for the pull request time. And great engineering teams have been doing this fordecades of like, Hey, we're going to have a design review. We're going to talk about how we're goingto build the thing as like a two page Google doc or whatever, 10 page, whatever, however.PRD, ERD, UNGRAD. Yeah. AR architecture requirements document. And then you go tosprint planning and you break it down into little tickets and you decide who's going to do what.It's like AI can help with all of this. You should, if you're just using AI to write thecode, you're missing out on a lot of the benefits that AI can bring to your SDLC. And a lot of peoplesay like, well, we don't need any of those meetings anymore because we have the loop, we have thedark factory things just fly around the loop, but it's like, okay, but if you want to actually movefaster and maintain quality, then like you should have these checkpoints before you go to actually

与 agent 互动、管理 agent 的工作」而设计的？然后我们再往外拉一层，问：我们怎么让它变成协作式的？怎么内建一个同步引擎、持久化流（durable streams）以及所有这些技术组件，好让我在用 agent 干活时能实时拿到人的输入和反馈，而不是等到开 PR 的那一刻？

而优秀的工程团队几十年来一直在做这件事：嘿，我们来做一次设计评审，我们用一份两页的 Google 文档、或者十页、随便多长，来讨论我们要怎么造这个东西。

PRD、ERD……

对，架构需求文档（ARD）。然后你去做迭代规划，把它拆成一张张小工单，决定谁做什么。这一切 AI 都能帮忙。如果你只是用 AI 来写代码，你就错过了 AI 能给你整个软件开发生命周期（SDLC）带来的很多好处。很多人会说：我们不再需要那些会了，因为我们有循环、我们有黑灯工厂，东西就自己在循环里飞。但问题是：如果你真的想跑得更快同时保住质量，那你就该在真正开始

[1:23:21] Dex Horthy

write the code and you should use AI to help with that. So we built this like cloud platform.That's kind of has like a Google doc style component where you can comment and the agent can surfacelike mock-ups and mermaid diagrams and HTML and all these things. So basically how do we make agentslike big must style? Every, everything's in the cloud. Everything's collaborative. I see all mycoworker sessions. They see all of mine. It's almost like the benefit that Slack had over emailwas that you didn't have to be in every conversation to know what was happening.You could maintain, you could see all these channels light up. You could check on them. Okay. I don'tcare about any of that. But if you saw a conversation that you cared about, you could jump in on that.And it's like, how do we do that for engineering work versus like, we really had these like verystrict, even when we called it agile, it was very waterfally, like PRD, ARD, tickets. Everyone goes andbuilds for a day. And then you get the PR back and then one person reviews it. How do you createthis more just like soup? And like, what is the data model for that world where you have likeagentic traces, you have documents, you have tasks and projects that group these things. You have

写代码之前有这些检查点，而且你该用 AI 来帮你做这件事。

所以我们造了这么一个云平台，它有点像 Google 文档那样的组件：你可以评论，agent 可以在里面呈现原型图（mock-up）、mermaid 图、HTML 这些东西。所以本质上是：我们怎么让 agent 变成多人协作（Figma 那种风格）？一切都在云上，一切都是协作的。我能看到所有同事的会话，他们也能看到我的。这有点像 Slack 相对邮件的好处：你不必在每一场对话里，也能知道正在发生什么。你能维持一种感知，你看到这些频道亮起来，你可以去瞄一眼——好，这些我都不关心；但如果你看到一场你在意的对话，你可以跳进去。

那么，我们怎么给工程工作也做出这个？现在我们的流程非常严格——就算我们管它叫敏捷，它其实很瀑布：PRD、ARD、工单，每个人去造一天，然后你拿到 PR，然后一个人评审它。你怎么把它变成更像一锅「汤」的东西？而且那个世界的数据模型是什么？你有 agent 的轨迹（trace），你有文档，你有把这些东西归组的任务和项目，你有

[1:24:25] Gergely Orosz

actual Git diffs being streamed everywhere where it's like, why would I review all the code at oncewhen I can just always, every, everybody's work lives in a shared environment that anyone can gointeract with. I mean, what it reminds me is like what, you know, GitHub did the software team beforeGitHub and its competitors, you might have a tracker somewhere. But most teams were just kind of likeinside the company, you didn't know what one one team was, I remember pre GitHub, like, you know,they had individual teams, they, some of them had like a board with stickers, but no one else in thecompany knew what they were doing. They were all working in isolation. And now when you have GitHub, or eventhe internal version of GitHub inside a company, you can always see when you go to a team, you seethe pull request flying, you can join in, you have history, it's all, it is all kind of connected.And it came together. And now it's like, you know, for a very long time, I was like, you'regoing to use GitHub or, or people will copy it. So do I sense that you're trying to build somethinglike this, this workflow for like, when you have the software factories, which are like dark factoriesand loops at a bunch of places? How can we have this, this new way of working, which which will

真实的 Git diff 在到处流动。为什么我要一次性评审所有代码？我完全可以随时看——每个人的工作都活在一个共享环境里，任何人都能进去互动。

这让我想到 GitHub 对软件团队做的事。在 GitHub 和它的竞争者之前，你也许在某处有个跟踪系统，但大多数团队在公司内部就是——我记得 GitHub 之前，各个团队是独立的，有些团队有一块贴便利贴的板，但公司里没有别人知道他们在干什么，他们都在孤岛里工作。而现在有了 GitHub，或者公司内部那套 GitHub，你随时可以走到一个团队那儿，看到 PR 在飞、你可以加入、有历史记录，一切都是连通的、聚在一起的。

所以很长一段时间我都觉得「你要么用 GitHub、要么有人会抄它」。那我是不是感觉到，你在试着造这样一套工作流——当你有了软件工厂、有黑灯工厂、到处都是循环的时候，我们怎么有一种新的工作方式，它用起来会很自然，但要想出来

[1:25:32] Gergely Orosz

feel natural, but like, coming up with it, like is hard work. It's counterintuitive.How can we do something that accomplishes what GitHub did, but like 10x better, like more canspecifically like more continuous and more real time and more collaborative than like,these discrete units of work that is like the pull request?

很难、很反直觉。我们怎么做出一个能实现 GitHub 当年成就、但好上十倍的东西——具体来说是更连续、更实时、更协作，而不是像 pull request 那样一个个离散的工作单元？

[1:25:50] Gergely Orosz

Well, I know I'm starting to understand why you're saying maybe we should kill the pull requestbecause pull request was invented by GitHub, right? Like it's, it is not part of Git, butthey do it as a way for you to do a code review merge before it goes in and be able to modifyit or like just reject it or et cetera.And it's probably a lot better than whatever we had before, which I guess was like emailingyour Git patch to Linus and ask him to merge it into the kernel or whatever.They still do it. It works for them. That's the point, but it only works for them.Yeah. I don't know anybody else who does that. I mean, I'm sure even before GitHub for you,you guys had what, like CVS or CVS, TFS. So if you have a lot of money for Microsoft,they made us use subversion at an undergrad because the guy who invented subversion, uh,was a U Chicago guy. The year after I graduated, they switched everybody to Git and I was like,damn, I learned a useless thing just for somebody's ego.Specifically for AI startups or startups like building on top of AI or building AI products.How important do you think location and network is, especially you are based in, in the,the Valley, we see research that AI startups are more frequently funded from here than normal

嗯，我开始明白你为什么说「也许我们该干掉 pull request」了——因为 pull request 是 GitHub 发明的，对吧？它不是 Git 的一部分，他们把它做成一种方式：在合入之前让你做代码评审，并且能改它、或者干脆拒绝它，等等。

而且它大概比我们之前那套好多了——之前我猜是把你的 Git 补丁用邮件发给 Linus，请他合进内核什么的。

他们现在还这么干，管用。

这就是重点：但它只对他们管用。

对，我不认识别的谁这么干。我是说，我猜就算在 GitHub 之前，你们用的是 CVS，或者 TFS——如果你有很多钱给微软的话。

我本科时学校逼我们用 Subversion，因为发明 Subversion 的那个人是芝加哥大学的。我毕业后第二年，他们把所有人换成了 Git，我当时就想：靠，我为了某人的自尊心学了个没用的东西。

具体到 AI 创业公司、或者在 AI 之上构建、做 AI 产品的创业公司——你觉得地点和人脉网络有多重要？尤其你是在硅谷。我们看到研究显示，AI 创业公司比一般创业公司更频繁地在这里

[1:26:54] Dex Horthy

startups as well. Do you see this advantage? And also do you see some disadvantages of beinga specific, may that be Silicon Valley or elsewhere?I don't have really strong opinions on this. Actually, like Paul Graham gave a talk in Swedenabout why SF is cool. Rather than just regurgitate that, I will, I will forward people onto that one.Um, we can put it in the show notes or whatever, but he talks about all of the dead dynamics ofSilicon Valley and the paid forward culture and the like people take you way more seriously justbecause you're based here. I lived in Chicago for a long time. I have a lot of really good friendsfrom high school, from college, from going up in LA and never before have I felt like so locked inwith like my people have more, never have I felt more seen, more connected. Like there's just somany people here again, talking about the founder thing, people who care deeply, who are incrediblycompetent, who like, we all have all the same types of problems. We love all the same types of things.Like I don't do LAN parties where we play video games, but all my buddies will come over and we'llsit in the office till 11. We'll just do coworking and like hack on cool fun, fun projects and stuff.

拿到融资。你看到这个优势吗？你也看到「待在某个特定地方」——不管是硅谷还是别处——的劣势吗？

我在这件事上没有特别强的看法。其实 Paul Graham 在瑞典做过一场演讲，讲为什么旧金山很酷。我就不复述了，我把大家引到那场演讲去，我们可以放在 show notes 里。他讲了硅谷所有的动力学、「先付出」（paid forward）的文化，以及「仅仅因为你在这里，人们就会更认真地对待你」这件事。

我在芝加哥住了很久，我有很多非常好的朋友——高中的、大学的、在洛杉矶长大的。但我从来没有像现在这样，感觉跟「我的人」如此契合，从来没有感觉如此被看见、如此有连接。这里就是有那么多人——还是那个创始人的话题——他们深深地在意，能力极强，我们都有同一类问题，我们都爱同一类东西。我不搞那种打游戏的局域网派对，但我所有的哥们儿会过来，我们在办公室坐到十一点，就一起办公、一起在很酷很好玩的项目上黑一黑。

[1:27:52] Dex Horthy

And like, you can't do that anywhere else. There's not enough like a critical mass for that to justhappen organically everywhere you go. And, and I absolutely love it. I wouldn't trade it for anything.Yeah. It's been a critical masses. It nails it on, on, on the head. When it comes to hiring, what types offolks are you hiring for specifically? Cause I'm interested in how hiring changes and what,what a standout engineer means for you and how you are trying to, you know, confirm that those traitsexist. In general, we, we are looking, um, for people who are, have really strong softwarefundamentals. So understand distributed systems, understand like the core fundamentals of CS andoperating systems and these kinds of things. I mean, you don't have to be a PhD in fricking kerneldesign or whatever, but it's a lot easier. We can, we can teach, we can teach somebody,I think to be a really good AI developer in a few months, you can build enough intuition where youare, you know, accelerated off the ground and you can go like keep growing there. It's really hardto teach someone a CS undergrad program and in three months.And what's the problem space that you're excited about and in software engineering or even product

这在别的任何地方都做不到。别的地方没有足够的临界质量（critical mass），让这种事在你走到哪儿都能自然发生。我完全爱这一点，我不会拿它换任何东西。

对，「临界质量」这个词说到点子上了。那说到招聘，你们具体在招什么样的人？因为我很好奇招聘是怎么变的、对你来说「出众的工程师」意味着什么、你又怎么去确认这些特质存在。

总体上，我们找的是软件基本功非常扎实的人：懂分布式系统，懂计算机科学和操作系统这些核心基本功。我是说，你不用是内核设计的博士什么的，但那样会容易很多。我觉得我们可以在几个月内教会一个人成为很好的 AI 开发者，你能建立起足够的直觉，让你脱离地面、加速起飞，然后一直成长下去。但你很难在三个月内把一个 CS 本科的全部内容教给一个人。

那在软件工程、甚至产品

[1:28:56] Dex Horthy

engineering or building products that you think in the next few years, it's going to be one of theinteresting things that you're going to be attacking. My co-founder could talk more aboutthis, but like, there's a lot of interesting things happening in, in real time and cloud andsandboxes and sync and kind of like using these new building blocks that have gotten really solid inthe last couple of years. We're big fans of the electric SQL team. We're users of durable streams.It was like, how can you build systems that kind of are a lot more spread out and distributed andalmost like decentralized. This is really interesting for coding because you want to be able to runcoding agents anywhere. You want to be able to run them for a short time, for a long time,on demand, on a schedule, all these things and have them all be part of this kind of likebrain. So I don't know, parts of what we're doing are really boring, like all our data is in Postgres.And then parts of what we're doing is really interesting. But there's a lot of distributedsystems problems. There's a lot of infrastructure problems. Like we are building tools for AI,but there's a lot of problems in building collaboration platforms that are really, really hard.

工程或者做产品这一块，未来几年你觉得哪个问题空间最让你兴奋、是你要去攻的有意思的东西之一？

这个我的联合创始人能讲得更多。但实时、云、沙盒、同步这些方向正在发生很多有意思的事，用的是这些过去几年才真正变扎实的新积木。我们是 Electric SQL 团队的粉丝，我们是 durable streams（持久化流）的用户。就是：你怎么造出那种更铺开、更分布式、几乎去中心化的系统？这对编码来说特别有意思，因为你希望能在任何地方跑编码 agent，你希望能跑很短时间、也能跑很长时间，能按需跑、也能按计划跑，而且让它们都属于这样一个「大脑」的一部分。

所以我不知道，我们做的事有些部分非常无聊，比如我们所有数据都在 Postgres 里；有些部分则非常有意思。但这里面有很多分布式系统问题、很多基础设施问题。我们是在为 AI 造工具，但「造协作平台」本身就有很多非常非常难的问题。

[1:29:54] Gergely Orosz

And there's a lot of new tech that makes it easier and more interesting, but it's stillby far from an easy problem.It sounds like what you're saying is like the infralayers, it's some extent a new infralayersbeing built and it'll take some time and, but it'll be like new, new blocks and it willeventually become the primitives, like for cloud we have from is already, but it tookfrigging decades to get those together or more.Yeah. You had AWS in what, like 2006, 2006. Yeah. And then you got Kubernetes a decade later.Yep. And as closing, what's a book or reading that you would recommend?

而且有很多新技术让它更容易、更有意思，但它仍然远不是一个容易的问题。

听起来你说的是：基础设施层，某种程度上是一层新的基础设施正在被造出来，它需要一些时间，但它会成为新的积木，最终变成原语（primitives）——就像云那样，现在已经是了，但把那些东西凑齐花了几十年甚至更久。

对。AWS 是哪年来着，2006 年？

2006 年，对。然后十年后才有了 Kubernetes。

对。

最后，你会推荐哪本书或者哪份读物？

[1:30:26] Dex Horthy

Something that you personally enjoyed.Nowadays, we talk a lot about refactoring by Martin Fowler. Classic. I think it's becausewe spent a lot of time improving the design of existing code and trying to figure out how to getmodels to build code that is easy to maintain and like easy to read and easy to understand and easyto build on. I feel like I probably have a better answer than that, but that's, that's what's top ofmind these days. We're reading a lot of like classics of software engineering, refactoring,clean code, the pragmatic programmer, like all that stuff is, I think is more relevant now than ithas ever been. Love it. Well, Dex, thanks so much. This was fun.This was a blast, dude. Thanks for, thanks for having me on. This was great. I had a lot of fun.I don't know about you, but I really enjoyed this conversation. Dex is such a big believer ingender coding, yet he's the one warning us that if you stop reading the code, you have about three tosix months before your code base becomes easier to rewrite than to fix. And this comes from Furt hasexperience. His team built a light soft software factory, ran it, and then had to shut it down.I also like the idea of the slow loop. Loop engineering feels like a somewhat meaningless

你自己特别喜欢的。

最近我们经常聊 Martin Fowler 的《重构》（Refactoring）。经典。我想那是因为我们花了大量时间在「改进现有代码的设计」上，在琢磨怎么让模型造出易维护、易读、易懂、易在其上继续构建的代码。我感觉我大概能给出比这更好的答案，但这就是最近最常出现在我脑子里的。我们在读很多软件工程的经典：《重构》《代码整洁之道》（Clean Code）《程序员修炼之道》（The Pragmatic Programmer），我觉得这些东西现在比以往任何时候都更相关。

太好了。Dex，非常感谢，这很有意思。

太爽了，哥们。谢谢你请我来，这很棒，我玩得很开心。

我不知道你们怎么想，我是真的很享受这场对话。Dex 是「agent 编码」非常坚定的信徒，然而正是他在警告我们：如果你停止读代码，你大概有三到六个月的时间，之后你的代码库就会变成「重写比修复更容易」。而这来自第一手的经验——他的团队造了一座关灯软件工厂，跑了它，然后不得不把它关掉。

我也喜欢「慢循环」这个想法。「循环工程」这个词对我来说有点

[1:31:28] Gergely Orosz

term to me. What Dex's team does is actually pretty boring. A cron job runs every night,fixes one issue or one anti-pattern, and opens one small pull request. The team wakes up to a code basethat's a little bit better every morning, and devs still need to review and approve it. This is apractice that, honestly, any engineering team could just adopt today. Finally, I really enjoyed thehistory lesson. The term software factory comes from a NATO conference in 1968. The idea of softwareused to build software with analogies to a factory is more than six years old, and every generation ofour industry has tried to automate more of the loop of building software. AI agents are just yet one moreattempt, although probably the most successful one. Do check out show notes below for the relatedThe Pragmatic Engineering Deep Dives that go even deeper into AI engineering and other related topics.If you've enjoyed this podcast, please do subscribe on your favorite podcast platform and on YouTube.A special thank you if you also leave a rating on the show. Thanks, and see you in the next one.

空洞。Dex 团队实际做的事其实相当无聊：一个定时任务每晚跑一次，修一个问题或者一个反模式，开一个小的 pull request。团队每天早上醒来，代码库都好了那么一点点，而且开发者仍然需要评审和批准。老实说，这是任何一个工程团队今天就可以采用的做法。

最后，我很喜欢那节历史课。「软件工厂」这个词来自 1968 年的一场北约会议。「用软件来造软件、并拿工厂来打比方」这个想法已经存在六十多年了，而我们这个行业的每一代人，都试图把「造软件」这个循环自动化得更多一点。AI agent 只是又一次尝试，尽管大概是最成功的一次。

请查看下方的 show notes，那里有相关的 The Pragmatic Engineer 深度文章，会更深入地讲 AI 工程和其他相关话题。如果你喜欢这档播客，请在你常用的播客平台和 YouTube 上订阅。如果你还能给节目留个评分，那就特别感谢了。谢谢，我们下期见。