How Software Factories Improve Themselves — Suraj Gupta, Warp

频道: AI Engineer
视频: https://www.youtube.com/watch?v=TN3mj92oZ8I
原文语言: en
统计: 共 10 轮 · Suraj 9

[0:12] Suraj

Um my name is Siraj Gupta. I lead Harness development at Warp. Um in case you haven't heard of Warp before, um we started off by building a state-of-the-art agentic development environment. Um we have about a million active users that use that. Um it was born out of a modern-day terminal. Um and nowadays we're building a um cloud agent platform to enable teams to build software factories. Um today I'm going to talk about um self-improving software factories, which I know might sound kind of buzzy, um but I'm going to break it down and hopefully you can walk out of here with something concrete that you can implement yourselves. Um you've probably heard of both of these terms, self-improvement and software factories, used throughout this conference, but I'm sure you've heard of them independently. Um you've heard about how self-improvement is about agents being able to um improve over time and agents and models um and, you know, phasing out humans from that loop, so that way the the improvement loop can be automatic. Um and I'm sure you've heard about software factories being um the way that like the software development paradigm is shifting from individual coding agents to building out automations that take work from triage up until production. Um but less has been said about how software factories themselves can self-improve over time, so that the the factory can get better, more efficient, faster. Um and I think this is going to become an extremely important part of our jobs as software engineers as we transition from building products to building factories and and maintaining them.

我叫 Suraj Gupta，在 Warp 负责 harness（包在模型外面、管提示词/工具调用/循环的那层程序）的开发。可能有人之前没听说过 Warp：我们一开始做的是一个最先进的 agentic 开发环境，现在大约有一百万活跃用户在用，它脱胎于一个现代化的终端。如今我们在做一个云端 agent 平台，帮团队搭建软件工厂。今天我要讲的是自我改进的软件工厂——我知道这听起来可能有点像在追热词，但我会把它拆开来讲，希望大家离开的时候能带走一些具体的、自己回去就能落地的东西。自我改进和软件工厂这两个词，大家在这届大会上大概一直在听，但我猜都是分开听到的。你们听过，自我改进讲的是 agent 和模型能随着时间不断变好，同时逐步把人从这个循环里撤出来，好让改进循环自动运转起来。你们肯定也听过，软件工厂讲的是软件开发范式正在转变：从一个个单独的编码 agent，转向搭建一整套自动化流程，把工作从分诊（问题进来先判断归谁、怎么处理）一路推进到生产环境。但很少有人讲，软件工厂本身怎么随时间自我改进，让工厂变得更好、更高效、更快。我认为，随着我们从做产品转向建工厂、维护工厂，这会成为我们软件工程师工作中极其重要的一部分。

[1:46] Suraj

Um so let's dive in. Um self-improvement is a pretty broad umbrella, but I'm going to focus on three concrete ways that agents can improve over time, agent factories can improve over time. I'm going to start with um skills, then we'll talk about persistent memory, and then I'll finish off by talking about model routing. Um so skills are a great way to give your agents procedural memory. I don't think I have to kind of spell that out for you. You describe a a procedure to your agent that you expect it to have to do a task over and over. Um for example, let's say you're building a triage agent, which is a pretty common thing you might build in a software factory. Um and you give it a skill on how you want it to go about reproducing issues. Um those skills get stale over time. Um humans give feedback to the agents. Agents learn things in their own um runs and trajectories. Um and the like the skill that you previously gave it becomes dated. So, how do we fix this and let the agent improve over time using that skill? Well, one way to do it is to deploy an outer loop agent. Um and this is what we've done in our own internal software factory that's been really productive. The idea is that your um your inner loop agent is the thing that's actually applying the skill. It's like doing the triage. And then you have this other outer loop agent that is observing your inner loop agent's runs and improving its skill over time by looking for um for mistakes it made or looking at feedback that the human has given it.

那我们开始吧。自我改进是个挺宽泛的大概念，但我会聚焦在三种具体方式上——agent 能随时间变好的方式，或者说 agent 工厂能随时间变好的方式。我先讲 skill，然后讲持久记忆，最后讲模型路由。skill 是给 agent 注入程序性记忆的好办法，这点应该不用我多解释：你把一套流程描述给 agent，对应的是你预计它要一遍又一遍去做的某个任务。举个例子，假设你在做一个分诊 agent——这是软件工厂里很常见的一类东西——你给它一个 skill，告诉它你希望它怎么去复现 issue。这些 skill 会随时间变得陈旧：人会给 agent 反馈，agent 也会在自己的一次次运行和轨迹里学到东西，于是你之前给它的那个 skill 就过时了。那怎么解决这个问题，让 agent 在用这个 skill 的过程中不断改进？一种做法是部署一个外层循环 agent。这正是我们在自己内部的软件工厂里做的，效果一直很好。思路是：内层循环 agent 负责真正应用这个 skill，也就是实际去做分诊；然后另有一个外层循环 agent，观察内层循环 agent 的运行，通过找出它犯过的错、查看人给它的反馈，随时间不断改进它的 skill。

[3:14] Suraj

Um so, I'm going to show an example of that. Um let me swap here. Cool. Um so, this is actually a skill that we've used internally at Warp. We've open-sourced our client repository and it's fully run as as a factory. And so, this is an example of an of a of a skill that's that's backing a triage agent that we have today. The idea is that um you you get an issue that comes in on GitHub, and you want to automatically run an agent to figure out like what sorts of um like it what sort of information did the user not provide that is going to help us figure out if this is something that we should build, an existing issue that we might actually just want to dedupe against or something else. Um and then we have a Sorry, let me just switch over here. And then we have a workflow that actually like runs that that triage um agent. Um this is a pretty simple workflow. It just Every time an an an issue gets submitted on the repo, this um this workflow runs. And then here's the actual outer loop agent, which is about looking at the the work that the triage agent has done and improving it over time. So, the idea is that the agent looks at the decisions that the triage the inner loop triage agent made, and it um collects feedback signals, which might be uh you know, thumbs up or thumbs down on the issues. Um it might be users commenting on the issue. It might be someone from Warp um an employee of Warp commenting on the issue, uh giving the agent feedback on what it did.

下面我来演示一个例子。我切一下屏幕。好。这其实是我们在 Warp 内部用过的一个 skill。我们把客户端代码仓库开源了，而且它完全是按工厂的方式在运转。所以这就是一个例子：我们现在有一个分诊 agent，背后支撑它的就是这个 skill。思路是：GitHub 上进来一个 issue，你希望自动跑一个 agent 去弄清楚，用户有哪些信息没提供，而这些信息能帮我们判断——这是我们应该去做的东西，还是某个已有的 issue、其实直接去重就行，又或者是别的情况。然后我们还有一个——抱歉，我切过去一下。然后我们有一个 workflow，专门负责跑这个分诊 agent。这个 workflow 很简单，就是每当仓库里提交了一个 issue，它就会跑一次。然后这里是真正的外层循环 agent，它负责审视分诊 agent 做过的工作，并随时间不断改进它。思路是：这个 agent 会查看分诊 agent——也就是内层循环的分诊 agent——做出的各种决策，并收集反馈信号，比如 issue 上的点赞或点踩，也可能是用户在 issue 下面的评论，还可能是 Warp 的人、某位 Warp 员工在 issue 下评论，给 agent 反馈它做得怎么样。

[4:42] Suraj

Um and then ultimately that leads to like some synthesis of of the of that um iteration. And so, we we end up basically updating our inner loop skill with this outer loop agents. Um and that's what leads to this self-improvement loop with skills. What's really nice about this is that um the So, in step four, you'll see we we added the triage skill, and then we open a pull request. So, that means that um all of the improvements to the inner loop skill are going to be tracked through get. So, you get like full observability into like how that skill is transformed over time. And that it also means that um a human is actually going to review those updates to the skill. So, that way, you know, this this outer loop agent doesn't make a mistake and ultimately cause your triage agent to to actually perform worse. Um yeah, so that's that's kind of what what I want to talk about for skills. Um let me talk about persistent memory now. So, I'm going to go back to my deck. Um cool. So, skills are great for remembering procedures, but what about everything else? So, like what what happens when your Sentry agent um looks for an issue and and finds an issue, does some work to like gather context about it, and fixes it, but then now you have you encounter a similar issue in the future.

最终这些会汇总成对这一轮迭代的提炼总结。所以说到底，我们就是用这个外层循环 agent 去更新内层循环的 skill，基于 skill 的自我改进循环就是这么来的。这套做法特别好的一点是——你看第四步，我们添加了分诊 skill，然后开一个 PR。这意味着对内层循环 skill 的所有改进都会通过 Git 留下记录，这个 skill 随时间是怎么演变的，你拥有完整的可观测性。同时这也意味着，真的会有人来评审这些对 skill 的更新。这样一来，就不会出现外层循环 agent 犯了错、最后反而让你的分诊 agent 表现变差的情况。好，关于 skill 我想讲的大概就这些。接下来讲持久记忆。我先切回幻灯片。好。skill 很适合用来记住流程，那其他的东西呢？比如说，你的 Sentry agent（Sentry 是一个错误监控服务）去查 issue，找到了一个，做了些工作收集相关上下文，然后把它修好了——可之后你又遇到了一个类似的 issue，会怎么样？

[6:00] Suraj

Your century your agent might get lucky and like be able to figure out the same root cause that it did the first time around, but that's not guaranteed. And and even if it does, you're you've probably wasted a lot of tokens re-gathering context on an issue that you've already fixed before. So, that's where something like persistent memory comes in. You can think of this as a fact store scoped to an agent. Your agent runs and similar to the skill improvement loop, you have an outer loop agent that's going to extract facts, learnings, outcomes from your from your inner loop agent. So, that way future runs of the agent can lean on what's already been done in the past. So, I'm going to show an example of this. Come back to the deck here. So, in warp or in Oz rather, like I mentioned, we built out a a cloud agent platform where you can run agents in the cloud. And for all for all these agents, you can attach memory stores to them. Again, a memory a memory store is a collection of facts. In this case, I have a century agent that has collected some memories over time about root cause analysis that it's done in the past. What's cool is like you can actually version this memory, you can update it as a human, create new memories or delete them yourselves, but primarily this this memory creation loop is driven by agents. You can also see where these memories were sourced from. So, I can actually like open up the run here to see Oh, well.

你的 Sentry……你的 agent 也许运气好，能找出和第一次一样的根因，但这没法保证。而且就算找出来了，你大概也已经浪费了大量 token，在一个以前修过的 issue 上重新收集上下文。这就是持久记忆这类东西派上用场的地方。你可以把它理解成一个限定在单个 agent 范围内的事实库。你的 agent 在运行，和 skill 的改进循环类似，你会有一个外层循环 agent，负责从内层循环 agent 那里提取事实、经验和结果。这样 agent 之后再跑的时候，就能借助过去已经做过的东西。我来演示一个例子，先回到幻灯片这儿。在 Warp——准确说是在 Oz 里——就像我前面提到的，我们搭了一个云端 agent 平台，你可以在云上运行 agent。所有这些 agent 都可以挂载记忆库。再强调一下，记忆库就是一组事实的集合。这个例子里，我有一个 Sentry agent，它随着时间积累了一些记忆，都是关于它过去做过的根因分析。很酷的一点是，这些记忆可以做版本管理；你作为人也可以更新它，自己新建记忆或者删除记忆；但这个生成记忆的循环主要是由 agent 来驱动的。你还能看到每条记忆是从哪儿来的。所以我其实可以在这里打开那次运行，看看——哦，好吧。

[7:22] Suraj

I think this my IP address here keeps changing and this my our staging platform is gated to an IP address, but anyways, you should like you can see where those memories were sourced from. So, that way you can be like, oh, like maybe that wasn't actually an important memory or that was kind of like a local maxima that I don't want to include in my memory store. Um and then when your agents continue to run, so in this case I have the sentry agent that's um that's done a bunch of runs after it's created those memories, it actually can lean on those memories in the future um when it's making um when it's triaging new issues. So in this case, you'll notice that um this agent ran, it did a bunch of work to do some root cause analysis, and then it ultimately reported back on which memories it used um uh in this case it it it it was able to use those um five memories that I had in that memory sort to to speed up the work that I did this time around. Um, cool. Uh let me go back to the deck. So, what's cool in Oz is that persistent memory works across all harnesses, so Warp Zone, proprietary harness, plot code, Kodak, it doesn't really matter what you're running on Oz, um memories will automatically be created for you.

我这边的 IP 地址好像一直在变，而我们的预发（staging）平台是限定 IP 地址才能访问的。不过总之，你是能看到这些记忆是从哪儿来的。这样你就可以判断：哦，这条其实算不上重要的记忆，或者那条有点像是局部最优，我不想把它放进我的记忆库。然后，当你的 agent 继续运行时——这个例子里，我的 Sentry agent 在生成这些记忆之后又跑了很多次——它之后在分诊新 issue 的时候，就真的能借助这些记忆。你会注意到，这个 agent 跑了一次，做了一堆根因分析的工作，最后汇报了它用到了哪些记忆。这次它用上了我那个记忆库里的五条记忆，加快了这一轮的工作。好，我回到幻灯片。Oz 很酷的一点是，持久记忆在所有 harness 上都能用：Warp 自己的专有 harness、Claude Code、Codex——你在 Oz 上跑的是什么其实都无所谓，记忆都会自动帮你生成。

[8:32] Suraj

Um, but you also have the control to to review those memories, version them, delete them, or edit them as you please. Um, all that being fully traceable. Um, cool. That's what I wanted to say about memory. Um, last thing I want to talk about here is model routing. So, um you may be wondering how this plays into self-improvement loops for um factories. Well, first I want to touch on why model routing is important in a factory in the first place. Um, it can become prohibitively expensive to run all of your agents that are doing simple things like triage or fixing simple CI failures with Opus. That's just like going to be really expensive over time, and I'm sure some of you have maybe been burnt by this or have heard stories of of or organizations being burnt by this. Um, so model like choosing which models you use for which tasks that is actually very important um and something that we're we've been working on at work. Um, so in Warp, you can actually use uh our out-of-the-box model routers, we call them our auto models. Um, over like as new models come out, we're all we're always evaluating like what are the best models when it comes to Pareto efficiency. And so, like if you're a new user and you just want to get started, it's much better than like pinning your model to like Opus or or or Haiku, for example. Um let me show you all um that experience in Warp.

但你也有完全的控制权，可以随意评审这些记忆、给它们做版本管理、删除或者编辑，而且这一切全程可追溯。好，关于记忆我想说的就这些。最后我想讲的是模型路由。你可能会好奇，这跟工厂的自我改进循环有什么关系。首先我想说说，为什么模型路由在工厂里本来就很重要。如果你所有的 agent，哪怕干的只是分诊、修简单的 CI 失败这种简单活儿，都用 Opus 来跑，成本可能会高到让人吃不消。时间一长，那会非常贵。我相信在座有些人可能已经在这上面吃过亏，或者听说过有的组织在这上面栽了跟头。所以，决定哪个任务用哪个模型，其实非常重要，这也是我们在 Warp 一直在做的事。在 Warp 里，你可以直接用我们开箱即用的模型路由器，我们管它们叫 auto 模型（自动选模型）。每当有新模型出来，我们都会持续评估：从帕累托效率的角度看，哪些模型是最好的。所以如果你是新用户，只想先上手用起来，这比把模型固定死成 Opus 或者 Haiku 之类的要好得多。下面我给大家演示一下 Warp 里的这个体验。

[9:47] Suraj

So, let me go here. Give me 1 sec. Got to drag this over here. So, here I have my Warp desktop um and you can see that um alongside being able to define models um being able to use Warp's auto models is one option, like these models here, but you can also define your own model routed um uh models, which uh basically boil down to like a set of rules that you can give um to us and we will like make decisions about model routing based on those rules. So, here I have an example of a configuration file that I've defined um where I've defined a few different rules about like I want database migrations done with with GLM fire with GLM or I want um you know, uh runbooks and API documentation done with Quen. Um So, this really allows you to like like to define classes of tasks and be able to say, "Okay, I want this this class of task done by this model because that's kind of what you've seen." At this point though, that's kind of more of an art than a science um and so uh what we're working on next is being able to define evals for um customer-facing evals where you can actually define what sorts of things you care about and what knobs you want to turn on these configurations and actually see, "Okay, is is Haiku better than fi- than GLM than Opus in a specific class of issues?" That's like specific to your workflows and not just some like generic benchmark that um that someone posted online.

我切到这边，稍等一秒，得把这个拖过来。这是我的 Warp 桌面端。你可以看到，在可以指定模型的地方，用 Warp 的 auto 模型是其中一个选项，就是这里这几个模型；但你也可以定义你自己的路由模型，说白了就是一组规则，你把规则交给我们，我们就根据这些规则来做模型路由的决策。这是我自己定义的一个配置文件示例，里面写了几条不同的规则，比如我希望数据库迁移用 GLM 来做，或者 runbook（运维手册）和 API 文档用 Qwen 来做。这样你就真的可以把任务划分成不同类别，然后说「好，这一类任务我要交给这个模型做」，因为你实际看下来就是这样。不过就目前来说，这更像是一门艺术，而不是科学。所以我们接下来在做的，是支持定义面向客户的 eval（评测）：你可以定义自己在意的是哪些东西、想在这些配置上调哪些旋钮，然后真正看到「好，在某一类具体的问题上，到底是 Haiku 更好，还是 GLM，还是 Opus？」——这是针对你自己的工作流的，而不只是网上某个人贴出来的某个通用 benchmark。

[11:19] Suraj

Um let me go back here. Give me a sec. Cool. Cool. Um and yeah, so let me uh I'm in my slideshow. So, here is kind of like a a high-level overview of how this works in Warp today. So, um internally we have our model routing rules, um but then we have like an eval sidecar that's determining um what sorts of tasks are actually better with which models. And we do this with a simple like best-at-k um approach where when you type a prompt, we run a bunch of agents in Oz uh across a different set of models. Um and we found success in like being able to determine, "Oh, like UI tasks are really well done with GLM. Um we don't really need to run those with Opus. It's much more efficient to run those with GLM." Um so, that's how we've been using it, but um sooner enough we're going to be building this into our product so that way our our customers can actually use it in their own workflows. Um Yeah, that's uh that's mostly what I had today. Um we're around in a booth around that side. So, if you're interested in this kind of stuff, come talk to me. I'm always down to talk about this kind of stuff. But yeah, thanks for your time today. That's all I had.

我切回来，稍等。好，好。嗯，我已经回到幻灯片了。这是 Warp 目前这套机制怎么运作的一个整体概览。我们内部有自己的模型路由规则，同时还有一个 eval 旁路（sidecar），专门判断哪类任务用哪个模型效果更好。我们用的是一个简单的 best-of-k（同一任务并行跑 k 个、取最好的）方法：你输入一个 prompt，我们就在 Oz 上用一组不同的模型跑一批 agent。靠这个办法，我们确实得出了一些结论，比如：「哦，UI 类任务用 GLM 做得非常好，其实没必要用 Opus 跑，用 GLM 跑效率高得多。」这就是我们目前的用法，不过很快我们会把它做进产品里，这样客户也能在自己的工作流里用上。好，今天我要讲的大概就这些。我们的展台在那边，如果你对这类东西感兴趣，欢迎来找我聊，这类话题我随时都乐意聊。好，感谢大家今天抽出时间，我就讲到这里。

[12:49]

[music]

（音乐）