Agentic SDLC at Uber — Uday Kiran Medisetty & Adam Huda, Uber
频道: AI Engineer
视频: https://www.youtube.com/watch?v=17-YSUHo6Lk
原文语言: en
统计: 共 22 轮 · Uday 12 · Adam 9
[0:12] Uday
Hey uh let's get started. Uh good morning everyone. I'm Udai. I'm here with my colleague Adam. We'll talk about our journey towards manage software factory. And in the beginning in the first part of the talk I'll talk about the key building blocks that we are investing in. And later Adam's going to talk about how we take all of these blocks to build an end to-end cohesive solution for our engineers. To set some context, we have few thousand engineers across 12 global tech sites. Over the last year, all of the investments we made in agentic AI have led to more than 70% of our PRs now either by local or cloud agents. And all of this led to twice the number of lines of code per engineer year-over-year. And this extends way beyond coding and we see it in every aspect of the engineering life cycle today.
嘿，我们开始吧。大家早上好。我是 Uday，和我一起的是我的同事 Adam。今天我们要讲的是我们走向「受管软件工厂」（managed software factory）的历程。演讲的前半部分，我会讲我们正在投入的几个关键构建模块；后半部分由 Adam 讲我们如何把这些模块组合起来，为工程师搭出一套端到端、连贯的解决方案。先交代一下背景：我们有几千名工程师，分布在全球 12 个技术站点。过去一年，我们在 agentic AI 上的所有投入，已经让超过 70% 的 PR 是由本地或云端 agent 生成的。这一切带来的结果是：每位工程师的代码行数同比翻了一倍。而且这远不止于写代码，今天在工程生命周期的每个环节我们都能看到这种变化。
[1:00] Uday
And we are also accelerating toil at a toil reduction at a massive pace. We handled more than 250 automated migrations 9 million lines of code automatically for our engineers. And before even the building blocks um you know one all the investments we made over the last six years on moving to monor repos moving to basil all of that also laid a really solid foundation for us to accelerate this. So the first I'll cover all of these six building blocks and Adam's going to talk about a specific example and show how that feature can be built end to end uh with all of these and all of these are in various stages of maturity and rollout within Uber. But we want to give everyone a sneak peek of what we are up to.
我们也在以极快的速度削减重复劳作（toil）。我们已经为工程师自动完成了 250 多项自动化迁移，涉及 900 万行代码。而且在说这些构建模块之前要提一句，我们过去六年在迁移到 monorepo、迁移到 Bazel 上的所有投入，也为我们加速这一切打下了非常扎实的基础。所以接下来我先把这六个构建模块讲一遍，然后 Adam 会用一个具体的例子，展示一个功能如何借助这些模块端到端地做出来。这些东西在 Uber 内部都处于不同的成熟度和推广阶段，但我们想让大家先睹为快，看看我们在做什么。
[1:46] Uday
So let's go to the building blocks the six building blocks one by one. The first one is model gateway. This is one of our earlier investments. The three things that we wanted to en ensure was no PII ever leaves our parimeter to any of the vendor by default and any guardrail that we add here the latency of that is strictly bounded and every request that goes through this whether it's uh and we need to be able to attribute per user per project and per team. So [snorts] we have a model gateway. We made sure all of our internal use cases, our coding hardnesses, our external use cases, they all go through one single OpenAI anthropic compatible endpoint. It goes through a series of middleares. The first one is identity and authentication using Spire. Uh we have a data anonymizer that redacts 20 plus PII types. We have a AI guard that has five specialized models that handles various parts of safety and policy that we want to ensure. And all of that runs under 100 milliseconds. We also are investing in all kinds of caching and token optimization strategies at this layer.
那我们就一个一个来看这六个构建模块。第一个是模型网关（model gateway），这是我们比较早期的投入之一。我们想确保三件事：一是默认情况下任何 PII 都绝不会离开我们的边界流向任何供应商；二是我们在这里加的任何护栏（guardrail），它的延迟必须有严格上限；三是每一个经过这里的请求，我们都要能按用户、按项目、按团队进行归因。所以我们做了一个模型网关。我们确保所有内部用例、我们的编码 harness、外部用例，全部走同一个兼容 OpenAI / Anthropic 协议的端点。请求会经过一系列中间件：第一层是基于 SPIRE 的身份认证；然后有一个数据匿名化器，可以脱敏 20 多种 PII 类型；还有一个 AI guard，内含五个专用模型，分别处理我们要保障的安全与策略的不同方面。所有这些加起来运行时间都在 100 毫秒以内。我们也在这一层投入各种缓存和 token 优化策略。
[2:52] Uday
And every request that goes through this, we are able to attribute to a specific project in our catalog. And we can attribute per caller, per user, per team both in real time but also in our data lake. This enables us to create all kinds of spend years and guardrails in a holistic way across our portfolio. We also use this layer for capturing audit log session traces which are then plugged into our benchmarking and all kinds of self-improvement loop efforts. And for an engineer at Uber, you take the vanilla client, you set the project ID and we and we take care of everything else. Today we have 800 plus projects internally going through this cumulatively handling more than 100 million model requests per day. This includes both the frontier models and also open source models whether that is hosted in our infrastructure or some of our vendors.
每一个经过这里的请求，我们都能归因到目录里的某个具体项目。我们可以按调用方、按用户、按团队做归因，既能实时看到，也能沉淀在数据湖里。这让我们能够在整个组合层面上，以整体视角设置各种支出额度和护栏。我们也用这一层来采集审计日志和会话追踪（session trace），然后把它们接进我们的基准测试以及各种自我改进循环的工作里。对 Uber 的工程师来说，你拿一个原生客户端，设置一下 project ID，剩下的全部由我们搞定。今天我们内部有 800 多个项目走这条通道，累计每天处理超过 1 亿次模型请求。其中既有前沿模型，也有开源模型，不管它是托管在我们自己的基础设施上，还是在某个供应商那里。
[3:46] Uday
The next is how do we provide tools to all of these models. Last year when we started on this journey we had thousands of internal APIs but none of them are agent accessible out of the box and we had so many other SAS tools and each one of them have different way to authenticate different way to set up which is a lot of hassle for for everyone and once you end up with enough MCPs they'll all add up to and and have a massive token tax similar to model gateway we have an MCP gateway that handles whole bunch of middleares for for engineers And we have an automated crawler that looks at our internal APIs and projects all of these into MCPS with one single config change. And we do the same thing even for our SAS MCPS whether it's Google, Slack, Jira, all of this, they go through the MCP gateway. We h we host them, we do the token exchange. So for all the engineers, they go through one single entry point, one common way to install any MCPS. This simplified a lot for all of our engineers and employees.
下一个问题是：我们怎么把工具提供给这些模型。去年刚启动这项工作时，我们有数千个内部 API，但没有一个是开箱即用、能让 agent 直接访问的。我们还有一大堆 SaaS 工具，每一个的认证方式不同、配置方式也不同，对所有人来说都很麻烦。而且一旦 MCP 装得够多，它们加起来就会产生巨大的 token 税。所以和模型网关类似，我们做了一个 MCP 网关，替工程师处理一整套中间件。我们还有一个自动爬虫，会扫描我们的内部 API，只要改一处配置就能把它们全部投射成 MCP。对我们用的 SaaS MCP 我们也是同样处理——不管是 Google、Slack、Jira，全部走 MCP 网关。我们来托管它们，我们来做 token 交换。所以对所有工程师来说，只有一个统一入口、一种通用方式来安装任何 MCP。这大大简化了所有工程师和员工的工作。
[4:49] Uday
And then the whole bunch of token optimization strategies. We initially had direct MCP pattern. Earlier this year we created Omni MCP which is one single MCP that you install which can discover and invoke any MCPS within the gateway. And couple of months ago we projected all of these MCPS into CLI pattern so that even the response doesn't eat up in your context. And of late we also have a code mode skill which is autoinstalled which on the fly creates Python scripts to hyper optimize some of the top MCP token consumer consuming use cases and all of this led to like now we have thousand plus MCP tools and uh just with these optimization efforts we've saved more than 40% fleetwide savings.
然后是一整套 token 优化策略。最初我们用的是直连 MCP 的模式。今年早些时候我们做了 Omni MCP——你只需要装这一个 MCP，它就能发现并调用网关内的任何 MCP。几个月前我们又把所有这些 MCP 投射成了 CLI 模式，这样连返回结果都不会占用你的上下文。最近我们还有一个会自动安装的 code mode skill，它能即时生成 Python 脚本，对那些消耗 token 最多的 MCP 用例做极致优化。所有这些加起来，现在我们有一千多个 MCP 工具，而仅靠这些优化工作，我们就在全公司范围内节省了 40% 以上。
[5:41] Uday
So once we have the models and the tools, we need a place to run all of this. For many years we had devpod which is our cloud remote environments. We we we had this because we had like large mono repos with millions of lines of code and this is how engineers work at Uber. And now we took what we had with devots and we agentified that. Now we need some environment for agents to run for longer period of time. They need to be quick. They need to be isolated. We can install any number of them and they need to be globally available across all of our sites. So we have a pre-provisioned Kubernetes balloon pods. When an agent requires a new environment to run, it can take one of that which is already pre-provisioned. It has all of the repositories already snapshotted. The search index is already built. So the agents can start working within a matter of seconds.
有了模型和工具之后，我们还需要一个地方来跑这一切。多年来我们一直有 Devpod，也就是我们的云端远程开发环境。之所以有这个东西，是因为我们有几百万行代码的大型 monorepo，Uber 的工程师就是这样工作的。现在我们把 Devpod 原有的东西拿过来做了 agent 化改造。现在我们需要一种能让 agent 长时间运行的环境：要快、要隔离、要能想装多少装多少，而且要在我们所有站点全球可用。所以我们有预先配置好的 Kubernetes balloon pod。当一个 agent 需要一个新环境来运行时，它可以直接取一个已经预配置好的——所有仓库都已经做好快照，搜索索引也已经建好，所以 agent 几秒钟内就能开始干活。
[6:32] Uday
The next thing we noticed is the the the roles of engineers are getting blurred. We used to offer a dev port per language flavor for Go, Java, Android and so on. Now we need agents to work across repositories and engineers also to work across repositories. So we have a mega dev port that has all of the repositories in one one common place and this is what we use for our autonomous coding agents now. And even for our non-engineer employees, we are providing a simple way for them to get started with any of the agent harnesses in matter of seconds. Then now we get to knowledge part of it and we jumped on this bandwagon earlier this year. Uh we started noticing engineers building tons of skills across many repositories. Um one three problems we noticed was there's a lot of duplication same skill being built by different engineers in different discovery and configuration was a huge hassle and a lot of skills were of superpar quality. So what we built an entire life cycle around skills. So we have core core skills and domain specific skills. All of that go into a managed skills marketplace. We have 2,500 skills there right now. Um and it goes through a whole bunch of lint checks, automated reviews which ensures a baseline skill quality for any skills that we have. And we also simplified the installation and discovery. So there is
接下来我们注意到，工程师的角色边界正在变得模糊。以前我们是按语言提供 Devpod 的，Go、Java、Android 各一种。现在我们需要 agent 跨仓库工作，工程师也要跨仓库工作。所以我们做了一个 Mega Devpod，把所有仓库放在同一个地方，这就是我们现在给自主编码 agent 用的环境。甚至对非工程岗的员工，我们也提供了一种简单方式，让他们能在几秒钟内用上任意一种 agent harness。接下来说知识这部分，我们是今年早些时候跟上这波潮流的。我们开始注意到工程师在很多仓库里建了大量的 skill。我们发现了三个问题：一是大量重复，同一个 skill 被不同工程师在不同地方各建一遍；二是发现和配置非常麻烦；三是很多 skill 质量不过关。所以我们围绕 skill 建了一整套生命周期。我们有核心 skill 和领域专属 skill，全部进入一个受管的 skill 市场（marketplace），现在那里有 2,500 个 skill。它们都要经过一整套 lint 检查和自动化审查，以保证我们所有 skill 有一个基线质量。我们也简化了安装和发现。所以现在有
[7:57] Uday
one single command to discover and install any plug-in in our ecosystem. And based on the engineer personas, we even autoinstall some of the default skills. So the agents automatically can pick up the right skill. You don't even have to even install them. And of late, we started working on collecting traces and comments and capturing continuous evals so that we can go give feedback back to the skill authors for skill improvements. And this is an area of big investment for us right now. and we have 2,500 skills and cumulatively more than 20,000 skill executions per day across our fleet. The next piece of knowledge is context graphs. Uh we we started noticing in our execution traces agents spending lot of time even trying to find basic context especially in our large monor repos. You need to identify where the service is located, what are the dependencies, um who owns it, what kind of patterns I need to follow. And all of this context is gathered across scattered systems across Uber. There's 20 to 30 different systems. Each needs its own skill skills, its own MCP to gather the context. And this burns tokens. This adds a lot of latency and it creates more unpredictable outcomes.
一条命令，就能发现并安装我们生态里的任何插件。而且根据工程师的角色画像，我们还会自动安装一些默认 skill，这样 agent 就能自动选到合适的 skill，你甚至都不用自己去装。最近我们开始做的是收集 trace 和评论、采集持续 eval，这样就能把反馈回传给 skill 作者来改进 skill。这是我们当下重点投入的一个领域。我们有 2,500 个 skill，全公司累计每天超过 2 万次 skill 执行。知识的下一块是上下文图谱（context graph）。我们在执行 trace 里注意到，agent 花了大量时间在寻找最基本的上下文上，尤其是在我们的大型 monorepo 里：你得搞清服务在哪儿、依赖是什么、归谁负责、该遵循什么模式。而这些上下文散落在 Uber 各个系统里，有二三十个不同的系统，每个都需要自己的 skill、自己的 MCP 去采集上下文。这会烧 token、增加大量延迟，还会让结果更不可预测。
[9:15] Uday
So we have one context graph. We took all of the information of how Uber runs into one context graph. This has 150 unique node and edge types. We have 40 million entries there right now. It captures all the way from how our mobile apps are built to our back end to our data lake. All the design docs, Jira, incident bugs, everything is connected and this enables agents to quickly find the right context within our ecosystem. We are now plugging all of our skills and use cases into the graph whether it's our on call RCAs whe it's a planning or data analysis or security scans and we see across all of this they we are improving uh the skills by a lot and I'm just showing a very simple example of asking a simple question of how many mobility trips in India are are cash this needs to understand the concepts of each of these which tables what kind of cities you need to create for this SQL with and without graph we see massive improvement in tokens, turns and latency and we see that across any earlier val that we did within our infrastructure and the last thing is how do we package all of this for everyone in the company to use. So we have uh our AI assistant called Cortana. All of the things that I mentioned so far, whether it's skills,
所以我们做了一个统一的上下文图谱。我们把 Uber 是怎么运转的全部信息都装进了这一个图谱。它有 150 种不同的节点和边类型，目前有 4,000 万条记录。它覆盖从我们的移动 App 怎么构建，到后端，再到数据湖的全部内容。所有设计文档、Jira、故障和 bug，全部连在一起，这让 agent 能在我们的生态里快速找到正确的上下文。我们现在正把所有 skill 和用例接进这个图谱，不管是 on-call 的 RCA、规划、数据分析还是安全扫描，我们看到所有这些 skill 都有很大提升。我这里只展示一个非常简单的例子：问一个简单问题——印度有多少出行订单是现金支付的。这需要理解每个概念、要用哪些表、要用什么城市维度来写这条 SQL。有图谱和没有图谱相比，我们在 token、轮次和延迟上都看到了巨大的改善，而且我们在基础设施内部做过的所有早期 eval 都是这个结果。最后一点是，我们怎么把这一切打包给全公司的人用。我们有一个 AI 助手叫 Cortana。我前面讲到的所有东西，不管是 skill、
[10:29] Uday
MCPS and context graph, they're all plugged into that in every surface possible whether it's on Slack, CLI, web. So anyone in the company, they can ask a simple question. It can look up the context graph, invoke any skill, check any code, check any code in any codebase and give an answer across any of these surfaces. And now we started allowing employees to even personalize that. You can hook up your custom skills, custom prompt and hook it up into your team Slack channel so that it it knows all of the things about that team and works like a the that teammate. And this this is a simple example of how you can invoke the same question before in Slack. Um and all of the employ more like one or more people can even collaborate on the same Slack channel.
MCP 还是上下文图谱，全部接进了它，而且覆盖所有可能的界面，不管是 Slack、CLI 还是网页。所以公司里任何人都可以问一个简单的问题，它可以查上下文图谱、调用任何 skill、查任何代码库里的任何代码，然后在任何一个界面上给出答案。现在我们还开始允许员工对它做个性化定制：你可以挂上自定义 skill、自定义 prompt，然后把它接进你团队的 Slack 频道，这样它就了解这个团队的一切，像一个队友一样工作。这是一个简单的例子，展示你如何在 Slack 里提出刚才那个问题。而且一个或多个人甚至可以在同一个 Slack 频道里协作。
[11:19] Uday
And we have the just in the last one month 300 unique personas created and more than 20,000 sessions per day. I'll now pass on to Adam who'll talk about how we take all of this and build uh take and ship a feature end to end.
仅在过去一个月，就创建了 300 个不同的 persona，每天有超过 2 万次会话。接下来我交给 Adam，他会讲我们如何利用这一切，端到端地做出并交付一个功能。
[11:34] Adam
All right. Thank you, Ud. All right. And as Ud said, we've got those building blocks. We're going to use those to power our software factory. So, we're going to take a feature here and show it going end to end through this. All right. First up, right, we need to have an idea, right? A good idea probably for this moment would be something around the World Cup, right? Would it be awesome if you were a writer and you were leaving a busy stadium if there was a better pickup location to get you away from the crowd? So, that's the idea, right? We have our idea. We're jamming on it in Slack here. Let's tag in Cortana, right? That's our AI assistant to help us with that idea.
好的，谢谢 Uday。就像 Uday 说的，我们已经有了这些构建模块，接下来要用它们来驱动我们的软件工厂。所以我们拿一个功能，展示它怎么端到端地走完整个流程。好，第一步，我们得先有一个想法，对吧？眼下一个不错的点子，可能是围绕世界杯的：如果你是一位乘客，正要离开一个拥挤的体育场，要是有一个更好的上车点能让你避开人群，那不是很棒吗？这就是我们的想法。我们在 Slack 里讨论这个点子，然后把 Cortana 拉进来——也就是我们的 AI 助手，来帮我们推进这个想法。
[12:08] Adam
Cortana with that context graph can help us determine whether this is a good business opportunity to go after. So, we can go here from Slack and now open Cortana into a web interface. And you'll see an example here of what that business research could look like, right? What other largecale venue events have happened before? What are some stadiums that would make sense here? From there, we start to think about the product requirements, right? This should be probably just a North America rollout since that's where the stadiums are. We can even then bring in Cortana to help us think about uh the Figma designs. We can create some initial mock-ups, right?
Cortana 借助上下文图谱，可以帮我们判断这是不是一个值得追的商业机会。我们可以从 Slack 出发，把 Cortana 打开到一个网页界面。这里你能看到商业调研大概长什么样：以前有过哪些大型场馆活动？哪些体育场适合做这个？接下来我们开始考虑产品需求：这大概应该只在北美上线，因为体育场都在那里。我们甚至可以让 Cortana 帮我们一起考虑 Figma 设计，做出一些初步的 mock-up。
[12:44] Adam
Do two variants here. We want to run an experiment A and B. So the the button strings here are different between the two. So we'll test those two variants and see which one performs better. Now we'll start to think a little bit about the design and Cortana 2 can help us think about what code changes we need to happen, right? What can we leverage that's in the app already? what screens and what can we leverage on the back end. Right? So this process before could take a long time. It could take weeks to get everyone aligned. Now we can compress this into a very short amount of time now. Right? And get to a prototype here very quickly.
这里做两个变体，我们想跑一个 A/B 实验，两个版本的按钮文案不一样。我们会测试这两个变体，看哪个表现更好。接下来我们开始考虑设计层面的事，Cortana 同样能帮我们思考需要做哪些代码改动：App 里现有的哪些东西可以复用？哪些页面？后端有哪些可以复用？以前这个过程可能要很长时间，让所有人对齐可能得花好几周。现在我们能把它压缩到非常短的时间，很快就能拿到一个原型。
[13:17] Adam
All right. So now we got to go build this. So we we hand off from that Cortana agent to what we have at Uber. We have a Minion agent. It's a Uber's cloud coding agent solution. All right. So you can use Minion in an interactive mode or you can run it in an autonomous mode as well. So I'm going to show you what this looks like. Um UD mentioned the dev pod building block. So this is powered by that dev pod. So it's got a full build environment and it can work across repos. So we're doing backend changes and the front end changes here too as well. We're going to see Minion kind of progress here and it's going to stop at just creating a draft PR and it's not going to push it to CI yet. The reason being is that we were seeing um uh that this was great for doing like toil sort of workloads but to build more advanced like endtoend features we really need to be able to validate uh the feature first and we want to prevent a lot of extra load coming on to CI. So if we can validate sooner before we push to CI um that would be a big benefit.
好，接下来我们得把这个东西真正做出来。于是我们从刚才那个 Cortana agent 交棒给 Uber 内部的另一个 agent——Minion agent，这是 Uber 自己的云端编码 agent 方案。Minion 既可以交互式地用，也可以跑在自主模式下。我来给大家演示一下它长什么样。刚才 Uday 提到过 dev pod 这个基础构件，Minion 就是靠 dev pod 驱动的，所以它有一套完整的构建环境，还能跨仓库工作——这里我们同时在改后端和前端。大家会看到 Minion 一步步往前推进，但它会停在「只创建一个草稿 PR」这一步，暂时不推到 CI。原因是我们发现，这种方式用来干那些琐碎的苦力活很好使，但要构建更复杂的端到端功能，我们真的需要先把功能验证一遍，同时也想避免给 CI 增加大量额外负载。如果能在推到 CI 之前更早地完成验证，收益会非常大。
[14:14] Adam
So that's what we're going to see here next on validation. Right? In the SDLC, we have an inner loop. We have the outer loop. Of course, we can have these be aensified where we're shifting more checks now to happen in this inner loop. Right? So, some of the checks that initially happen that we've had there previously is like this the static analysis sort of checks. When those are detected, now we fix those. But we can shift things to happen in inter loop. Things like visual validation. So, we can launch an simulator with a skill, grab a screenshot from the simulator, compare it to the Figma specs. We can also bring up the service and our backend staging environment and compare the front end and the backend integration together.
这就是接下来要看的验证环节。在 SDLC 里，我们有内循环（inner loop）和外循环（outer loop）。当然，我们可以把内循环做得更「密」——把更多检查前移到内循环里来做。比如，以前最早就有的那类检查是静态分析，检测出问题之后我们现在会直接修掉。但我们还可以把更多事情挪到内循环里，比如视觉验证：我们可以用一个 skill 拉起模拟器，从模拟器里截一张图，拿来和 Figma 的设计稿做比对。我们还可以把服务拉起来，连上后端的 staging 环境，把前端和后端的集成放在一起比对。
[14:53] Adam
So now that we've moved a um now that we've done uh that part uh we move to the outer loop where CI uh typically happens errors can still happen on CI right uh so self-healing CI is something that we've implemented here where we can fix a lot of the issues that you hit on CI code review is another thing that happens in the outer loop but this is another thing that we've shifted we've moved parts of code review to happen in the inner loop right the outer loop code review can have a powerful model use reasoning a skill to do a deeper review. And in the interloop, we can have a smaller model that runs uh faster with a with a with a medium model.
那么，这部分做完之后，我们就进入外循环，也就是通常跑 CI 的地方。CI 上照样可能出错，对吧？所以我们在这里实现了自愈式 CI（self-healing CI），能自动修掉你在 CI 上碰到的很多问题。代码评审也是外循环里的一个环节，但这也是我们做了前移的一块——我们把代码评审的一部分挪到了内循环里。外循环的代码评审可以用一个强模型，借助推理能力和一个 skill 做更深入的评审；而在内循环里，我们可以用一个更小、跑得更快的中等模型来做。
[15:30] Adam
Now, another key thing here, right, is this if this is an autonomous diff coming from minion, we want to give a human reviewer some confidence that this diff has gone through a lot of self-improvement already, right? That not just touching that initial generation that happened, but all these other steps have happened. And so on the PR, you will have a table attached that says all these different checks that it went through, including the screenshots. All right, so we've got a lot more code coming through the software factory now, right? Let's talk about maintenance. Right, maintenance is even more important. Um, what we have set up now is we can actually enroll our feature or skill into u our feature or ser uh service into maintenance uh skills. So these are uh some examples of those skills that we have. Uh feature flag cleanup, right? We had two variants of that world cup uh modal.
这里还有一个关键点：如果这是 Minion 自主生成的一个 diff，我们希望给人类评审者一些信心——这个 diff 已经经过了大量的自我改进，不只是最初生成那一下，后面这些步骤全都跑过了。所以在 PR 上会附一张表，列出它经过的所有这些检查，包括截图。好，现在我们的「软件工厂」里流过的代码多了很多，那我们来聊聊维护。维护反而更重要了。我们现在搭好的机制是，可以把自己的功能或者服务「注册」到各种维护 skill 里去。这是我们现有的一些 skill 的例子，比如 feature flag 清理——刚才那个世界杯弹窗（modal）有两个变体，
[16:19] Adam
Now that the the B variant is no longer needed, we can have that scheduled on a loop. So the key thing here is that this is actually a managed loop that you go to, right? We don't want thousands of loops being set up across the company without any bounds. You have a managed surface that you go to to set up the loop. So it runs on Sunday when we know we have better uh CI capacity available. U we also don't want to overwhelm engineers that Monday morning with a bunch of extra diffs. We want to control how many diffs they're seeing on Monday as well. Another cool key thing here is that when that ski uh skill runs and makes those diffs, those diffs will get comments and either get landed or not landed. That's all good label data that we can use to improve the skill itself.
现在 B 变体已经不需要了，我们就可以把清理工作安排到一个定时循环里。这里的关键在于，这其实是一个受管控的循环（managed loop），你得到一个统一的地方去设置它。我们不希望全公司冒出成千上万个毫无边界的循环。你要去一个统一管理的界面去建这个循环，它会在周日跑，因为我们知道那时候 CI 容量更充裕。我们也不想让工程师周一早上一来就被一堆额外的 diff 淹没，所以还要控制他们周一看到的 diff 数量。另外还有一点很酷：当这个 skill 跑起来并产出这些 diff 之后，这些 diff 会收到评论，然后要么被合入、要么不被合入——这些都是非常好的标注数据，可以反过来用来改进这个 skill 本身。
[17:00] Adam
And then at a kind of monthly cadence, we're looking to see what skills can we now learn um from in our incident reviews and turn those into new maintenance skills that we can apply to all of our services. All right, so you've seen uh these parts of the SDLC that we've identified. There's other parts too like monitoring. Have you seen the uh building blocks that you can use to power those and the architecture underneath them? One of the other things that we're really thinking about now is bottlenecks, right? We're now we're putting more strain on our infrastructure. So we're trying to anticipate where our CI capacity needs to be and make the right foundational investments there. There's only so many experiments that we can feasibly run as well. So that's another bottleneck. And then lastly too, right, decision- making, right? It's not about you know can we build we know we can probably build it now it's more of a question of should we build it all right so that's what we have for you today and the next talk is going to be actually from Uber as well and if you want to learn more about our agentic code review aa and will be presenting that next uh in this room thank Thank you.
然后我们大概按月的节奏去看：从事故复盘（incident review）里能学到哪些新的 skill，再把它们变成新的维护 skill，应用到我们所有的服务上。好，大家已经看到了我们识别出来的 SDLC 的这几个环节，当然还有别的环节，比如监控。大家也看到了可以用来支撑这些环节的基础构件，以及它们底下的架构。我们现在在认真思考的另一件事是瓶颈：我们正在给基础设施施加更大的压力，所以我们在努力预判 CI 容量需要做到什么程度，并在那里做好正确的基础投入。我们实际能跑的实验数量也是有限的，这是另一个瓶颈。最后还有一点，就是决策。问题已经不再是「我们能不能做」——我们知道现在大概率能做出来——而更多是「我们该不该做」。好，今天我们要分享的就是这些。下一场演讲同样来自 Uber，如果你想进一步了解我们的 agent 化代码评审，接下来 AA 和 Will 会在这个房间里讲。谢谢大家。
[18:22]
[music]
（音乐）