The 5 Levels of Self-Driving Production — Eric Schwartz, Traversal

频道: AI Engineer
视频: https://www.youtube.com/watch?v=y-OVWZD4j6U
原文语言: en
统计: 共 14 轮 · Eric 13

[0:01] Eric

[music] Hey everybody. Um, my name is Eric. I'm a product manager at Traversal and today we're going to talk about self-driving production. So, just a quick primer on what I'll cover. Uh I'm going to talk a little bit about how AI agents are changing the software development life cycle. Uh the problems that that introduces for engineers and how AI for site reliability engineering fits in uh which is where traversal comes into play. And then I'll talk about a few examples of um what this looks like in practice for enterprises that we work with at Traversal uh and a bit about how we've built our AIS surre uh to deliver the results that we're seeing with Fortune 500 companies. So uh I guess to start some quick context on uh kind of the different phases of software engineering. Uh so sure you're all familiar but at a high level you could think about software engineering as kind of three buckets. There's system design like what is it that I want to build? What's the architecture that I want to build? There's development which is actually writing the code building the thing. And then there's troubleshooting. So once you build something and it's interacting with the real world, it starts to break uh and you need to fix it. And so with the advent of coding agents, the development portion has shrunk.

大家好，我叫 Eric，是 Traversal 的产品经理，今天我们来聊聊自动驾驶式生产环境（self-driving production）。先简单交代一下我要讲的内容。我会讲一讲 AI agent 正在怎样改变软件开发生命周期，这给工程师带来了哪些问题，以及用 AI 来做站点可靠性工程（site reliability engineering，SRE）是怎么切进来的——这也正是 Traversal 派上用场的地方。然后我会讲几个例子，看看在我们 Traversal 合作的企业里，这件事实际落地是什么样子，也会稍微讲讲我们是怎么搭建自己的 AI SRE，才做出了我们在 Fortune 500 公司那里看到的这些结果。那么开头先交代一点背景，说说软件工程的几个不同阶段。相信各位都很熟悉，不过粗略来讲，你可以把软件工程看成三大块。一块是系统设计，也就是我到底想做什么？我想搭一个什么样的架构？一块是开发，就是真正去写代码、把东西做出来。然后还有一块是排障（troubleshooting）。东西一旦做出来、开始跟真实世界打交道，它就会开始出故障，你就得去修。而随着编码 agent 的出现，开发这一块缩小了。

[1:34] Eric

development is a lot faster. Folks like me without any formal computer science training can now contribute u production code. And so this is great. Development's a lot faster. Teams can do 10x more. But then what happens to either end of the spectrum? What happens to system design and what happens to troubleshooting? The hope is that you spend more time on the design part thinking creatively about what it is that you want to build. But what we actually see with the enterprises that we work with is that um more and more time is being spent on troubleshooting. Uh much more code is being written. People have less understanding of the code that's being pushed into production. And so what you end up with is more issues, more comp uh more complex environments. Uh, and instead of spending all your time on the creative architecture and design, your teams are bogged down on troubleshooting. And the the data kind of bears this out. This is a big problem. Um, by some accounts, enterprises are spending upwards of $400 billion a year. Uh, 40% of executives say that this is a problem that their teams face. And on average, engineers are losing seven plus hours each week uh just troubleshooting when they're on call.

开发快了很多。像我这样没受过任何正规计算机科学训练的人，现在也能贡献生产代码了。这很好。开发快了很多，团队能做的事是原来的 10 倍。可是这条光谱的两头会怎么样呢？系统设计会怎么样，排障又会怎么样？大家的期望是，你能把更多时间花在设计那一头，创造性地去想自己到底要做什么。但我们在合作的企业那里实际看到的是，越来越多的时间被花在了排障上。写出来的代码多了很多，大家对推到生产环境里的代码却理解得更少了。所以最后你得到的是更多的问题、更复杂的环境。你的团队没能把时间都花在有创造性的架构和设计上，反而陷在排障里脱不开身。数据也多少印证了这一点。这是个大问题。按某些统计，企业每年为此要花掉 4000 亿美元以上。40% 的高管说这是他们团队面临的问题。而工程师在 on call 值班的时候，平均每周光是排障就要搭进去 7 个多小时。

[2:51] Eric

And this is going to become an increasing issue that we'll all hear more about, that we'll all experience more over time as tools like cloud code and codeex and cursor, which are fantastic, um uh they are fantastic, but they result in more code and more complexity. And so what are we supposed to do about it? Um there are all kinds of great observability tools. Tools like data dog, elastic, splunk, service now where I worked for several years before joining traversal. Um these are great but uh there's only so much that they can do like these tools will tell you that something is broken. They'll point out all the things that have broken. Um they'll point out things that are maybe correlated but they won't tell you the root cause of the issue. And when I worked on observability products at Service Now, this was something I heard time and again from the customers that I served, which was like, you're telling me what's broken, you're not telling me why, and you're not telling me what to do about it. And so, no matter how many dashboards you create in these tools, it's not really helping teams keep up with the pace of innovation uh and complexity that's being added to their environments.

而且这会成为一个越来越突出的问题，往后我们都会越来越多地听到它、越来越多地亲身碰上它，因为像 Claude Code、Codex、Cursor 这样的工具——它们非常棒，确实非常棒——但它们带来的是更多的代码和更高的复杂度。那我们该拿它怎么办呢？市面上有各种很不错的可观测性工具。像 Datadog、Elastic、Splunk、ServiceNow 这些——我在加入 Traversal 之前，在 ServiceNow 干过好几年。这些工具都很好，但它们能做的也就这么多。这些工具会告诉你有东西坏了，会把所有坏掉的东西都指出来，会指出一些可能有相关性的东西，但它们不会告诉你问题的根因是什么。我在 ServiceNow 做可观测性产品的时候，这话我从自己服务的客户那里听了一遍又一遍，大意就是：你告诉了我什么坏了，却没告诉我为什么坏，也没告诉我该拿它怎么办。所以不管你在这些工具里建多少个 dashboard（仪表盘），它都没法真正帮团队跟上创新的节奏，也跟不上他们环境里不断叠加的复杂度。

[3:59] Eric

And this is not just something we believe at Traversal. This is kind of a widely held view. um Google who kind of wrote the S sur handbook like the bible of site reliability engineering uh has commented on this directly and we also heard recently from anthropic u quotes where they described in their own words that LLM are not great at this problem. They're not great at pointing out what is the root cause of an issue. Now like why is this so hard? What makes this so challenging? Um in this particular example that we have on the screen like which was based on something we saw with one of our customers. Um if a checkout API is failing uh you might need to make five to 10 hops across like dozens of services and pabytes of data to actually get to the root cause. And so in a very small contained environment, perhaps there's a single seasoned SR at your company that knows the full stack and can debug everything and has all the tribal knowledge to sift through all your data. But if you're a Fortune50 enterprise or a Fortune 100 enterprise, there's no single engineer that has all of that context. And so what you end up with is a war room with 50 engineers like pulled in uh and many many hours trying to debug this issue because everyone has their own particular slice of context. No one has kind of the full scope and breadth of this. And so no, it's really challenging for a human or a human paired with cloud code to get kind of the full spectrum of context needed to jump from a checkout API is failing over here five hops away to the expired TLS certificate that maybe caused the issue.

而且这不只是我们 Traversal 自己这么认为，这算是一个被广泛认同的看法。Google——SRE handbook（SRE 手册）可以说就是他们写的，那本书堪称站点可靠性工程的圣经——就直接谈过这一点；我们最近还听到了 Anthropic 的一些原话，他们用自己的话讲，LLM 并不擅长这个问题，不擅长指出一个问题的根因是什么。那这件事为什么这么难？到底难在哪儿？就拿屏幕上这个例子来说，它取材于我们在一家客户那里看到的情况。如果一个 checkout API 出故障了，你可能得在几十个服务、PB 级的数据之间跳上 5 到 10 跳，才能真正找到根因。在一个很小、很封闭的环境里，也许你公司里有那么一位资深 SRE，他懂整个技术栈，什么都能调，脑子里装着所有口口相传的经验（tribal knowledge），能把你所有的数据筛一遍。但如果你是一家 Fortune 50 或者 Fortune 100 的企业，就没有哪一个工程师能掌握全部这些上下文。所以最后的局面就是开一个 war room（作战室），拉进来 50 个工程师，花上很多很多个小时去排查这个问题，因为每个人手里都只有自己那一小片上下文，没有人掌握它完整的范围和广度。所以说，不管是一个人，还是一个人配上 Claude Code，都很难拿到所需的全部上下文，从这边的「checkout API 出故障了」，跳到五跳之外那张可能是肇事者的过期 TLS 证书。

[5:46] Eric

And so that's kind of what motivated u our founders to to launch traversal. Fundamentally the belief that we have at traversal is that uh root cause analysis is not an observability problem. It's a it's a causal problem. And so that's why our four founders, three of them come from academia. One comes from quant finance. Uh and they've dedicated like decades of their lives towards causal machine learning which is how do you find cause and effect? uh and have applied it to this problem because it's a massive one that enterprises are really struggling with. And so that kind of brings us to the topic of this talk which is self-driving production. Um what we're building at Traversal is the ability for an enterprise to basically have a self-driving production environment. What that means is a closed loop system where uh as issues break the AI system finds the issue the finds the issue or the incident or the alert finds the root cause of what caused that issue in the first place puts up a fix verifies that it's been fixed and the loop is closed with hopefully without ever having to page or disrupt any of your team members so that they can keep focusing on building. And this is a spectrum. U a lot of companies are not fully there yet. We like to think of it in terms of levels almost like self-driving cars uh or autonomous driving. We think about it in the same frame. So level zero is where most companies are today where everything is fully manual. You're pulling a bunch of people into a Slack war room or onto a Zoom call. You're spending hours debugging things. Level one might be where you have some rules and automations. And we're hearing a lot about like loops that teams are building now with tools like cloud code or cursor or codeex. Uh and so these might be like rulebased automations where you can build out a very structured specific workflow in response to an alert. Those rule-based automations kind of break

这大概就是促使我们的创始人创办 Traversal 的原因。归根结底，我们在 Traversal 的信念是：根因分析（root cause analysis）不是一个可观测性问题，而是一个因果问题。所以我们的四位创始人里，有三位来自学术界，一位来自量化金融。他们把自己几十年的人生都投在了因果机器学习（causal machine learning）上，也就是怎么找出因和果，然后把它用到了这个问题上，因为这是个巨大的问题，企业正被它搞得焦头烂额。这也就把我们带到了这场演讲的主题：自动驾驶式生产环境。我们在 Traversal 做的，就是让一家企业基本上能拥有一个自动驾驶式的生产环境。意思是一个闭环系统：一旦出了问题，这套 AI 系统会发现这个问题——发现这个问题、这起事故或者这条告警，找出最初导致它的根因，提交修复，验证确实修好了，然后闭环，而且最好是从头到尾都不用去呼叫（page）或者打扰你团队里的任何人，让他们能继续专心做开发。这是一条光谱。很多公司还没有完全走到那一步。我们喜欢用等级来看它，差不多就像自动驾驶汽车，或者说自主驾驶那样，我们用的是同一套框架。Level 0 是今天大多数公司所处的位置，一切完全靠人工。你把一堆人拉进一个 Slack 上的 war room，或者拉上一个 Zoom 会议，花好几个小时排查问题。Level 1 可能是你有了一些规则和自动化。我们现在常听到团队用 Claude Code、Cursor、Codex 这样的工具搭各种循环（loop）。这些可能就是基于规则的自动化，你可以针对一条告警，搭出一个结构非常明确、非常具体的工作流。而这些基于规则的自动化往往

[7:51] Eric

down when it's a novel situation. And usually a sub one is something you haven't seen before. There's not really a runbook for it. And so the rule-based automation breaks down there. And level two and level three is kind of where you start to get a little more automation built in. In level three, perhaps you have like a a homegrown agent that is really good at debugging a issues for a specific service or a specific set of alerts. But really where we see the leap is going from level three to level four, which is cutting across an entire environment. So hundreds of services like hundreds of repos, thousands of lo log indexes. It's quite challenging to homegrow a solution that can handle that. And level five is kind of the holy grail, which is not only can we diagnose issues across a full production environment for a large enterprise, but also put up the fixes and verify the fixes as well. And so, uh, we're really proud that at Traversal, we get to work with and serve some of the biggest companies in the world. Companies like American Express, who I spend a lot of time with personally, uh, Pepsi, Digital Ocean, Capital One. Um, and it's really cool to see the volume and scale of data that Traversal is able to handle. So, you know, trillions of logs that are generated, trillions of spans, tens of billions of metrics and events, um, and hitting 80% plus root cause uh, for these high severity incidents at this scale is a technical feat that we're pretty proud of.

一碰上没见过的新情况就不灵了。而一起 Sev 1（最高严重级别的事故）通常就是你以前没见过的东西，并没有现成的 runbook 可用。所以基于规则的自动化到这儿就失效了。Level 2 和 Level 3 大概是你开始内建更多一点自动化的阶段。到了 Level 3，也许你有一个自研的 agent，它特别擅长排查某一个特定服务、或者某一组特定告警的问题。但我们看到真正的飞跃是从 Level 3 到 Level 4，也就是横跨整个环境。那可是几百个服务、几百个仓库、几千个日志索引。要自研出一套能应付这种规模的方案是相当难的。Level 5 算是圣杯：不光能在一家大型企业的整个生产环境里诊断问题，还能提交修复，并且把修复也验证了。所以我们非常自豪，在 Traversal 能跟全世界最大的一些公司合作、为它们服务。像 American Express——我个人在他们身上花了很多时间——还有 Pepsi、DigitalOcean、Capital One。看到 Traversal 能处理的数据量和规模，真的很酷。你想，产生的日志是万亿级的，span 是万亿级的，指标和事件是几百亿级的，在这种规模下，对这些高严重级别的事故做到 80% 以上的根因命中，是一项让我们相当自豪的技术成就。

[9:26] Eric

And so, beyond just these kind of results at a high level, um, as a product manager, I like to think about the use cases that we build for our customers. And so there's there's a few different types of scenarios and use cases that we found that can deliver a lot of value. I'll dig into a few in more detail, but at a high level, there's alert intelligence, a product that I spend a lot of time on, which is helping on call teams triage and prioritize hundreds or thousands of alerts that they may be receiving in a single day, helping you deal with alert fatigue. There is incident root cause analysis, which is where we got started. How do we help you find the root cause, pinpoint the root cause of an incident in minutes when it otherwise would have taken hours and dozens of engineers? There is self-healing, which is not only telling you what the root cause was, but putting up a fix to address and mitigate that issue. There's production support, which is the the variety of questions and issues that your team might be dealing with in any given day, wanting to chat with their production environment, get specific data points that maybe aren't part of a a broader incident, but are still taking up a lot of time and toil for your teams. And then pre-production, there is code resilience. Like when you're putting up changes, how do you make sure that this code is making your system more resilient and reliable over time?

除了这些宏观层面的成果之外，作为产品经理，我喜欢从我们为客户打造的使用场景来想问题。我们发现，有几类不同的场景和用例能带来很大的价值。其中几个我待会儿展开细讲，先整体过一遍：首先是告警智能（alert intelligence），这是我自己花了很多时间的一个产品，它帮值班（on-call）团队对一天之内可能收到的几百上千条告警做分诊、排优先级，帮你对付告警疲劳。然后是事故根因分析，我们就是从这儿起家的：怎么帮你在几分钟之内找到、精确定位一次事故的根因——换作以前，这得花上好几个小时、动用几十位工程师。还有自愈（self-healing），不光告诉你根因是什么，还会提交一个修复，把这个问题处理掉、缓解掉。还有生产支持，也就是你的团队每天都可能碰上的各种各样的问题和状况：想跟自己的生产环境对话，想拿到某些具体的数据点——这些事也许不属于哪一次大事故，但照样在耗掉团队大量的时间、带来大量 toil（重复琐事）。再往前到上生产之前，还有代码韧性（code resilience）：你提交改动的时候，怎么确保这些代码是在让你的系统随着时间推移变得更有韧性、更可靠？

[10:47] Eric

And so at Traversal, we cover kind of the whole gamut of use cases. Um, but I'll dig into alert intelligence and incident root cause just to give you a bit of a more detailed sense of what that might look like, what that looks like, and I'm happy to answer any questions after about the others. So for alert intelligence, the example that I have here is uh based on our engagement with Pepsi. And so Pepsi uses traversal across a bunch of their different applications, but a very core one is for their supply chain. They have a lot of internal tooling and software to make sure that um the raw materials and the finished product can move through their supply chain. And a very critical piece of that is making sure that finished product gets from their warehouse to their trucks to their retailers at the end. And um before traversal their team would get thousands or tens of thousands of alerts every single week. Uh and at any given time a single engineer might have a backlog of 700 alerts. This is kind of a crippling state to be in. Like you have so much noise that you don't know where to look. You don't know what's broken and things slip through the cracks. Uh or you just try to keep up and burn out. And so this was a big problem that Pepsi was facing and they used traversal uh to help parse the signal from the noise of these tens of thousands of alerts. And so now instead of each engineer having a backlog of 700 alerts, they get a very pri a prioritized and filtered set of alerts that have already been pre-investigated.

所以在 Traversal，这一整套使用场景我们基本都覆盖了。不过我会重点讲告警智能和事故根因这两个，让大家更具体地感受一下它实际是什么样子；至于其他几个，结束之后我很乐意回答任何问题。先说告警智能，我这里的例子来自我们和 Pepsi 的合作。Pepsi 在很多不同的应用上都在用 Traversal，但其中非常核心的一块是他们的供应链。他们有大量的内部工具和软件，用来保证原材料和成品能在供应链里顺畅流转。其中极为关键的一环，就是确保成品从仓库装上卡车、最终送到零售商手里。在用 Traversal 之前，他们的团队每周都会收到几千甚至几万条告警。任何一个时间点上，单单一位工程师手里就可能积压着 700 条告警。这种状态基本是能把人压垮的。噪音多到你根本不知道该往哪儿看，不知道到底是什么坏了，于是有些问题就这么漏掉了；要不然你就拼命去跟，最后把自己耗到倦怠。这就是 Pepsi 当时面对的一个大问题，他们用 Traversal 来帮忙，从这几万条告警的噪音里把信号分辨出来。所以现在，每位工程师不再是积压着 700 条告警，而是拿到一组排好优先级、筛选过的告警，而且都已经预先调查过了。

[12:27] Eric

We flag to them the alerts that are actually worth digging into. We flagged to them opportunities to reduce alert noise, whether that's updating the underlying alert rules in their code, uh dismissing alerts that don't need to be addressed right now, uh or creating tickets for alerts that are maybe technical debt that can be handled later on but aren't super important to deal with at this moment. And so we've seen a lot of great results with Pepsi uh on this alert use case and many of our other customers as well. Um the next example is around incident root cause analysis and and this is a case study from our engagement with annex um where I've personally spent a lot of time myself. Um, so before working with traversal, what an incident looked like at American Express is anywhere from five to 10 teams get paged, anywhere from 20 to 50 engineers, and it could take, in this particular example, 60 minutes, but it could take hours or days to get resolution to to an incident. You can imagine if customers aren't able to pay their credit card bill, like that's a big problem. Or if they're not able to log into their mobile app, that's a big problem. And so now uh traversal is the first responder to every single incident that's created at American Express. And so what that looks like is the second the bridge is declared. That's the term they use internally. The second the incident is declared traversal is dispatched within 3 minutes we'll post a very detailed root cause analysis in the Slack channel where that incident is being managed. We can post updates to Service Now tickets if they want to as well. And rather than paging five teams and 58 engineers because you have that initial analysis, it's a lot less chaotic, you have a sense of what actually broke. And either no teams get paged or maybe you page the one or two teams just to verify the findings that traversal posted. Uh and so we save in this case like 50 plus engineers uh from

我们会把真正值得深挖的告警标出来给他们，也会把能降低告警噪音的机会标出来给他们——不管是更新他们代码里底层的告警规则，还是把眼下不需要处理的告警直接关掉，或者给某些告警建工单：这些可能属于技术债，可以留到以后再处理，眼下并不是特别要紧。在这个告警场景上，我们在 Pepsi 看到了很多很好的结果，在其他很多客户那里也一样。下一个例子是事故根因分析，这是我们和 Amex 合作的一个案例，我自己在这上面也亲自花了很多时间。在跟 Traversal 合作之前，American Express 的一次事故是这样的：会有 5 到 10 个团队被呼叫（page），20 到 50 位工程师；具体到这个例子，花了 60 分钟，但一次事故要花上几个小时甚至几天才能解决，也是有可能的。你可以想象，如果客户没法还信用卡账单，那是个大问题；如果他们登不上手机 App，那也是个大问题。而现在，American Express 每创建一起事故，Traversal 都是第一响应者。具体是这样的：bridge 一宣布——这是他们内部的叫法——也就是事故一宣布，Traversal 就被派出去，3 分钟之内，我们就会在处理这起事故的那个 Slack 频道里贴出一份非常详细的根因分析。如果他们需要，我们也可以把更新同步到 ServiceNow 的工单里。这样就不用去呼叫 5 个团队、58 位工程师了——因为有了这份初步分析，场面没那么混乱，你对到底是什么坏了心里有数。结果要么一个团队都不用呼叫，要么只呼叫一两个团队，核实一下 Traversal 贴出来的结论就行。所以在这个例子里，我们帮 50 多位工程师省下了

[14:28] Eric

the time and headache of being paged uh into this incident. Um, and that's especially appreciated when these incidents are happening in the middle of the night. Like this is a a good night of sleep that 53 engineers can have now. And so those are those are two of the use cases that we're especially proud of at Traversal, but like I mentioned, there's a bunch of others. Maybe in the last couple minutes I'll just highlight like AI site reliability engineering is a very hot space. You've probably, if you're familiar, you've probably heard of a lot of companies in the space or claiming that they're solving these problems. Um, we think that there's a few really key questions you should be asking uh that are critical to getting this right. So, the first is can your AI surre see all of your production data? If you're only giving it a slice or if it has gaps in its understanding, it's going to be very hard to get to a very detailed root cause. The second is can it search through this data can be it's often pabytes of data hundreds of billions of logs um without blowing up costs or taking down your observability infrastructure. Um this is really not trivial to do at scale for a Fortune 100 enterprise.

被呼叫进这起事故的时间和头疼。要是事故发生在大半夜，这一点就格外让人感激了——等于现在有 53 位工程师能睡上一个好觉。这就是我们在 Traversal 特别引以为豪的两个使用场景，不过就像我前面说的，还有不少别的。最后这几分钟，我想强调一点：AI 站点可靠性工程（AI SRE）是个非常火的赛道。如果你对这块有所了解，大概听说过很多公司在做这个，或者号称自己在解决这些问题。我们认为，有几个非常关键的问题是你应该问的，它们直接决定这件事能不能做对。第一个：你的 AI SRE 能不能看到你全部的生产数据？如果你只给它其中一小块，或者它的理解里有缺口，那就很难得出非常细致的根因。第二个：它能不能把这些数据搜个遍——这往往是 PB 级的数据、几千亿条日志——同时既不让成本爆掉，也不把你的可观测性基础设施拖垮？对一家《财富》100 强企业来说，要在这种规模上做到这一点，真的不是件容易的事。

[15:44] Eric

The third is can it map out the relationship relationships between all of these entities in your data. even if you're able to ingest and read through all this data, if you don't know what to do with it and if you're lost, it's kind of worthless. And then four, does it get better and smarter over time autonomously without having to dedicate engineering resources to maintaining a bunch of markdown files or having a bunch of forward deployed engineers taking up your your team's time to maintain the system knowledge? And finally, u can it make multiple hops? Can it find the non-trivial, non-obvious root cause that's far away from the initial symptom in a matter of minutes? What we've seen is like anything more than 5 minutes and you've kind of lost the plot. You've lost the patience of the on call team and people will fall back onto their own habits. So, it needs to be really good. It needs to be really fast. And I won't go into this in too much detail, but this is kind of how we think about these five questions at traversal. So at the bottom of the stack is how we think about kind of a analyzing all your data uh without increasing cost. So basically how do we ingest and map all of the data uh that we're connected to and integrate with all of that data. In the middle of the stack is how we make sense of that data. And so we call that our production world model. How do we map out all the relationships between all the dent the data that we're ingesting? And then finally, how do we surface that and package that up in either use cases or user experiences that are valuable to your team? And so there's a lot of components to getting this right u especially at Fortune 500 scale.

第三个：它能不能把你数据里所有这些实体之间的关系梳理出来？就算你能把这些数据全都接进来、从头读一遍，要是不知道拿它怎么办、自己都迷失在里面，那基本也没什么价值。第四个：它会不会随着时间推移自主地变得更好、更聪明——而不需要你专门投入工程资源去维护一堆 markdown 文件，也不需要一帮 FDE（forward deployed engineer，前线部署工程师）占用你团队的时间来维护系统知识？最后一个：它能不能连跳多步？能不能在几分钟之内，找到那个并不简单、并不显而易见、离最初症状很远的根因？我们看到的情况是，只要超过 5 分钟，基本就没戏了：值班团队的耐心已经耗光，大家会退回到自己的老习惯里去。所以它必须非常好，也必须非常快。这部分我就不讲太细了，不过这大致就是我们在 Traversal 思考这五个问题的方式。技术栈的最底层，是我们怎么在不增加成本的前提下分析你所有的数据。说白了就是，我们怎么把接入的所有数据都摄取进来、做好映射，并且和这些数据全部打通集成。技术栈的中间层，是我们怎么去理解这些数据，我们把它叫做生产环境世界模型（production world model）：怎么把我们摄取进来的所有数据之间的全部关系梳理出来？最后一层，是我们怎么把这些呈现出来，包装成对你的团队有价值的使用场景或用户体验。所以要把这件事做对，涉及的组成部分非常多，尤其是在《财富》500 强这种规模上。

[17:32] Eric

And so um that's a bit about what we've been building at Traversal, what we're delivering at Traversal. The core pieces like I mentioned are the production world model. So how we map and make sense of all the data that we integrate with and then what we call uh what we call refer to as our causal search engine. Basically the the harness and mechanism that we give our agents to search through all of that data uh that we've mapped out. And so the production world model and the causal search engine are how we believe we are going to help Fortune 500 enterprises get to the level five of autonomy fully self-driving production.

以上就是我们在 Traversal 一直在构建的东西、正在交付的东西的一个大概。核心的几块，就像我刚才提到的，一是生产环境世界模型，也就是我们怎么对集成进来的所有数据做映射、做理解；二是我们称之为因果搜索引擎（causal search engine）的东西，说白了就是我们交给 agent 的那套 harness 和机制，让它们能在我们梳理好的所有数据里做搜索。我们相信，正是靠生产环境世界模型和因果搜索引擎，我们能帮《财富》500 强企业达到第五级（Level 5）自主：完全自动驾驶式的生产环境。

[18:12] Eric

Cool. Thank you for your time.

好了，谢谢大家的时间。

[18:30]

[music]

（音乐）