How We Built an Agent That Improves Itself — Zubin Aysola, Weights & Biases

频道: AI Engineer
视频: https://www.youtube.com/watch?v=XyV6bSMyq-I
原文语言: en
统计: 共 15 轮 · Zubin 14

[0:16] Zubin

Okay. Hello. Thank you for showing up. Uh my name is Zuben Isola. Uh I work at Weights and Biases. I'm building the Arya agent which is something that we released to general availability on Monday. You might have seen Tim's talk uh one of the pe my peers at Weights and Biases who I work with uh on the main stage yesterday. So I'm going to give you a little bit of a deep dive into how we evaluate and build the Arya agent uh and in particular uh the evaluation framework that we use and how we use Arya to sort of self-reinforce itself in the loop of doing research for itself uh and talk about some weights and biases things. So yeah, this is a rip from some of the slides that I presented at Nurups, but uh I think there's some interesting things about like using uh agentic harnesses. I I imagine most people in the audience or at this conference are all excited to build agent harnesses or maybe use cloud code with skills in a variety of different packages, but there's, you know, pretty strong differences in the way that you package these AI agents if you use them with different tool calls, etc. on a variety of different benchmarks. And so inspired by that, we're doing some work at Weights and Biases to build our own agent harness to do research for you in the weights and biases platform. Um, you know, and then there's probably some other things that I don't need to convince people of as to like why we want software agents and why a software harness as opposed to like just letting the, you know, LM run free and call tools or whatever else. Uh, but that's not for here. Uh, but one big problem presents itself, which is basically the thing that I go to bed thinking about every day, which is, you know, benchmarks, evaluations, the agents, and how you configure them are all covariant. And so like if you're trying to apply principled evaluations to any sort of system that dynamically changes, you need to have good measurement to see

好，大家好，谢谢各位来听。我叫 Zubin Aysola，在 Weights & Biases（W&B）工作，正在做 ARIA 这个 agent，我们周一刚把它正式发布（GA）。大家昨天可能看过 Tim 在主会场的演讲，他是我在 W&B 一起共事的同事之一。今天我想稍微深入地讲讲我们是怎么评测和构建 ARIA agent 的，尤其是我们用的那套 eval（评测）框架，以及我们怎么让 ARIA 在给自己做研究的循环里实现某种自我强化，顺带也聊聊 W&B 的一些东西。这部分内容是从我在 NeurIPS 上讲过的几页幻灯片里搬过来的，不过我觉得在使用 agentic harness（包在模型外面、管提示词/工具调用/循环的那层程序）这件事上，有些地方挺有意思。我猜在座的各位、或者说来参加这次大会的大多数人，都热衷于自己搭 agent harness，或者用 Claude Code 配上以各种形式打包的 skill。但你会发现，同样的 AI agent，封装方式不同、配的工具调用不同，放到各种 benchmark 上，表现差别是相当大的。受此启发，我们在 W&B 也在做自己的 agent harness，让它在 W&B 平台里替你做研究。还有些事我想就不用费劲说服大家了，比如为什么我们需要软件 agent、为什么要套一层软件 harness，而不是干脆让大模型放飞自我、想调什么工具就调什么。这些今天就不展开了。但有一个大问题摆在面前，基本上也是我每天睡前都在琢磨的事：benchmark、eval、agent 本身，还有你怎么配置它们，这几样全都是相互牵连、一起变的。所以如果你想给任何一个动态变化的系统做严谨的 eval，就得有好的度量手段，才能看清

[1:49] Zubin

how the system actually performs. And you really want to see how the system performs in both your production environment and your offline environment. Uh you know if you're RLP build from a long time ago, you might think about like the sim tore gap uh which is like basically how do we translate simulation environments into the real world environments etc. Uh and so there's sort of two things that I'll talk about uh and then give a live demo of which is weights and biases weave for agents which is basically in my opinion at least the best observability platform for both production and offline tracing of agents. So I use it predominantly from the offline side where I build simulation environments run the Arya agent over them and then track them in weave and my entire team does that as well. So I can see all of our performance in offline metrics. And then the other half of my team works on the production side of things, deploying the agent, logging the same things in the exact same format so that I can rip those production traces into our environments and then hill climb on them or resolve our errors. And it's a pretty nice flywheel. And then obviously the other thing that I'm here to talk about is Arya, the weights and devices agent itself, which is basically the thing that we use to now build itself because it's sophisticated enough that it can actually do that offline hill climbing uh by itself. So I'm going to just jump into a super quick live demo and then we'll talk through some other slides uh about how we built the offline evaluation framework. So if you hop into the weights and biases platform, this is a demo project that I called Arya researches Arya here. Uh and I'm just going to chat to Arya and ask it to basically do auto research for itself.

这个系统实际表现如何。而且你会希望同时看到它在生产环境和离线环境里的表现。如果你是早些年做过强化学习的人，可能会联想到 sim-to-real gap（仿真到现实的差距），说白了就是怎么把仿真环境里的东西迁移到真实世界的环境里，诸如此类。所以今天我主要讲两样东西，还会现场演示一下。第一样是 W&B 面向 agent 的 Weave，至少在我看来，它是目前做 agent trace（运行轨迹记录）最好的可观测性平台，生产环境和离线两边都能用。我主要是在离线这一侧用它：我搭仿真环境，让 ARIA agent 在上面跑，再在 Weave 里追踪这些运行，我们整个团队也都是这么干的，所以我能从离线指标里看到我们所有的表现。团队的另一半人负责生产这一侧：部署 agent，用一模一样的格式记录同样的东西，这样我就能把那些生产环境 trace 直接搬进我们的环境里，拿它们做爬山优化（hill climb，逐步往更好的方向优化），或者解决我们的错误。这是一个挺漂亮的飞轮。另一样我今天要讲的，当然就是 ARIA，也就是 W&B 的这个 agent 本身。现在我们基本上就是用它来构建它自己，因为它已经足够成熟，能自己完成那种离线爬山优化了。那我先直接来一段超快的现场演示，然后再过几页幻灯片，讲讲我们是怎么搭建这套离线 eval 框架的。进到 W&B 平台里，这是一个演示项目，我给它起名叫「ARIA researches ARIA」。我就直接跟 ARIA 聊，让它基本上给自己做一轮自动研究（autoresearch）。

[3:10] Zubin

And so this is a relatively long prompt that I had it generate in actually one of these other sessions. Um but if we look at it, what it's going to do is it's going to take my codebase that is logged as a weights and biases artifact uh which we can see over here. It's going to launch training jobs uh against this codebase which is our offline evaluation framework. It's going to review production traces that we have, add new tasks to hill climb against uh and do stuff like that. And so, you know, this thing's going to cook for a little bit. The evaluations take a little bit of time to run. And so, we'll see Arya uh basically try to write a new variant for itself. The other side of this equation, right, is the production side of things. And so, this is a sanitized view of internal customer traces or internal traces on Arya. So we'll actually see very quickly uh that Arya will actually pop up here the exact conversation that we're running um right here. But in this trace I'm going to use this as an opportunity to take one of these production traces that are logged in weave and put them into my offline evaluation framework so that I can hill climb on them. And so I'm just going to do that live. We'll ask you know for a relatively simple task. Um this is going to be a little interesting because it's going to be slightly self-referential but I'll go here. I'll pop up in a new Arya tab and I'll just ask it hey take this trace log it into our offline evals I don't think spelling matters too much and then run the candidate and production agent on it and we'll have it cook for that as well and we'll see those evaluations get logged into weave as they're running um so that's basically the tight version of what I do basically every day which is look at production traces in our weights and biases weave platform run evaluations in our codebase which is our offline client evaluation sandbox uh and then deploy new versions of the agent uh

这段 prompt 比较长，其实是我在另一个会话里让它生成的。不过我们来看看它要做什么：它会拿我的代码库——这个代码库作为一个 W&B artifact 记录在案，就在这边能看到——然后针对这个代码库启动训练任务，这个代码库也就是我们的离线 eval 框架。它还会复盘我们手上的生产环境 trace，加一些新任务作为爬山目标，诸如此类。所以这东西得跑上一阵子，eval 跑起来需要点时间。待会儿我们就能看到 ARIA 基本上在尝试给自己写一个新的变体。这个等式的另一边是生产这一侧。这里是内部客户 trace 的脱敏视图，或者说是 ARIA 的内部 trace。其实很快我们就会看到，ARIA 会把我们现在正在跑的这段对话原样显示在这里。不过我想借着这条 trace 顺便演示一下：把记录在 Weave 里的某条生产环境 trace 放进我的离线 eval 框架，好拿它做爬山优化。那我就现场来做。我们让它做个相对简单的任务。这会有点意思，因为多少有点自我指涉，不过我就这么来吧。我新开一个 ARIA 标签页，直接问它：嘿，把这条 trace 记进我们的离线 eval 里——拼错几个字应该无所谓——然后让候选版本和生产版 agent 都在上面跑一遍。这个也让它跑一会儿，eval 边跑边会记录进 Weave，我们都能看到。这基本就是我每天日常工作的浓缩版：在 W&B 的 Weave 平台上看生产环境 trace，在我们的代码库——也就是我们的离线 eval 沙箱——里跑 eval，然后部署新版本的 agent

[4:47] Zubin

and work with my team to do that. And if we look at our production project, this is, you know, a bunch of traces that we have over the last seven weeks of our nightly uh like CI jobs that run where we evaluate the agent in its production format and other candidate variants that we cut. And so we can see our relative performance over time. Uh like you know, for a while the CI broke, we had Arya have to fix itself last night, but you know, we're getting about like 66% performance on some of the tasks, etc. And so we have a variety of tasks um that we train the agent to do. And so now I'm just going to jump into a material version of the presentation to go a little bit deeper into the theory behind building a simulation environment for offline hill climbing for basically building a software agent where traditionally you might expect that you are really robust if you're doing reinforcement learning but I mostly prompt engineer these days because the sophisticated models are relatively good at performing tasks at weights and biases and so we're really working on building skills for the software agent more than doing reinforcement learning. But I think applying the same methodology to how you build the agent.

并和我的团队一起把这些事做下来。再看我们的生产项目，这里是过去七周里我们每晚跑的 CI 任务积累下来的一大批 trace，这些任务会评测生产形态的 agent，以及我们切出来的其他候选变体。所以我们能看到相对表现随时间的变化。比如有一阵子 CI 挂了，昨晚还得让 ARIA 自己把自己修好，不过总体来说，我们在部分任务上大概能拿到 66% 左右的表现。我们训练 agent 去做的任务有很多种。好，现在我切到正式的幻灯片部分，更深入地讲讲背后的理论：怎么为离线爬山优化搭建仿真环境，说到底就是为了构建一个软件 agent。传统上你可能会预期，做强化学习的话，这套东西会搞得非常稳健；但我现在主要是在做 prompt 工程，因为现在这些强大的模型在 W&B 的任务上已经做得相对不错了，所以我们主要是在给这个软件 agent 构建 skill，而不是做强化学习。但我觉得，把同样的方法论用到构建 agent 上

[5:40] Zubin

So the same robustness of how you simulate environments uh is really helpful for our work. So we benchmark a bite-wise identical version of the agent in our production environment and our simulated environment. There's a lot of text on these slides that I and Claude sort of slop out. Uh but you know we'll just go for the graphics. Um so I think the exciting part here is that both using the logging framework from weave as well as just in the way that we've designed the system our research and production code are exactly the same. And in fact, there's like a 4-hour sync job that happens between production to our research environment so that we don't get any drift when you, you know, researchers are cutting new variants of the agent, new skills, etc., and doing their hill climbing. And so, we have this nice little tight loop that basically mirror each other on two sides of the stack from our deployment layer and our offline benchmarking layer. Uh, and then, you know, we expose internally as a team a bunch of run eval commands to generate score trajectories. And so that's what we were talking about with this like large project where we have a bunch of trajectory examples where if I click into one of these, you know, we can see like our relative scores of a variety of different metrics when it loads. Um, but essentially what we get is these super robust traces of the agent rolling out over time. And there's a lot of signal to reap from this. And I think the thesis behind this is generate a ton of traces and then decide what you're going to do with said ton of traces as you try to figure out how the agent performs. you're measuring emergent properties and then trying to align the agent in particular directions. And so the two ways that that happens is I look manually at the tasks or like we would do in that first example, I would ask Arya to review the roll out that it generated for itself or

也就是用同样稳健的方式去模拟环境，对我们的工作非常有帮助。所以我们会在生产环境和仿真环境里，对逐字节一致（byte-wise identical）的同一版 agent 做 benchmark。这些幻灯片上字很多，是我和 Claude 一起随手糊出来的，我们就只看图吧。我觉得这里最让人兴奋的一点是：一方面借助 Weave 的日志框架，另一方面也得益于我们设计系统的方式，我们的研究代码和生产代码是完全一样的。实际上，从生产环境到研究环境还有个每 4 小时跑一次的同步任务，这样研究员在切 agent 的新变体、写新 skill 之类、做爬山优化的时候，就不会出现任何漂移。于是我们有了一个很紧凑的小循环，技术栈的两侧——部署层和离线 benchmark 层——基本互为镜像。然后我们在团队内部开放了一堆 run eval 命令，用来生成分数轨迹。刚才那个大项目说的就是这个，里面有一大堆轨迹样本，我点开其中一个，等它加载出来，就能看到我们在各种不同指标上的相对得分。但本质上，我们拿到的是 agent 随时间一次次运行留下的非常扎实的 trace，从里面能挖出大量信号。我觉得这背后的核心思路是：先生成海量 trace，然后在搞清楚 agent 表现如何的过程中，再决定拿这些 trace 做什么。你在度量的是涌现出来的特性，然后试着把 agent 往特定方向上对齐。具体有两种做法：一种是我手动去看这些任务；另一种就像刚才第一个例子那样，我会让 ARIA 复盘它给自己跑出来的那次 rollout（一次完整运行），或者

[7:06] Zubin

review some other rollout and decide what went wrong or what went well and try to reinforce that behavior through prompting or anything else. Uh and then you know the actual method behind the agent is relatively simple. We want to test on a variety of the different models both models offered through corewave inference models offered through you know the foundation model players etc. And so we define you know a relatively agnostic software stack for how we treat compaction and how we prepare context and how we assemble UI payloads etc. And the again mental pattern behind this is to really try to make it very very simple to have lots of mutations of the exact same configuration. You want to basically YAML define different configurations of the agent to sort of get multiple parallel uh variants and then test them all and see what happens because ultimately you can have insight into how to make the problem better. But if we take an adage from old reinforcement learning training or just simple model training, it's just better to run more experiments than fewer. And so that's the thesis behind the agent harness itself too. And so then you know we talk about like the sandbox stack behind the agent. And I think that you know there's a lot of mentions of code mode here at this conference and other things like that. And one thing that we really want to see happen because we want to build the best product for our customers is to just have a sandbox environment that they can do anything they wanted. So for example prior to yesterday I wasn't entirely certain that Arya was going to be able to do a bunch of parallel executions of itself. And so I asked it both while I was writing this presentation and preparing for this to just create the sandbox environment uh for itself to run a bunch of parallel executions of its own research loop.

复盘别的某次 rollout，判断哪里出了问题、哪里做得好，再通过 prompt 或者别的手段去强化这些行为。至于这个 agent 背后的实际方法，其实相对简单。我们想在各种不同的模型上做测试，既有通过 CoreWeave（W&B 的母公司）推理服务提供的模型，也有各家基础模型厂商提供的模型，等等。所以我们定义了一套相对不绑定具体模型的软件栈，规定怎么处理上下文压缩（compaction）、怎么准备上下文、怎么组装 UI payload 等等。这背后的思路还是那个：要让同一套配置衍生出大量变异版本这件事变得非常非常简单。基本上你就是想用 YAML 定义 agent 的不同配置，从而得到多个并行的变体，然后全部测一遍，看看会发生什么，因为最终你总能从中看出怎么把问题做得更好。不过借用老派强化学习训练、或者说普通模型训练里的一句老话：实验多跑总比少跑好。这也正是 agent harness 本身背后的核心思路。接下来聊聊 agent 背后的沙箱技术栈。我觉得这次大会上很多人都提到了 code mode 之类的东西。而我们真正希望实现的一件事——因为我们想给客户做出最好的产品——就是给他们一个想干什么都行的沙箱环境。比如说，直到昨天之前，我其实都不太确定 ARIA 能不能并行跑一大堆它自己的实例。所以在写这份演讲稿、为今天做准备的时候，我就让它自己去搭一个沙箱环境，用来并行跑一大堆它自己的研究循环。

[8:36] Zubin

Right. Uh, and so like having that unconstrained environment is really useful for getting the agent to do emergent things. And so, you know, I I'm not really sure what the six phrases per record uh means, but I think from a theory standpoint, we want to build then a good pattern for generating these simulation environments. And so we have a relatively agnostic DAG that you might think about in a traditional machine learning context, which is like we take a configuration, which are these YAML files. We hydrate them. So we load the live data that we need to. We set up an environment and the environments and weights and biases are relatively expensive because we have lots of production data that we might want to test against or other things like that. These are full machine learning training logs. We might have to simulate uh GPU executions if you're doing auto research etc. That's a relatively intensive process and so you might want to parallelize that. Then you would rehydrate because oftentimes there's runtime configurations that you can't encode in a YAML specification and so you hot patch that data back into the config config. Uh and then you run the agent and running the agent is trivial.

对。所以，有这样一个不设限的环境，对于让 agent 做出一些涌现式的行为非常有用。嗯，我其实也不太确定这个「six phrases per record」是什么意思，但从理论上讲，我们接下来想要的是一套生成这些仿真环境的好模式。所以我们有一个相对通用的 DAG（有向无环图），你可以放到传统机器学习的语境里去理解：先拿一份配置，也就是这些 YAML 文件；然后 hydrate（填充数据），也就是把需要的实时数据加载进来；接着搭环境——在 Weights & Biases，搭这些环境的成本相对较高，因为我们有大量生产数据可能要拿来做测试，诸如此类。这些可都是完整的机器学习训练日志。如果你在做自动研究（autoresearch）之类的事，我们可能还得模拟 GPU 执行。这个过程相当吃资源，所以你可能会想把它并行化。之后还要再 rehydrate（二次填充）一遍，因为很多时候有些运行时配置没法写进 YAML 规格里，所以你得把那些数据以热补丁（hot patch）的方式打回配置里。然后就是跑 agent，而跑 agent 本身是最简单的一步。

[9:33] Zubin

You run it, as I mentioned at the beginning, in the bite-wise identical version that you have in production, and then you score it. And scoring is where I spend a lot of my time, which is basically trying to think about the robustness of how you measure results. Uh, and I imagine most people here are pretty good at thinking about things like that. And I think it's really worth spending the time once you have a configuration of how to run evaluations, thinking about the robustness of our emails. uh a Slack message that I probably can't show live on the screen that I got from one of my teammates today made me very happy where he basically just spent all of yesterday thinking about the health of our evaluations and the drift between our evaluations and production. Uh and I think that kind of mental model of like you know why are things working well why are things not working well? What are the gaps that we see in these two patterns is really useful. Um in our case weights and biases Arya scores itself in two patterns which is normatively which gives us basically did we pass a task or not and then relativistically where we can set styles based on one variant where it asks questions to the user and one variant where it doesn't ask questions to the user and we can sort of see which one behaves better uh with a relative scoring. So a pretty traditional formulation from a reinforcement learning standpoint. And then of course if you're running this in parallel you got to tear it down because you want to start it again and you don't want to clobber your teammates's work etc. Uh and so we'll see what happens in this project because a bunch of my teammates are running eval right now and so we'll see how Arya performs.

就像我开头提到的，你用和生产环境逐字节一致的版本去跑它，然后给它打分。打分是我花时间最多的地方，基本上就是在琢磨：你衡量结果的方式到底够不够稳健。我想在座大多数人都很擅长思考这类问题。而且我觉得，一旦你把跑评测的配置定下来，真的很值得花时间想一想我们的 eval 够不够稳健。今天有个队友给我发了一条 Slack 消息——大概没法直接投到屏幕上给大家看——让我特别开心：他基本上把昨天一整天都花在了琢磨我们评测的健康状况，以及评测和生产环境之间的漂移上。我觉得这种思维模式非常有用：事情为什么顺利、为什么不顺利？这两者之间我们看到了哪些差距？在我们这里，W&B 的 ARIA 用两种方式给自己打分：一种是规范性打分，基本上就是告诉我们任务过没过；另一种是相对打分，比如我们可以设定不同的风格，一个变体会向用户提问，另一个变体不向用户提问，然后通过相对打分看看哪个表现更好。从强化学习的角度看，这是相当传统的一种设定。当然，如果你是并行跑的，跑完就得把环境拆掉，因为你还要重新开始，也不想把队友的工作给覆盖掉之类的。所以我们等会儿看看这个项目里会发生什么——我好几个队友此刻正在跑 eval，我们就看看 ARIA 表现如何吧。

[10:47] Zubin

Uh and then you know more traditionally on tasks I think the eval tasks are just YAML specifications that we define as a starting condition of an environment with a bunch of user configurations as well as then an ending condition that we sort of want to get to. Uh in our case tasks are flows from users. So we simulate them in three ways. Uh we do just simple text which is like hey here's an instruction to Arya. You might imagine that the task that it generates in this chat over here um where it's logging a specific trace from production uh into our local framework actually looks like you know just a simple question that I asked it basically which was that question of Arya doing auto research itself um we can just ask it to continue um as it does that and that task is just a single question to it there's other variants of the tasks that exist as well where we simulate a language model that has a persona from a user and we ask it, hey, pretend to be this user, ask certain questions in a particular order so that we can simulate multi-turn environments and interactions with our agent. Uh, and then you just run a bunch of these. So, we have 886 tasks. We categorize them by levels. We expose them to our product team so that they can decide whether or not the tasks are good enough or they reflect things that we care about from our benchmarks, etc.

然后，任务这块就比较传统了。我们的 eval 任务就是一份份 YAML 规格：定义一个环境的起始条件，外加一堆用户配置，再加上一个我们希望最终达到的结束条件。在我们这里，任务就是用户的使用流程。我们用三种方式来模拟它们。一种就是简单的文本，比如「嘿，这是给 ARIA 的一条指令」。你可以想象一下，它在这边这个对话里生成的那个任务——就是把生产环境里一条特定的 trace 记录进我们本地框架的那个——实际看起来就只是我问它的一个简单问题，基本上就是让 ARIA 给自己做自动研究的那个问题。它做的过程中，我们可以直接让它继续。那个任务对它来说就只是一个单独的问题。任务还有别的变体：我们会模拟一个带着某位用户人设（persona）的语言模型，跟它说「嘿，假装你是这个用户，按特定顺序问某些问题」，这样我们就能模拟多轮的环境，以及用户和我们 agent 之间的交互。然后就是把这些跑上一大堆。我们一共有 886 个任务，按级别分类，还开放给产品团队，让他们来判断这些任务够不够好、有没有反映我们在 benchmark 里关心的东西，等等。

[11:54] Zubin

And then we just run them a bunch of times. Yeah. And then, you know, the ultimate thing here is that the trajectory is like the meat and the data that I live behind. And so I look at our production traces that exist in our traffic and our offline traces that exist. And I really try to exploit the behavior patterns from both of those. And so we build tooling ourselves to understand our traces. Uh we have like this behavior trace project etc that we run in both prod and offline. And yeah that's the eval flywheel. So you know fundamentally every production miss or every production goodness as well because I think it's useful to hill climb in a positive direction uh becomes a task for our agent framework. And so we use that to sort of define the agent and make it better. And these are, you know, our real scores straight from weave. So we have a bunch of different categories that we measure ourselves on. I think the agent is really good at conceptually guiding you. We want to make it better at doing some error analysis for projects. You know, we try to make these as difficult as possible for the agent to really test the metal of how this thing performs. And so I'll hop back to Arya now.

然后我们就把它们跑上很多遍。对。说到底，这里最核心的是轨迹（trajectory）——那才是真正的干货，也是我天天泡在里面的数据。所以我会去看我们流量里的生产环境 trace，也会看我们的离线 trace，并且真的会想办法把这两边的行为模式都利用起来。为此我们自己做了一些工具来理解我们的 trace，比如这个 behavior trace 项目之类的，在生产和离线两边都在跑。对，这就是 eval 飞轮。所以从根本上说，生产环境里的每一次失手——还有每一次表现好的地方，因为我觉得往正向做爬山优化也很有用——都会变成我们 agent 框架里的一个任务。我们就靠这些来定义这个 agent、让它变得更好。这些是直接从 Weave 里拿出来的真实分数。我们在一大堆不同的类别上衡量自己。我觉得这个 agent 很擅长在概念层面给你引导；我们希望它在给项目做错误分析这块做得更好。我们会尽量把这些任务设计得越难越好，真正考验一下这东西的成色。好，那我现在切回 ARIA。

[12:52] Zubin

There's sort of five minutes left um with sort of a demo of what it's actually doing. So right now it's going to run a bunch of code as it's trying to rip this production trace and log it into the framework. If I hop into this uh live demo that I just triggered with that nice long prompt, you know, Arya is going to tell me the research runs that it did. So we can click through some of these links. Uh I asked it to write a report using weights and biases reports from the uh run that it just had. If we scrub this, we can see that this is the most recent production trace that it just logged. It wrote itself a new task, ran it, and then scored it. And so this is like the production variant getting benchmarked on our specific task. If we look at the report that it generated, this is the demo that I asked it to do. Um, so you know, it ran a real womb agent trace, turned it into a WBF regression task. WBAF is the factory that I build, which is the weights and biases agent factory. That's the principle behind our offline benchmarking. It identified that the problem was that we weren't calling weave.log, one of our SDK calls properly in the sandbox. It replicates the source trace, which we can click into here, uh, and then runs a bunch of variants of the agent to see how it makes itself better.

还剩五分钟左右，我们来演示一下它实际在做什么。它现在正在跑一堆代码，想把这条生产环境 trace 扒下来、记录进框架里。我切到刚才用那条长长的 prompt 触发的这个现场演示，ARIA 会告诉我它跑了哪些研究 run，我们可以点开其中一些链接看看。我让它用 W&B Reports，把它刚才那次 run 写成一份报告。拖一下这里，可以看到这是它刚刚记录进来的最新一条生产环境 trace。它给自己写了一个新任务，跑了一遍，然后打了分。所以这就相当于生产变体在我们这个特定任务上做了一次 benchmark。再看看它生成的报告，这就是我让它做的演示：它拿一条真实的 agent trace，做成了一个 WBAF 回归任务。WBAF 就是我搭的那个工厂，即 Weights & Biases Agent Factory，也是我们离线基准测试背后的核心。它找出了问题所在：我们在沙箱里没有正确调用 weave.log——这是我们的一个 SDK 调用。它复现了源 trace（可以在这里点进去看），然后跑了一堆 agent 变体，看看怎么让自己变得更好。

[13:50] Zubin

And so, you know, it added a hill climb target uh which was basically like here's what to do to fix it. Ran the trace and then we can see if it gives us a little explanation of exactly what happened at the end. Uh how did the prod versus candidate variant perform and you know this is now what I live in. And so I think the flywheel here is exceptionally compelling for me where instead of going back to my cloud code and writing offline benchmarks and trying to think about what I want the agent to do, I'm just going to live in this platform instead and hop between my production tracing project where I'm looking at traces roll in as they come through over time where AR is, you know, building itself and then hop back into my offline evaluation project to sort of see what my team is doing, what I'm doing, and what changes I want to make. what change did we make to the prod skill or to the agent variant etc. And you know to sort of like watch this thing roll out. I think a big challenge that we have is it's really easy to go like auto mode for some of these tasks where you want to just see the agent do everything. And I haven't written a line of code in maybe eight months because I just tell Claude to write all my code for me. And that's a really nice pattern, but it doesn't absolve yourself of the thought behind how we want to make improvements. And I think using these tools to improve themselves is very valuable because you get to spend more of your time in the gray of actually trying to think how we make this system better, right? And so if you're building an agent, I implore you to use weights and biases to both track that agent etc. Uh we have a really excellent model tracking platform as well. So some of the other demos that we've shown here are Arya training machine learning models on H200's running on core infrastructure and doing auto research for Karpathy's nano chat etc or full-scale production projects.

于是它加了一个爬山目标（hill climb，逐步优化的方向），基本上就是「要修好它，该这么做」。它跑了这条 trace，然后我们可以看看它最后有没有给出一小段说明，讲清楚到底发生了什么：生产变体和候选变体各自表现如何。这就是我现在天天泡着的地方。所以我觉得这个飞轮对我来说格外有吸引力：我不用再回到我的 Claude Code 里写离线 benchmark、琢磨我想让 agent 干什么，而是直接待在这个平台上——在我的生产环境 tracing 项目里，看着 trace 随时间一条条涌进来，ARIA 就在那里构建它自己；然后再切回我的离线评测项目，看看团队在做什么、我在做什么、我想做哪些改动，我们对生产环境的 skill 或 agent 变体改了什么，等等。然后就看着这一切慢慢铺开。我觉得我们面临的一大挑战是：做某些任务时，你很容易直接进入 auto mode（自动模式），就想看 agent 把所有事都包办了。我大概有八个月没亲手写过一行代码了，因为我所有的代码都直接让 Claude 写。这种模式确实很舒服，但它并不能免除你去思考「我们到底想怎么改进」。我觉得用这些工具来改进它们自己非常有价值，因为你能把更多时间花在那片灰色地带里，真正去想怎么让这个系统变得更好，对吧？所以如果你在做 agent，我强烈建议你用 W&B 来追踪这个 agent，等等。我们还有一个非常出色的模型追踪平台。我们在这里展示过的其他 demo，还有 ARIA 用核心基础设施上的 H200 训练机器学习模型、给 Karpathy 的 nanochat 做自动研究，以及一些完整规模的生产级项目。

[15:25] Zubin

But I think the exciting thing in this case for me is that I don't need to leave this platform in order to do my job now right. I have full observability from a production standpoint. I have full observability from an offline standpoint. And if we look at, you know, the things that Arya is saying, it's running lots of evaluations on my behalf to improve itself. And that auto mode that I just mentioned is really exciting because now I get to spend all of my time thinking about how I can make this system reinforce itself better. The tight little guard rails that I can put around it to make it a more useful tool for all of you and then inherently for me. Um, but yeah, that's a that's a quick demo. We can we can look at the production call trace that we have here from Arya. This is the evaluation that it just ran. So if we look into like the actual trace, we'll see that, you know, AR did a bunch of work. It ran this predict and score column. We have a bunch of tool calls that ran. Uh and you know, if we hop back into the chat, we'll see the actual variant that it ran. Um it looks like the candidate variant just had a tight little prompt that we injected into the system prompt or in one of the skills to sort of solve that exact SDK error uh and mitigate it.

但对我来说，这件事最让人兴奋的是：我现在干活完全不用离开这个平台了，对吧。生产环境这边我有完整的可观测性，离线这边我也有完整的可观测性。再看看 ARIA 在说什么：它正替我跑大量评测，来改进它自己。我刚才提到的那种 auto mode 真的很让人兴奋，因为现在我可以把所有时间都花在思考怎么让这个系统更好地自我强化上——想想我能给它套上哪些贴身的小护栏，让它对你们所有人、进而对我自己，都成为更有用的工具。嗯，不过就这样，这就是一个快速演示。我们可以看看这里 ARIA 的生产环境调用 trace。这是它刚跑完的评测。如果点进实际的 trace，会看到 ARIA 做了一大堆工作：它跑了这一列 predict and score，还有一堆跑过的工具调用。再切回聊天界面，就能看到它实际跑的那个变体。看起来候选变体只是往 system prompt 或者某个 skill 里注入了一小段精炼的 prompt，专门用来解决、缓解那个 SDK 报错。

[16:21] Zubin

And that kind of replication from production to simulation to agent then defining the improvement pattern is the thing that I really would like to leave everybody with. So that's coreweave Arya. Uh welcome to weights and biases. I hope that was an interesting talk in sort of how we decide to build uh our offline evaluation metrics, how we reap data from production and the flywheel of trying to build these agents and the things that I think about on a day-to-day basis. Uh and yeah, let's get started. Thank you all so much.

这种从生产环境复现到仿真环境、再由 agent 自己定出改进模式的链路，就是我最想留给大家的东西。以上就是 CoreWeave 的 ARIA。欢迎来到 Weights & Biases。希望这场分享还算有意思：讲了我们是怎么决定搭建离线评测指标的、怎么从生产环境收割数据、打造这些 agent 的飞轮，以及我每天都在琢磨的那些事。好，那我们开始吧。非常感谢大家。

[17:04]

[music]

（音乐）