Evaluating the Fine-Grained Planning Abilities of Web Agents

频道: YC Root Access
视频: https://www.youtube.com/watch?v=6f4K7-DvJrk
原文语言: en
统计: 共 22 轮 · Ankit 11 · Surgan 10

[0:03]

[music]

[音乐]

[0:07] Ankit

We're here with Surgan at YCML. Surgan's a master student at CMU and just presented this work at EMNLP last year. Surgan, tell us a little bit about what you worked on.

我们今天在 YCML 现场请到了 Surgan。Surgan 是 CMU 的硕士生，去年在 EMNLP 上发表了这项工作。Surgan，跟我们聊聊你做的东西吧。

[0:16] Surgan

Yeah, thanks a lot. Yeah, my work is titled on the fine-grained planning abilities of VLM agents. So, a little bit about web agents. Um So, as the web agents execute their task, for instance, you know, find this cheapest 4K monitor, they get input, you know, of UI screen. And then they first think about how to solve this task. Which is like, you know, coming up with plans, strategy, and so on. That's called planning state. And based off of this thought, they predict a very specific interaction that they have to do on this particular screen. They keep doing this for like, you know, five steps, 10 steps, or whatever is the limit and produce a final output. Um and then we have some rubrics to evaluate that, some verifiers, or whatever the approach we want to take. So, evaluating the final output, in my opinion, is slightly easier. Uh so, I'm I'm concerned with what happens to evaluating this plan, this step.

谢谢。我这篇工作讲的是 web agent 的 fine-grained planning（细粒度规划）能力。先说说 web agent 是怎么回事：agent 执行任务的时候，比如「找到最便宜的 4K 显示器」，它拿到的输入是一张 UI 界面，然后它会先想这个任务该怎么解——也就是给出计划、策略之类的东西，这一步叫 planning（规划）。基于这个 thought，它再预测出在当前这个界面上具体要做哪一个交互动作。这个循环它会反复走五步、十步，或者走到设定的上限为止，最后产出一个结果。然后我们再用一些 rubric、verifier 之类的东西去评估这个结果，具体用哪种方法都行。在我看来，评估最终输出这件事相对容易一些。我真正关心的是：中间这个 plan、这一步，该怎么评。

[1:04] Ankit

So, you're saying this is like an off-the-shelf VLM web agent does this type of step, and you're trying to evaluate this middle planning step.

所以你的意思是，一个现成的 VLM web agent 就是这么一步步走的，而你想评的是中间这个 planning 环节。

[1:13] Surgan

Oh, so So, let's think uh what would an approach to evaluate this would be. Let's say you want to evaluate the planning step. Since the plan is generated with the past context, you would need that. And the plan is about what you would do in the future. So, you would also need that output. Therefore, for this particular guy, you actually need the whole thing. Right? And this thing, it's like 10 15 of those. And the context is not cheap. Because for web agents, you actually need like an image, a bunch of tool calls, and a bunch of different, you know, text. In addition, uh the models are slightly getting better, and they don't make obvious errors. So, they are will be subtle. So, therefore, finding a problem with this thing would be like a needle in the haystack problem, right? So, what we are saying is evaluating each of the plan part to evaluate the planning skill is a great way you should do it. You know, you should run the traces, get this right, and comment on planning. But, we should also focus and figure out alternatives, complementary things. Let's take an analogy from visual reasoning. Let's say there is a task, visual question answering task. What is the animal to the left of the lion?

对。那我们想想，要评这一步会怎么做。假设你要评 planning 这一步：计划是基于此前的上下文生成的，所以那些上下文你得有；而计划讲的又是接下来要干什么，所以后面的输出你也得有。也就是说，光为了评这么一个点，你实际上得把整条轨迹都搬上来。而这样的点，一条轨迹里有十个、十五个。上下文本身还不便宜——web agent 的上下文里有图像、一堆 tool call，还有各种文本。另外，模型现在越来越好了，不太犯那种一眼就能看出来的错，错误会很微妙。所以在这么长的东西里找问题，就成了大海捞针。我们想说的是：逐段去评每一部分 plan、以此来评 planning 能力，这当然是好办法，你应该跑轨迹、把这件事做对、再对 planning 下判断。但我们也该去找一些替代的、互补的做法。可以借视觉推理打个比方。比如有一个视觉问答（VQA）任务：狮子左边的那只动物是什么？

[2:17] Ankit

Yeah.

嗯。

[2:17] Surgan

It seems like a single task to all of those of us, but it's not that. I mean, small is first going to ground the lion, you know, understand what's to the left and right. Locate the animal, and then come up with the right answer.

对我们所有人来说这好像就是一个任务，其实不是。模型得先把狮子定位（ground）出来，再搞清楚左边右边分别是哪边，然后找到那只动物，最后才给出正确答案。

[2:27] Ankit

And our brains are doing all of these things, too.

我们的大脑其实也在做这几件事。

[2:28] Surgan

Yeah, all of these things, yeah. So, and we have to get all of that right to get this right, right? So, I'm thinking how do I break down planning, which is the ability to come up with thoughts to solve this action, into these fine-grained skills, and put a number to the capability of the model for each of these skills, and then comment on the planning ability.

对，都在做这些事。而且这几步全做对了，最后那个答案才对，对吧？所以我在想的就是：能不能把 planning——也就是「想出该怎么做这个动作」的能力——拆成这些 fine-grained 的单项技能，给模型在每一项技能上的能力打一个分，然后再来评价它的 planning 能力。

[2:47] Ankit

Yeah.

嗯。

[2:47] Surgan

So, to discover these skills, we have like a semi-automatic way. We run it for a bunch of, you know, task C task. Um, yeah, for a bunch of C task, and then we discover a bunch of these things. So, what we figure out is agent is always trying to think about if it is going in the right direction. For instance, if it's an add to product, it's like, "Oh, I have to buy this product. Should the next state be checkout?" It's like, "Oh, does checkout come before add to product?" So, it's always sort of doing the temporal ordering of the web page states. Second, whenever it's at this add to product, and it knows like, "I want to do checkout." It's always thinking about, "Oh, what will help me go from here to here?" So, it's like predicting the future, and predicting how to go to future. So, the skill would be future state prediction. And often, agent is not just doing, you know, taking deterministic one action. It's like it has a bunch of options. It's trying to evaluate these options, and then trying to figure out which is the best. So, it has multiple plan choices, action choices, and then selecting the one.

为了找出这些技能，我们用了一套半自动的方法。我们在一批种子任务上跑，从中归纳出这么几项。我们发现，agent 一直在判断自己是不是走在正确的方向上。比如任务是把商品加入购物车，它会想「我要买这个东西，那下一个状态该是结算页吗？结算是排在加购前面吗？」——这其实就是在给网页状态做 temporal ordering（时序排列）。第二，当它停在加购这一步、又知道自己接下来要去结算，它一直在想「什么动作能把我从这里带到那里？」——这就是在预测未来状态、以及预测怎么走到那个未来，所以这项技能叫 future state prediction（未来状态预测）。还有，agent 往往不是只有一个确定动作可走，它面前有一堆选项，它要去评估这些选项、挑出最好的那个——也就是在多个 plan 选项、动作选项里做选择，这就是 action selection（动作选择）。

[3:43] Ankit

And and how are you implicitly assessing, let's say plan assessment? How do you measure the agent's ability to

那你具体是怎么去隐式地衡量——比如说 plan assessment（计划评估）？你怎么量化 agent 的这个能力……

[3:48] Surgan

Yeah, so this portion discovers the skill agents is deploying and we isolate them. In the next one, we operationalize these skill discovery, actually evaluate their, you know, capability on that skill. We create some simple tests which evaluate

对，前面这一块是把 agent 在用的技能找出来、把它们隔离开。下一步我们把这些找出来的技能操作化，真正去评它在这项技能上的能力。我们会造一些很简单的测试来评……

[4:05] Ankit

Like you create some counterfactuals to determine if

就像是造一些反事实（counterfactual）来判断……

[4:07] Surgan

Yeah, yeah. Very simple test, yeah. This is just telling you what skills they are deploying and you should make the skill level evaluation. And once they can get wrong, the other skill they are also deploying is error correction. Now, to your question, how am I making this evaluation very useful? So, couple of constraints. I want to make this cheap because if it is as expensive as possible, then it's no use. And I want to make it more interpretable. You know, getting a good number for each of this is very important so that I can give you another, you know, detail in your evaluation. So, what I do is we take existing data sets. It's like totally, you know, repurposing the existing data sets. We don't sort of generate new data. We don't have any human involvement. It's just having a very easy way to generate synthetic data. For instance, uh for temporal ordering, like whether this webpage comes before or not, you take two screenshots, like Google Scholar homepage and the final task. And then you're like, "Okay, if you want to solve this, which of the following come will come first?" So, agent will answer this question.

对对，非常简单的测试。前面那部分只是告诉你它在用哪些技能、告诉你应该做技能层面的评测。另外，一旦它做错了，它还会用到的一项技能是 error correction（纠错）。回到你的问题：我怎么让这套评测真的有用？有两个约束。第一，我要它便宜——如果它跟原来那套一样贵，那就没意义了。第二，我要它更可解释——每一项都拿到一个清楚的分数，这很重要，这样我才能在你的评测里多给你一层信息。所以我的做法是：拿现成的数据集来改造，完全是把已有数据集重新利用，我们不新造数据，也不需要任何人工参与，就是用一种很省事的方式生成 synthetic data（合成数据）。举个例子，temporal ordering——判断这个网页是不是排在前面——你就取两张截图，比如 Google Scholar 的首页和任务终态那一张，然后问：「要完成这个任务，下面这两张哪一张会先出现？」让 agent 来答这道题。

[5:06] Ankit

Makes sense.

有道理。

[5:07] Surgan

This will evaluate this skill and for all of them you will have these type of questions. All of them combined will give you a number for each of these skills which will correlate to the final planning capability. So, it's different. Earlier you were evaluating that

这道题就把这项技能评了；每一项技能都有这类题目。把它们合起来，你就能给每项技能拿到一个分数，而这些分数是和最终的 planning 能力相关的。所以这跟之前不一样了——之前你评的是……

[5:20] Ankit

exact parts?

……具体的那几段？

[5:21] Surgan

Yeah. Now, it's a generic skill.

对。现在评的是通用的技能。

[5:23] Ankit

And so, what does that look like in terms of your ultimate results? Like how how should we think about how well this works?

那最终结果长什么样？我们该怎么看这套方法的效果？

[5:27] Surgan

So, first result we have is uh these are like simple tests. So, the average of all, most of the models are bad, like below 50%. So, ideally we would assume these are like simple tests, right? I mean, these are open source because of computer constraints, but again, even them being less than 50% is a very bad result, very bad performance for them. So, it's like we really need to improve on each of these dimensions. Second, we see that the average performance of these planning is very correlated to the end task performance. Models being good on these skills is actually correlated to the end task performance. So, now what we are saying is you should definitely run the models on your tasks, get the traces, evaluate the thought, but it's like an offline test. You will take my simple test, put it on your model like for $10, get a preliminary evaluation on these independent dimension, and you can filter out a lot of them. And then you should run and do this complex test. So, now it's also like it's it's like you know where it is wrong. If this is wrong, then first do a GRP or what you want to do, and then you get to do this complex. That's how we added dimension to this planning evaluation. And that's how we think it's going to be like a It's not replacement, it's like complementary to the existing evaluation that we do. Yeah.

第一个结果是：这些题其实都很简单，但所有题平均下来，大部分模型都很差，低于 50%。按理说这些都是简单题嘛。当然，受算力限制我们测的是开源模型，但即便如此，低于 50% 也是很糟糕的结果、很糟糕的表现。所以这几个维度上我们确实都还有很大的提升空间。第二，我们看到这些 planning 技能上的平均表现，和最终任务表现高度相关——模型在这些技能上强，确实对应它在端到端任务上也强。所以我们现在的说法是：你当然应该把模型放到你自己的任务上跑、拿到轨迹、去评那些 thought，但那是很重的活；你可以先做我这套离线测试——把我的简单测试套到你的模型上，花 10 美元，就能在这几个独立维度上拿到一份初步评估，先筛掉一大批，然后再去跑那个复杂的评测。而且这样你还知道它到底错在哪：如果是某一项不行，那就先去做个 GRPO 之类的处理，再进到复杂评测。这就是我们给 planning 评测加上的一个维度。我们觉得它不是替代品，而是对现有评测的补充。

[6:35] Ankit

Oh, awesome Thanks so much for joining us. This is great. Thank you. Appreciate it. Thanks.

太棒了，非常感谢你来跟我们聊，收获很大。谢谢，感谢。