The Unreasonable Effectiveness of Separating the Task from the Model — Maxime Rivest & Isaac Miller

频道: AI Engineer
视频: https://www.youtube.com/watch?v=GgLQ02aO-hs
原文语言: en
统计: 共 42 轮

[0:01]

[music] Please welcome to the stage Maxim Rest and Isaac Miller. [applause] Wow. Isaac, myself, all of the DSPI community are so grateful to be here today to get to talk to you about AI programming DSPI and the unreasonable effectiveness of separating the task from the model, its harness, and all of the implementation details.

[音乐] 有请 Maxime Rivest 和 Isaac Miller 上台。[掌声] 哇。我和 Isaac，以及整个 DSPy 社区，都非常感激今天能站在这里，和大家聊聊 AI 编程、DSPy，以及「把任务与模型、harness 和所有实现细节分离开」所带来的、好到不讲道理的效果。

[0:38]

When you think about it, in programming, if we want to repeat the tasks often, we make it a function. We believe the same should be true for AR programs. Functions are awesome. Functions are reusable, composable, testable, and optimizable.

仔细想想，在编程里，如果一件事我们要反复做，我们就会把它写成一个函数。我们认为 AI 程序（AI programs）也应该如此。函数很棒，函数是可复用、可组合、可测试、可优化的。

[0:58]

To make a function, you give it a name. You define some inputs, some outputs, and then you have some implementation logic inside of it. You get to reuse your functions thousands of times. You can optimize it, but you can also compose it into bigger programs.

要写一个函数，你给它起个名字，定义好输入、输出，然后在里面写一些实现逻辑。你可以把这个函数复用成千上万次，可以去优化它，也可以把它组合进更大的程序里。

[1:16]

One of the really nice things about functions is that you can also package it and distribute it and someone else can use it and they just need to know about the contract on top of it to use it and they can treat it as a black box. DSPI brings all of these properties to AI programs. And so DSPI is an open source software in Python that lets you,

函数还有一个很好的特性：你可以把它打包、分发出去，别人拿来用时只需要知道它对外的契约（contract），就能把它当成一个 black box 来使用。DSPy 就是把这些特性全都带给了 AI 程序。DSPy 是一个用 Python 写的开源软件，它让你——

[1:41]

like I said, bring these properties to your AI workflows and AI programs. And it gives you all of the toolings you need to do that. Why do you want that? Well, we have been inventing a lot of terms in our fields in the last three years. It's growing fast. We have new models coming every other week. We have new techniques, new strategies. And if you're like me, you want to try all of them. But will any of these new specific techniques coming out

就像我刚说的，把这些特性带到你的 AI 工作流和 AI 程序里，并且提供你所需要的全套工具。为什么你会想要这些？过去三年，我们这个领域造了一大堆新名词，发展得非常快。每隔一周就有新模型出来，还有新技术、新策略。如果你和我一样，你会想把它们全都试一遍。但这些在不同时间点冒出来的具体技术，

[2:14]

at a different time really help on your task, on your job? Well, these are all just implementation tactics and you want to put them inside of clear contract. If for your repeated AI task you define an input interface and an output interface, you get to play in the internals. You get a lot of agility.

真的能帮到你手上的任务、你的工作吗？说到底，它们都只是实现层面的战术，你应该把它们放进一个清晰的契约里面。如果你为那些会反复执行的 AI 任务定义好输入接口和输出接口，你就可以随意折腾内部实现，从而获得极大的灵活性。

[2:37]

Let's make it concrete for AI. So my first AI program I made when I discovered DSI was that I had some invoices from my farm and I wanted to extract them to do my taxes. I wanted to extract the tax values from there. Then another AI program I did is that on my keyboard in my computer, I have a little command that reads my keyboard shortcuts, read my clipboard, and will correct the grammar for me. Sometimes I actually wanted to also rewrite for

我们把它落到 AI 的具体场景上。我发现 DSPy 之后写的第一个 AI 程序，是因为我农场里有一堆发票，我要报税，想把里面的税额抽取出来。我做的另一个 AI 程序是：在电脑键盘上设一个快捷命令，它会读我的剪贴板，然后帮我纠正语法。有时候我还想让它顺便改写得更清楚一些，

[3:08]

clarity. So I have another program that takes text, just rewrites it for clarity, put it back on my keyboard keyboard, and that's a command, and then I can like have a lot of agility and bring it different places inside of that. I can change it however I want. A new model comes up and I can change that. It's super easy because my interface is fixed like that. I'll skip that one.

所以我又写了一个程序，接收一段文本，把它改写得更清晰，再放回剪贴板，也是一个命令。这样我就有了很大的灵活性，可以把它用到各种不同的地方，想怎么改就怎么改。出了新模型，我随时可以换掉，非常简单，因为我的接口是固定的。这个例子我就跳过了。

[3:30]

But they're not uh restrained to very easy things and small input outputs. You can be very ambitious with AI programs. So in this examples, you could have your entire inbox and a new email coming in and you want to compose a new drafted reply.

但这些并不局限于很简单的事情、很小的输入输出。AI 程序完全可以很有野心。比如在这些例子里，你可以把整个收件箱加上一封新来的邮件作为输入，让它帮你起草一封回信。

[3:46]

We can do that in the Aspire with RLM recursive language models. This is an idea that came from around our community or more like things we probably all do agentic engineering or vibe coding. You can give it a spec a repository and you get a PR. Those are repeatable tasks.

这在 DSPy 里可以用 RLM（recursive language models，递归语言模型）来做。这个想法来自我们社区周边。或者说更像是我们大家可能都在做的事情——agentic 工程或者 vibe coding：你给它一份 spec 和一个代码仓库，它给你产出一个 PR。这些都是可重复的任务。

[4:06]

And so as I have been telling you when you fix that boundary you can focus on the how on the top and then inside of it you can have a little chat with just a simple prompt. You can integrate on that prompt. Agents come out you change it to be an agent. Tools gets invented you add tools and then we get into loop engineering. You put that inside of it too. Anything on the outside of it doesn't change your integration and and

所以正如我一直在讲的，当你把边界固定下来，上层你只需要关注「怎么做」，而在内部，一开始你可以只放一个简单的 prompt 做一次普通调用。你可以不断迭代这个 prompt。agent 出来了，你把它换成 agent；工具被发明出来了，你加上工具；再往后我们进入 loop engineering，你也把它塞进去。外面的一切都不会因此改变，你的集成方式也不会变，

[4:33]

anything else doesn't change. And when you have such a hard boundary, you can also start to automatically optimize. But how can you automatically optimize with just that simple signature? This is not enough. This is not enough to specify your task. And even before Chat GPD came out, the creator of DSPI had started to land on this idea that you need three things to specify your task.

其他东西也都不用动。而当你有了这样一条硬边界，你还可以开始做自动优化。但只靠一个简单的 signature（签名），你怎么自动优化呢？这是不够的，这还不足以完整刻画你的任务。其实早在 ChatGPT 出现之前，DSPy 的作者就已经逐渐形成了一个想法：要刻画清楚你的任务，需要三样东西。

[5:01]

And if you have this language and this ability to express your task in a programming language, you can start to automatically optimize and delegate away the implementation details. So the first one is what should happen. This is instructions. The signatures that I've been showing you are part of that. Here on the screen, you see the beginning of a real script in DSP. You set your model at the top. you configure that and it's

而当你拥有这套语言、拥有用编程语言表达任务的能力时，你就可以开始自动优化，把实现细节委托出去。第一样东西是「应该发生什么」，也就是 instructions（指令）。我前面给你们看的那些 signature 就属于这一类。屏幕上你看到的是一个真实 DSPy 脚本的开头：你在最上面设置模型、做好配置，而这部分

[5:28]

fully independent of the signatures here where you have natural language instruction to extract all taxes and um and if it's illeible to output zero then you say it I'm going to give you an input it's going to be a string I want you to give me an output and it's going to be a string and a float this is natural language expressing my needs this is very powerful and efficient if you think about it if you have a friend

和下面的 signature 是完全独立的。在 signature 里，你用自然语言写下指令：把所有税额抽取出来，如果字迹无法辨认就输出 0，你直接这么说就行。我会给你一个输入，它是一个字符串；我要你给我一个输出，它是一个字符串加一个浮点数。这就是用自然语言表达我的需求。这非常强大，也非常高效。你想想看，如果有个朋友

[5:52]

over coming to play a board game with you and You give them the instructions and they're ready to play. But if you want to do like alpha go or alpha zero and you tell them you're just going to learn from example, you're going to have a long night.

来找你玩桌游，你把规则说明念给他听，他马上就能开始玩。但如果你想搞成 AlphaGo 或者 AlphaZero 那样，告诉他「你只能从例子里自己学」，那这一晚上有得熬了。

[6:07]

And then the second one is what must happen. There are some constraints you have that they have to be listened to. They have to be enforced. The best way to do that is with code. So I want you to go to the third line, a fourth line. You have self extract and selfrecheck.

第二样东西是「必须发生什么」。有些约束是必须被遵守、必须被强制执行的。做这件事最好的方式是用代码。请大家看第三行、第四行，这里有 self.extract 和 self.recheck。

[6:28]

You can see we're doing a predict on the extract taxes and we're doing a chain of thought on the extract taxes. The first one is a vanilla program. The second one makes it do some reasoning. Now I'm taking them inside in the forward and you can see in the if not bread tax. This is a requirement I have that if my first simple vanilla program doesn't extract my taxes, I want you to rerun with more reasoning. I mean, I got

可以看到，我们对 extract taxes 做了一个 Predict，又对它做了一个 ChainOfThought。第一个是最朴素的版本，第二个会让它做一些推理。然后我在 forward 里把它们用起来，你可以看到那句 if not pred.tax。这是我的一个硬性要求：如果第一个简单版本没能把税额抽出来，我要求它用更多推理重跑一遍。毕竟，

[6:57]

to get my taxes right. And then another requirement I have is if the value is below zero throw, I want to show that to a human. I don't want to let you go. This will not change. Like even if I have AGI, I would hope it doesn't make mistake. But whatever is in the predictor, if they make these mistakes,

我的税必须算对。我还有另一个要求：如果数值小于零就抛出异常，我要把它交给人来看，不能就这么放过去。这一点是不会变的。就算真的有了 AGI，我当然希望它不犯错，但只要 predictor 里的东西还会犯这些错，

[7:16]

I still want these things to be true. So the last one is what good looked like. And when I was young, I was on the farm with my dad and I asked him, "How do you know that this tree is a maple?" And he couldn't tell me. He couldn't give me the instruction on how to know this tree is a maple. And he certainly couldn't give me code on how to know this tree is a maple. And so through time with example I learned how to know that a

我就依然要保证这些约束成立。所以最后一样东西是「好的结果长什么样」。我小时候在农场里跟着我爸，我问他：「你怎么知道这棵树是枫树？」他说不上来。他没法给我一套判断枫树的说明，更没法给我一段判断枫树的代码。于是我只能靠时间和大量例子，慢慢学会怎么认出一棵树

[7:44]

tree is a maple. But this is not limited to things like classifying plants. It's also for all of the long tails in your specifications that are things that are more latent. These are sometimes a reason why you would do internship and you would have a mentor and a mentee.

是不是枫树。但这不只适用于给植物分类这种事。它同样适用于你需求规格里那些长尾的、更隐性的部分。这也正是为什么会有实习制度、会有导师带学徒的原因。

[8:01]

You're looking at a lot of of examples and there are long tales of successful behaviors that you have to see and learn. Now that you have all of these, you have express fully. You have all these three languages you can put together. You have the specs, the code,

你要看大量的例子，那些成功行为里有很长的长尾，你必须亲眼见过才能学会。现在这三样你都有了，你就完整地表达了你的任务。你有这三种语言可以组合起来：spec、代码，

[8:15]

and the evals. And now your goal is fully specified. And so you can start optimizing. You can use things like Japa on your metrics and on your program. And you can start optimizing. At the beginning of the Aspire, the chip didn't exist. The models were not good enough to optimize.

还有 eval。到这里，你的目标就被完整刻画出来了，于是你可以开始做优化。你可以用 GEPA 这样的东西，作用在你的 metric 和你的程序上，开始优化。在 DSPy 早期，GEPA 还不存在，那时候模型也还没好到可以做优化。

[8:33]

And so we were using code to find few shots examples to make the base models uh act in the proper way. Then models got better and so we could automatically optimize instruction. And in the future we are starting to be able to be liberated more and more from the implementation details and delegate that away. And at the end our hope in the Aspire is that you can stick to all of that and then just the news and the implementation details will be automated

所以我们当时是用代码去寻找 few-shot 示例，让基础模型表现出正确的行为。后来模型变强了，我们就可以自动优化 instruction。而在未来，我们正越来越多地从实现细节里被解放出来，把这些交出去。DSPy 最终的期望是：你只需要盯住这三样东西，剩下的新玩意儿和实现细节都能自动帮你搞定。

[9:02]

for you. Isaac will talk to you a lot more about what has been released in the last year, what we're releasing now, and all of the future plans we have. Thank you. [applause] Thanks, Max. So, we've given you a pretty big abstract overview of specs, code, and evals, but these aren't things that are just restricted to the academic sphere.

接下来 Isaac 会详细讲讲过去一年我们发布了什么、现在正在发布什么，以及未来的全部规划。谢谢大家。[掌声] 谢谢 Max。刚才我们给了大家一个关于 spec、代码和 eval 的宏观抽象概览，但这些绝不只是学术圈里的东西。

[9:27]

These are used in production by some of the biggest enterprises for massive gains. And we see two main benefits when you use DSP in the enterprise. First is that your implementation becomes cheaper. When you're flexible to what the implementation is, you can use the bitter lesson to search over different solutions, find something that solves your problem cheaply. And you can use this to scale to data sizes that weren't

一些最大的企业已经在生产环境中使用它们，并取得了巨大的收益。企业使用 DSPy 时，我们看到两个主要好处。第一，你的实现变得更便宜。当你对「具体怎么实现」保持灵活时，你就可以借助 bitter lesson（苦涩的教训）去搜索不同的解法，找到能便宜地解决问题的方案。你还能用它把规模扩展到

[9:55]

possible with a more expensive implementation. Shopify 550 times cheaper. They're able to do that because they went from an expensive model to a cheap model, but they could keep the same emails, keep iterating on their business logic inside, and try new things.

用昂贵实现方式根本处理不了的数据量。Shopify 把成本降到了原来的 1/550。他们能做到，是因为从一个昂贵的模型换成了便宜的模型，同时可以沿用同一套 eval，继续在内部迭代业务逻辑、尝试新东西。

[10:13]

There's three awesome case studies here, and you should check them out after the talk. They give you a lot of details on how you can do this in your own enterprise. Now, part of the reason why you want to build in the DSPI ecosystem is that we're constantly adding new techniques for you to try. And it's important to know none of these techniques we add will definitely solve your problem because that's your job. What we can do

这里有三个很棒的案例研究，演讲结束后大家可以去看看。它们讲了很多细节，告诉你怎么在自己的企业里做到这一点。而你之所以想在 DSPy 生态里做开发，其中一个原因是我们在不断加入新技术供你尝试。有一点很重要：我们加进来的这些技术，没有一个是一定能解决你的问题的——那是你的工作。我们能做的，

[10:39]

is we can solve sub problems for you that make your implementation easier. For instance, Alex Zang, a PhD student at MIT, came out with this paper called recursive language models. Recursive language models are a way to solve some kinds of long context programs. And guess what? We can bring this in to DSPI for you to try see if it helps your long context tasks. Maybe it will, maybe it won't. But the thing is, it's one line

是帮你解决那些子问题，让你的实现更容易。举个例子，MIT 的博士生 Alex Zhang 发表了一篇论文，叫 recursive language models（递归语言模型）。RLM 是解决某一类 long context 问题的方法。猜怎么着？我们可以把它接进 DSPy，让你试试看它对你的 long context 任务有没有帮助。也许有用，也许没用。但关键在于，只需要一行代码，

[11:08]

and your signature stays the same. That's what's important here. Everything gets to stay constant and you get to see if this solves your problem or not. And we've had a number of examples of this just in the last year from people building in and around the DSPI community. We've had RLMs. We've had Jeepa which is an incredible prompt optimizer out of Berkeley. Better together multiodule gpo. All these are incredible research innovations that you

你的 signature 完全不用动。这才是重点。所有其他东西都保持不变，你就能验证它到底能不能解决你的问题。仅仅在过去一年里，来自 DSPy 社区内外的开发者就贡献了很多这样的例子。我们有了 RLM，有了 GEPA——一个出自伯克利的、非常出色的 prompt 优化器，还有 Better Together、multi-module GRPO。这些都是了不起的研究成果，

[11:35]

get to try in your implementation just by being in the DSP ecosystem. And we have more coming in DSP4. I'm excited to talk to you about two of those today. DSPI Flex and Qualitative Learning. DSP.flex is a new kind of module. In DSpay, when we let you optimize things,

而你只要身处 DSPy 生态，就能直接在自己的实现里试用它们。DSPy 4 里还有更多新东西。今天我想重点讲其中两个：DSPy Flex 和 Qualitative Learning。dspy.flex 是一种新的 module。在 DSPy 里，当我们让你去优化某些东西时——

[11:57]

it started with few shot examples, then it became prompts, and now that's becoming code. for any function that you want to implement. You can actually learn a harness over time to solve that function. And this is completely custom. And you don't care about the implementation as long as it solves your business problem. What you've created ways to measure because you've defined the three core parts of specs, code, and

一开始是 few-shot 示例，后来变成了 prompt，而现在它正在变成代码。对于任何你想实现的函数，你其实都可以随着时间学出一套 harness 来解决它。而且这完全是定制化的。只要能解决你的业务问题，你根本不用关心具体实现是什么。你之所以能有衡量的手段，是因为你已经定义好了三个核心部分：spec、code，还有

[12:23]

evals. The second thing I'm excited to talk about is qualitative learning. One of the hard hard problems in AI engineering is building evals. And there's a few reasons why this is hard. One is that defining what good looks like is really challenging for any real world problem.

eval。我想聊的第二件事是 qualitative learning（定性学习）。AI engineering 里最难的问题之一，就是构建 eval。这件事难，有几个原因。第一，对任何真实世界的问题来说，定义「什么才算好」都非常有挑战。

[12:46]

The second is that when you define good often times you have to lose detail. If an email is good or bad contains a lot less information than if you know what could change in that email in order to improve. And the third is that whenever you create a hill and a data set, you're really trying to create a proxy for reality. What if instead we could use reality to inform our ebs automatically?

第二，当你去定义「好」的时候，往往不得不丢掉很多细节。「这封邮件好还是不好」所包含的信息，远远少于「这封邮件里改哪些地方才能变得更好」。第三，每当你造出一个可以爬的「山坡」和一个数据集时，你其实是在为现实制造一个代理指标。那如果反过来，我们能直接用现实来自动指导我们的 eval 呢？

[13:13]

What qualitative learning asks is how do we decrease this question? How do we decrease assistance? And it's a research question right now. But what we believe is that models are now good enough to interpret whatever textual feedback is present in the environment and convert that into evals and a hill that the model can climb. And so as you get more feedback from production, its traces,

qualitative learning 要问的是：我们怎么缩小这个差距？怎么减少对人工介入的依赖？这目前还是一个研究问题。但我们相信，现在的模型已经足够强，可以解读环境中出现的各种文字反馈，并把它转化成 eval，转化成一个模型可以往上爬的「山坡」。于是，当你从生产环境拿到越来越多的反馈——它的 trace、

[13:37]

its user actions, its product analytics, it's asking you, it's the model asking you questions about how data should be represented. As you do this, the model can iteratively refine the hill over time and continue climbing it to solve your actual business problem.

它的用户行为、它的产品分析数据，还有模型主动来问你的问题——模型会来问你，数据应该怎么表示。在这个过程中，模型就能一轮一轮地打磨这个「山坡」，并持续往上爬，最终解决你真正的业务问题。

[13:55]

And DSP focuses on these kinds of last mile problems. We have a really strong research ecosystem and we collaborate really closely with them. And that's part of the beauty is that we can see the problems that happen in applied AI engineering. Sol define them, build a benchmark and then solve them with techniques and then we get to democratize the results of that to everyone because it's open-source open research.

而 DSPy 关注的正是这类「最后一公里」的问题。我们有一个非常强的研究生态，也和他们合作得非常紧密。妙就妙在，我们能看到应用型 AI engineering 中真实发生的问题，把它们定义清楚，建一个 benchmark，然后用技术手段去解决，接着再把成果普惠给所有人——因为我们是开源、开放研究的。

[14:21]

Now, one common question is what happens when we have AGI? Well, even when we have an incredibly smart model, the model won't know how to solve your problems. It won't know how to do your tasks or have your context. And so this genre of last mile learning is trying to ask how do we efficiently do this learning intelligence is very different from being all knowing.

有一个常见的问题是：等我们有了 AGI 会怎么样？其实，哪怕我们有了一个极其聪明的模型，它也不知道该怎么解决你的问题。它不知道怎么做你的任务，也没有你的上下文。所以这一类「最后一公里学习」想问的是：我们怎样才能高效地完成这种学习。智能，和无所不知，完全是两回事。

[14:47]

If you were to ask Albert Einstein to help you with your emails, he'd probably ask what's an email. But if you AGI will know how to do your emails. Nevertheless, it won't know how to actually solve your problem and interact with the people you need to interact with. It won't understand your relationships without learning this context over time.

如果你让 Albert Einstein 帮你处理邮件，他大概会先反问一句「什么是 email？」当然，AGI 是知道怎么处理你的邮件的。但即便如此，它也不知道该怎么真正解决你的问题，怎么跟你需要打交道的人打交道。如果不随着时间去学习这些上下文，它就理解不了你的人际关系。

[15:12]

Since 2022, DSPI has been focused on these three core ideas of specs, code, and evals, all defined as a programmatic interface. We've certainly evolved over time, and new techniques are incredible. We've gone from evolving few shots to prompts to now harnesses and now evolving your emails over time, too.

从 2022 年开始，DSPy 就一直专注于这三个核心理念：spec、code 和 eval，而且全部以编程接口的方式来定义。这些年我们当然也在不断演进，新技术非常惊艳。我们从演化 few-shot 示例，到演化 prompt，再到现在演化 harness，甚至现在还能随着时间演化你的邮件。

[15:35]

But what you need to ask for any of these new techniques is how do they help you solve harder problems or solve your own problems better? And you should ask this question in a datadriven manner. You should look at this new technique say how can I apply this to the business problem that I have? You should define your problem and you should hold your prompts, models, and code accountable to the problem that you need them to solve.

但面对任何一项新技术，你都要问：它能不能帮你解决更难的问题，或者把你自己的问题解决得更好？而且你要用数据驱动的方式来问这个问题。看到一项新技术，你应该问：我怎么把它用到我手上的业务问题上？你应该把你的问题定义清楚，并且让你的 prompt、模型和代码，都为你要解决的那个问题负责。

[16:04]

And what's awesome about when you build in this way where you have flexible implementations, what you unlock is you unlock the ecosystem of all the techniques that anyone in this room is constantly inventing. You unlock access to the collective intelligence of everyone here, all sharing techniques together.

而用这种方式来构建——也就是让实现保持灵活——最棒的地方在于，你解锁了整个生态，解锁了在座每一个人不断发明出来的所有技术。你解锁了这里所有人共享技术所汇聚成的集体智慧。

[16:26]

So, if you want to build reliable AI software, I encourage you to come check out DSP. We're completely open-source, open research, and we're here to help you solve your problems by building reliable software. We have a Discord that you should come join. And when you come up with the next technique, you should come contribute it to DSP. And we can help you distribute it and make this awesome technique available for everyone. Thank you.

所以，如果你想构建可靠的 AI 软件，我很建议你来了解一下 DSPy。我们完全开源、开放研究，我们就是来帮你通过构建可靠软件来解决问题的。我们有一个 Discord，欢迎大家来加入。等你想出了下一项新技术，欢迎来给 DSPy 贡献。我们可以帮你把它分发出去，让这个很棒的技术惠及所有人。谢谢大家。