How to Build AI Agents That Check Their Own Work | Jared Zoneraich

频道: Peter Yang
视频: https://www.youtube.com/watch?v=0YeeJHYy-Vc
原文语言: en
统计: 共 104 轮 · Jared 58 · Peter 37

[0:00]

Let the model cook and don't try to push it in certain directions. My running for 9 hours straight now. It's crazy. More Devons are being launched from other Devons or programmatically than by humans. We're not trying to replace coding. We're trying to build abundance of good software and really multiply an engineer. You want the engineer to bring taste and bring this highle decision-m but agents should be doing everything else and the agent shouldn't be bottlenecked on the engineer. All right. Hey everyone. I'm really excited to host Jared building in residence at cognition today. Jared has a lot of background building AI agents. So going to ask him about you know how to build a good AI agent, have him demo Devon's latest capabilities and also talk about you know the pretty competitive AI coding and knowledge workspace. Welcome Jared.

让模型自己发挥，别硬把它往某个方向推。我的 agent 已经连续跑了 9 个小时了，太疯狂了。现在由其他 Devin 或程序自动启动的 Devin，比人类手动启动的还多。我们不是要取代编程，我们是想创造优质软件的富足，真正放大一个工程师的产出。你希望工程师带来品味、做高层决策，但其余一切都该交给 agent，而且 agent 不应该被工程师卡住瓶颈。好了，大家好！今天非常兴奋能请到 Cognition 的 Builder in Residence——Jared。Jared 在构建 AI agent 方面有很深的积累，所以今天我会问他怎么才能做出一个好的 AI agent，请他演示 Devin 的最新能力，也聊聊竞争相当激烈的 AI 编程和知识工作领域。欢迎 Jared。

[0:49] Jared

Thanks for having me Peter. Why don't we start with some high level questions like how do you think someone should think about building an agent to today and what what is the number one mistake that one companies make when doing this stuff? I think first maybe I'd want to break it down into like two different types of build what building an agent means and primarily I guess the first one is building a userfacing agent or something that your users are interacting with a chatbot or something like that and the second type is probably where my mind goes of an agent that you use so I spoke to a lot of DevOps teams recently that are building internal agents for their team to work better which is like there's similarities and differences is I think when we're talking about an external agent and building things on top of the model, I I think one of the biggest mistakes and one of the most important things is you just have to keep in mind model progress and you want to be building in a way that lines up with honestly the road map of the model labs and the people building models and you want to be building in a way that is not fighting gravity so to speak. So for example, there was a lot of I guess how long have LLMs kind of been around now? Only like 2 3 years, four years, something like that. Since the first chat GBT and things like tool calling, things like reasoning, remember chain of thought, people would kind of prompt their prompt their agent in a very specific way to get this reasoning and now it's built into the model. For things like this, there's a lot of kind of hacks we still need to do to get the model to follow instructions well. And my general highle advice is to build your own road map in a way that assumes these hacks are going to be less and less important and these are not going to be the defensibility of your company. So today we've already reached a point where a lot of teams have gone from large DAGs of this node connects to this node that connects to this prompt that connects to this prompt and it's and it actually might be coming back as of like yesterday with this new kind of like dynamic workflow stuff that Anthropic is working on. But still the highle principle of let the model let the model cook and don't try to push it in certain directions. It's just going to make your life easier. It's going to save dev time and you're going to you're going to need less scaffolding around agents as the models get better and they're pretty good already.

谢谢你邀请我，Peter。——那我们先从一些宏观问题聊起：你觉得今天应该怎么思考构建一个 agent？公司做这类事情时最常犯的头号错误是什么？——我想先把「构建 agent」拆成两种不同的类型。第一种是构建面向用户的 agent，也就是你的用户直接与之交互的东西，比如一个聊天机器人；第二种是我脑子里首先想到的：你自己用的 agent。我最近和很多 DevOps 团队聊过，他们在给自己团队做内部 agent 来提升工作效率。这两者有相似也有不同。当我们讲面向外部的 agent、在模型之上做东西时，我认为最大的错误之一、也是最重要的一点是：你必须时刻记住模型在进步，你的构建方式要和模型实验室、和做模型的人的路线图对齐，不要「逆着重力」去构建。举个例子——LLM 到现在才多久？从第一版 ChatGPT 算起，也就两三年、四年吧。像 tool calling、reasoning 这些——还记得 chain of thought 吗？人们曾经用非常特定的方式去写 prompt 来获得推理能力，而现在这些都内置进模型了。类似地，今天我们还需要很多 hack 才能让模型好好遵循指令。我总体的高层建议是：规划你自己的路线图时，要假设这些 hack 会越来越不重要，它们不会成为你公司的护城河。今天很多团队已经从那种巨大的 DAG——这个节点连那个节点、这个 prompt 连那个 prompt——走出来了。当然，就在昨天，随着 Anthropic 在做的动态 workflow 这类新东西，这种做法可能又有点回潮。但高层原则依然是：让模型自己发挥（let the model cook），别硬把它往某个方向推。这会让你的日子好过很多，省下开发时间，而且随着模型变好，你需要的 agent 脚手架会越来越少——何况它们现在已经相当不错了。

[3:19] Peter

Got it. So basically like start with like a prompt and some like tools that the agent can use and just kind of iterate from there, right? Is that kind of

明白。所以基本上就是：从一个 prompt 加上一些 agent 能用的工具开始，然后在这个基础上迭代，对吧？大概是这个意思？

[3:26]

Yeah. I almost think tool engineering is going to be the new the new thing. just figuring out which tools to give the model and assume that the model's going to figure it out and and be able to try things out and and the prompt in that case and the skills in that case are like a cheat sheet of what tools to use for what. But otherwise, the model will just try everything. just to get a little bit tactical here like so let's let's say the initial prompt maybe it's just like tools kind of like what you're agent for like customer support or some or something and then like specifically like your advice is not to include a bunch of stuff like think ultra hard or like take the step you know here's one two three steps how you proceed like kind of just give us some principles instead of like very explicit instructions is that what you're thinking

对。我甚至觉得 tool engineering（工具工程）会成为新的关键——搞清楚该给模型哪些工具，然后相信模型自己会想明白、会去尝试。这种情况下，prompt 和 skill 更像一张「什么场景用什么工具」的小抄。除此之外，模型自己会把各种办法都试一遍。——我们来聊点具体的战术。比如说最初的 prompt，可能就是工具加上「你是客服 agent」之类的设定。你的建议是不是：不要塞一堆「think ultra hard」、「第一步第二步第三步该怎么做」这种东西——给一些原则，而不是非常明确的步骤指令？你是这个意思吗？

[4:10] Jared

yeah so I think everything's case by case here so it's very hard to give blanket advice but the I think the mental model or the heristic you should have is if it feels like you need to give these very ultraspecific prompt instructions today, don't treat that as the your your defensibility or your differentiation in the market because the models are going to get better and you're going to need less and less of that. And even today, you see, I was working on a project the other day and on the best models today, you just give a very simple one line like there's a weird bug with this login flow and it'll go and figure it out. And that was not true a year ago. That was not true probably six or eight months ago. Uh, and that means that if your prompt, if your whole company relies on this fancy prompt that says you need to reason and do this, the models are going to catch up.

对。我觉得这里一切都要具体情况具体分析，很难给放之四海而皆准的建议。但你应该有的心智模型、或者说启发式是：如果你今天觉得必须写那种极其具体的 prompt 指令才行，别把它当成你的护城河或者市场差异化，因为模型会越来越强，你需要的这类东西会越来越少。即便是今天你也能看到——我前几天在做一个项目，在当下最好的模型上，你只要给一句非常简单的话，比如「这个登录流程有个奇怪的 bug」，它就会自己去把问题找出来。一年前这做不到，六八个月前大概也做不到。这意味着，如果你的 prompt、你整个公司都依赖那个「你需要先推理、再这样那样做」的花哨 prompt，模型迟早会追上来。

[5:09] Peter

Yeah, dude, this is a big question, right? What actually is defensible for like a company to build an agent themselves? Obviously, if you have proprietary data, that's somewhat defensible. Otherwise, I feel like there are a lot of AI rapper companies that are just prompts, right? They're just like prompts and skills.

是啊，老兄，这可是个大问题，对吧？对一家自己做 agent 的公司来说，到底什么才是有护城河的？显然如果你有专有数据，那算一种护城河。否则我感觉市面上有一大堆 AI 套壳（wrapper）公司就只是一堆 prompt，对吧？就是 prompt 加 skill。

[5:24] Jared

Everything's a prompt and skill. Everything boils down to prompts and skills, right? I I it just maybe you're prompting humans, but I think well maybe we could use cognition as an example. Uh

一切都是 prompt 和 skill。所有东西归根结底都是 prompt 和 skill，对吧？也许你只是在「prompt 人类」而已。不过——要不我们就拿 Cognition 当例子来说吧。

[5:38]

yeah,

好，

[5:38] Jared

I don't think nobody uses the rapper term like oh cognition is just a rapper because I think it's very clear we've built a like differentiation uh one on the harness itself. the harness is really good but also through the go to market motion through the FDE and the I mean FD is the talk of the term of the day right but uh I

我觉得没有人会用「套壳」这个词说「Cognition 只是个套壳」，因为很明显我们建立了差异化：一是 harness 本身——我们的 harness 真的很好；二是通过 go-to-market 的打法，通过 FDE——forward deployed engineering 可是当下的热词——

[6:02] Peter

sorry what's what's FD for for deployed engine

不好意思，FDE 是什么的缩写？forward deployed……？

[6:04] Jared

for deployed engineering so kind of embedding engineers with customers and stuff like that but yeah what is defensibility I I'm somewhat less interested in that question I think if you build something cool that is actually interesting great entrepreneurs can figure it Yeah. Defensibility is just like shipping and moving fast and like reacting to all the latest models.

Forward deployed engineering（前置部署工程），就是把工程师嵌到客户那边一起干活之类的。不过说回护城河——我对这个问题其实没那么感兴趣。我觉得只要你做出了真正有意思的酷东西，优秀的创业者自然能想出办法。对，护城河也可以就是持续发布、快速行动、对所有最新模型快速做出反应。

[6:27] Peter

Yeah. Defensibility could just be moving fast. Exactly.

对，护城河也可以就是「跑得快」。没错。

[6:30] Peter

Yeah. Yeah. And and it's funny you mentioned for deploy engineers because I feel like I don't know. I I feel like even like anthropic and openi like they have pretty small product teams, right? And but they have a ton of sales people and like for deployed engineers because like the market is actually not AI native builders that are sitting on Twitter. the market is like, you know, a centure or like these companies that have no idea what the hell this stuff even is,

是啊，你提到 forward deployed engineer 挺有意思的。因为我感觉像 Anthropic 和 OpenAI，他们的产品团队其实都挺小的，对吧？但他们有一大堆销售和 forward deployed engineer。因为真正的市场并不是坐在 Twitter 上的 AI 原生 builder，而是像 Accenture 这种对这些东西完全摸不着头脑的公司，

[6:53] Peter

right? So, you got to train. I'm not sure where you're seeing cognition. Like, is cognition are a lot of the customers like pretty AI native or like trying to get started trying to figure this stuff out?

对吧？所以你得去培训他们。我不确定 Cognition 那边是什么情况——Cognition 的客户里，是很多本来就相当 AI 原生的，还是很多刚起步、还在摸索这些东西的？

[7:01] Jared

Yeah, we live in a very interesting world here because we're on Twitter. I don't know about you, but I'm on Twitter a lot. And uh I and Twitter is the cutting edge for what it's worth. We're like whatever percent. We are living in the future. We're using all the new models when they come out. We're using all the new techniques. And yeah, there's a lot of world that there's a lot of the world that's using co-pilot and and using pad to have autocomplete with code and and that's and that's not even a joke. This is this is a very true thing and uh I think where cognition has had a lot of success and I've kind of sat into a lot of these customer meetings too as like I guess my role is kind of interesting because I get to live in the startup world and also kind of bring that energy to the enterprise world and back and forth and some of the stuff the enterprise world actually has down better than the startup world in terms of security and some stuff like that. But anyway, my point is yes, these a lot of the world doesn't understand agent loops and is not AI pill so to speak and it's very you could look at forward deploy motions and say oh that's because the product is not good enough for people to self-s serve and I think it's always good to have criticism against like how you could be better at just being more seamless but I really think what's happening is we had a massive technology innovation on what agents can do. And my agent's running for 9 hours straight now. It's crazy. And what that means is because it's exploded so quickly, there's so many people who like would normally in these other industries take years to become AIDS. But if you could actually sit down with them and show them and I remember this from when kind of agent agentic coding or coding agents were first becoming a thing at my last startup when we'd like sit down with every engineer and like watch me do cloud code so you could see how good it is and I think the same thing's true if you can you show people the magic and kind of like nobody nobody believes how good these things are.

是的，我们活在一个很有意思的世界里，因为我们泡在 Twitter 上。不知道你怎样，反正我刷 Twitter 很多，而 Twitter 就是最前沿。不管怎么说，我们只是很小的一撮人，活在未来：新模型一出来我们就用，新技术一出来我们就上。而世界上还有一大片人在用 Copilot、在用 Tab 自动补全写代码——这真不是开玩笑，是千真万确的事。我觉得 Cognition 很成功的地方——我也旁听了很多客户会议，我这个角色挺有意思：既能活在创业圈，又能把那股劲头带进企业圈，再来回穿梭。说实话，有些东西企业界反而比创业圈做得好，比如安全之类的。总之我想说的是：世界上很多人确实不理解 agent 循环，也还没被 AI「点醒」（AI-pilled）。你可以看着 forward deploy 这种模式说「那是因为产品不够好，用户没法自助上手」——保持这种批评、想着怎么做得更丝滑当然是好事。但我真心认为正在发生的事情是：agent 能做的事出现了一次巨大的技术跃迁——我的 agent 现在已经连续跑了 9 个小时，太疯狂了。因为爆发得太快，很多其他行业的人按正常节奏可能要好几年才会被 AI 点醒。但如果你能真的坐下来演示给他们看——我记得在我上一家创业公司，coding agent 刚兴起的时候，我们会和每个工程师坐在一起：「看我用 Claude Code」，让他们亲眼看到它有多强。现在也一样：你把这种魔法展示给人看——说真的，不亲眼看到，没人会相信这些东西已经这么强了。

[9:08] Peter

Yeah. until you show it to them. Right.

对，直到你演示给他们看。没错。

[9:09] Jared

Exactly. The thing that makes the forward deploy motion at cognition unique is that we it's less about just AI education and more about hey let's sit together and use agents to solve this actual problem you have and like in this process we're actually taking tickets off their board and that's how you really prove it to them.

正是。Cognition 的 forward deploy 模式独特的地方在于：它不只是做 AI 科普，而是「来，我们一起坐下来，用 agent 解决你手头这个真实的问题」——在这个过程中我们是真的在帮他们消掉看板上的工单，这才能真正让他们信服。

[9:30]

Got it. Got it. This episode is brought to you by Riverside. I've used Riverside for years to record my podcast because it records in 4K resolution each person locally. So, the audio and video still comes through clean even if a guest Wi-Fi gets shaky. But the reason I love it now more than ever is what happens after we stop recording. If I go in here that I can use these AI tools to remove pauses, remove filler words, and just clean up the recording. And I I can also edit the transcript directly and it will automatically generate clips with captions ready to publish to YouTube, Spotify, and all types of social media platforms all from one place. As a oneperson creator business, that matters a lot. Riverside is the upgrade your content workflow needs. Try it at creators.side.com/peryang and use code petery at checkout to get one month completely free. That's creators. side.com/peryang. Now back to our episode before we have you demo like using using dev. Let me ask you this. So you know before cognition you worked at prompt layer so you have a lot of knowledge about kind of observing how these agents work and and I I think obviously there's kind of eval and stuff like that. I just feel like okay my point of view is that when companies think about building agents they tend to over complicate things. they like they want to build like an orchestrator and like all this kind of stuff or like they want to build really complicated emails when like you haven't even shown this stuff to a real customer yet. So how do you think about the roll out phase of these agents, right? Once it gets to production, it should be pretty reliable and like not go off the rails, but like how do you kind of go through the phases of getting there? Well, there's this classic trope of the AI psychosis when someone's just comes to you and they said, "I I've spent the last month building my personal cloud code inventory system and they have like 13 agents doing different things." And yeah, I think I agree with you. Uh, perfect is always the enemy of complete. And a lot of these things, less is more. With a lot of these things, you don't actually need tons of scaffolding to get an MVP out. And I think the other advantage of kind of living in this AI age, on one hand, everybody can build so much quicker, so more polish is expected, but on the other hand, AI users kind of understand that it's a probabilistic technology and it's not. is sometimes it's going to need a little bit more proddding and users I think are a little bit more forgiving with really novel stuff in AI. So I guess yeah my my take on it is teams should not it is a common failure case that teams get stuck on evals and stuck on building the perfect testable system because they're coming from this world of der deterministic code and they're like okay just like in the old world I needed to have tests for every single thing I need to do that in my agent and frankly you're not going to be able to do that and some of the best teams believe it or not are just shipping while the prod without eval set. So I would say if if you're a team that's building like one of your first flagship agent products or something like that, done is better than perfect. And don't get stuck on planning it out and just do it. And you'll be surprised at how good these things are and how easy it is to get to 80%. And now the last mile is the hard part. And that's going to take a year. And that's like 80% with agents. You could take an hour and then the last part takes a long time. But a lot of people don't get to the 80% because they want to plan out the whole 100%.

明白，明白。本期节目由 Riverside 赞助。我用 Riverside 录播客很多年了，因为它以 4K 分辨率在每个人的本地录制，就算嘉宾的 Wi-Fi 不稳，音视频依然清晰。而现在我更爱它的原因是录完之后的部分：我可以用这些 AI 工具去掉停顿、去掉口头语，把录音清理干净；还能直接编辑文字稿，它会自动生成带字幕的短视频片段，一站式发布到 YouTube、Spotify 和各种社交媒体平台。对一个一人公司的创作者来说，这非常重要。Riverside 是你的内容工作流需要的升级。访问 creators.riverside.fm/peteryang，结账时用码 peteryang 可免费获得一个月。现在回到节目。——在请你演示 Devin 之前，我先问你这个：你在 Cognition 之前在 PromptLayer 工作，所以你对观察这些 agent 怎么运行很有经验，显然还有 eval 这类东西。我的观点是：公司在考虑构建 agent 的时候，往往把事情搞得过于复杂——他们想搞一个 orchestrator、搞一大套东西，或者想建特别复杂的 eval，可你连真实客户都还没见过呢。所以你怎么看 agent 的上线阶段？到了生产环境它应该足够可靠、不能脱轨，但你是怎么分阶段走到那一步的？——嗯，有个经典的「AI 走火入魔」的段子：有人跑来跟你说「我过去一个月都在搭我的个人 Claude Code 库存管理系统」，里面有 13 个 agent 各干各的。我同意你的看法：追求完美永远是「做完」的敌人。很多时候少即是多，你其实不需要一大堆脚手架就能把 MVP 发出去。活在这个 AI 时代还有一个特点：一方面人人都能更快地做东西，所以大家对打磨程度的期待更高了；但另一方面，AI 用户也理解这是概率性技术，有时候需要多戳它两下，用户对真正新颖的 AI 产品其实更宽容。所以我的看法是：团队卡在 eval 上、卡在构建完美可测试系统上，是个常见的失败模式——因为他们来自确定性代码的世界，觉得「旧世界里我每样东西都要有测试，我的 agent 也得这样」。坦白说，你做不到的。而且信不信由你，一些最强的团队就是没有 eval 集直接上生产。所以如果你的团队在做你们第一个旗舰 agent 产品之类的东西：done is better than perfect（完成胜过完美），别卡在规划上，直接动手。你会惊讶于这些东西有多好、做到 80% 有多容易。而最后一公里才是难的部分，那可能要花一年。agent 就是这样：80% 可能一小时就到了，最后那部分要很久。但很多人因为想把 100% 全规划好，反而连 80% 都到不了。

[13:09] Peter

Yeah. Yeah. I mean, the last mile is basically still like human like human craft and taste, you know, like with with all the AI stuff I do, like I always spend the last 10% actually looking at it like looking at AI's output, you know,

对对。我是说，最后一公里说白了还是人的手艺和品味。我做的所有 AI 相关的事情，最后 10% 我一定会亲自去看 AI 的产出，你懂的，

[13:21] Peter

right? And that's the differentiation. Yeah, that is the firstation. Otherwise, just putting all slop like all over the place, you know.

对吧？而这就是差异化所在。对，这就是差异化。不然的话，就是把 AI slop（AI 垃圾产出）撒得到处都是。

[13:28] Jared

Exactly. I I have a I I'm building one of the projects I'm working on right now is kind of like open claw style assistant, meaning I say openclaw style meaning it has a heartbeat and it's proactive. And building the harness, I've like spent very little time on the evals. had Devon build evals for me and look at the evals and iterate on its own. But I'm saving the part where I'm actually sitting down and tweaking it until the end when everything else is built in the scaffolding and you're getting the responses and the responses are pretty good and then when I want it to be really good. That's kind of like the finishing touches is my opinion. And the finishing touches as we know take way more time.

没错。我现在手头在做的一个项目，是个类似 OpenClaw 风格的助理——我说 OpenClaw 风格，是指它有「心跳」、会主动出击。在搭 harness 的过程中，我几乎没在 eval 上花时间——我让 Devin 帮我建 eval，让它自己看 eval 结果、自己迭代。而我真正坐下来亲手调校的部分，是留到最后的：等其他脚手架都搭好了、回复已经相当不错了，到我想让它变得真正好的时候——那才是收尾的精修。而我们都知道，精修花的时间反而多得多。

[14:12] Peter

And and just to real quick question like what what kind of evals are you building for it? like like just like does the code run evals like yes no evals you know end to end testing kind of eval

快速问一句：你给它建的是什么样的 eval？就是「代码能不能跑」那种 yes/no 的 eval，还是端到端测试那种 eval？

[14:23] Jared

yeah in this case so without I haven't released it yet so without giving away the full spec of it it's it's a personal assistant chatbot something like that and it proactively can notify you but you could also chat with it and I am the emails I have it doing right now I basically just told Devon make a bunch of examples for data recall, for tool use, for response length. And basically, I just had Devon build a sanity tech or if you're if you're using a GPT a lot, build a lot of smoke tests is the is the word GPT 5.5 loves. But I had it build a lot of these sanity checks. So it has basically purely as a tool that Devon could then use to check its own work and then iterate on its own and like see how far it could get autonomously.

嗯，这个项目我还没发布，所以不透露完整的 spec——它是个个人助理聊天机器人之类的东西，能主动给你发通知，你也能和它聊天。我现在让它跑的 eval，基本上就是我跟 Devin 说：给数据召回、工具使用、回复长度各做一批示例。基本上我就是让 Devin 建了一套 sanity check——如果你常用 GPT 的话，「smoke test（冒烟测试）」是 GPT-5.5 最爱用的词。我让它建了一大堆这种 sanity check，纯粹作为一个工具，让 Devin 之后可以用来检查自己的工作、自己迭代，看看它能自主走多远。

[15:21] Peter

Okay, got it. Okay, so some of it is maybe just like like browser use, computer use kind of like just clicking through the flows and you know

OK，明白。所以其中一部分可能就是 browser use、computer use 这类——把流程一步步点一遍之类的？

[15:28] Jared

it does that but in this case I am evaluating the agent response uh that it's building. So, it's basically a few programmatic endpoints that result in a markdown file and then Devon just reads the markdown file to see what the old outputs and the new outputs are and makes its own decisions.

它也会做那些，但这里我评估的是它构建的那个 agent 的回复。基本上就是几个程序化的 endpoint，最后产出一个 markdown 文件，然后 Devin 读这个 markdown，看旧输出和新输出分别是什么，自己做判断。

[15:48] Peter

Got it. Okay. Got it. All right, man. Well, do you want to show us your Devon setup and maybe we can start with I guess what makes Devon actually unique different from like, you know, codeex and cloud code.

明白，明白。好了老兄，那你要不给我们展示一下你的 Devin 配置？也许可以先从这个讲起：Devin 到底有什么独特之处，跟 Codex、Claude Code 有什么不一样。

[16:01] Jared

Totally. Happy to. Happy to. So maybe maybe I'll give a brief before diving into the share screen, maybe I'll give a brief overview. So I think there's a few things that make Devon unique. Maybe high level is the easiest way and philosophically is the easiest way to explain it first. So I think the first thing is at cognition our main focus is software engineer. So we I use Devon for a lot of stuff like design and whatnot, but like that's not our main focus. Our main focus is how do you write rigorous real software and how do you improve the software engineering life cycle and that means we have a lot of time spent on testing on on verified code on how the agent verifies its own code on computer use and stuff like that. And uh I think we really stand out if you are doing hardcore software engineering and you need to write code that doesn't break. And that's why we have so much traction in kind of like Wall Street banks and a lot of these Fortune 100 enterprises because their code can't break and they're doing this real kind of not vibe coding. they're doing coding and uh I think uh that so that's like I think everything kind of boils down from there and from there there's a few assumptions we've made that I think stand out so we've been building like Devon as a cloud agent from day one and that means we focus a lot on asynchronous work so you I guess there's two types of work and I think we all probably are starting to experience this synchronous work when you're behind your desk at work and you're doing a single stream. So maybe you're designing the landing page for something and you're going back and forth with it or async work which is you're working on eight different things and you're you're managing this agent and then going back and managing this agent and I I think this cloud agent paradigm is the future. I think it's everyone's starting to agree that it's the future. It's just faster and and it enables a lot of stuff in Devon that I think is pretty unique such that when I started when I started at Cognition I spent a lot of time talking with everyone on the edge like every single engineer like how do you use Devon? What's your setup? What's your trick? And one of the best things I heard and the biggest shift for me and this is what I'll kind of show you is having Devon set spin up a bunch of other child Devons and be the orchestrator of other agents. And the cool thing is every child Devon is its own VM. So it has its own computer. It has its own computer use. It has a display. It's able to test its own code. And then this master agent is able to pull down those other child dev and send messages to them. So I think cloud agents is our one way we stand out just the maturity of that product. I think the ecosystem as well. So we have Devon desktop which is our new kind of newly launched desktop product. We have the Devon CLI. We have the Devon cloud. And they all talk to each other. And it's really I think that's the second big way we stand out. It's a multiplayer tool. It's a tool for the team as opposed to just an individual developers kind of like Switchblade. And uh

当然，乐意之至。进入共享屏幕之前，我先简单讲个概览。我认为 Devin 有几点独特之处，先从最高层面、最哲学的角度讲最容易。第一点：在 Cognition，我们的主攻方向是软件工程。我自己也用 Devin 做设计之类的很多事，但那不是我们的主业。我们的主业是：怎么写出严谨的、真正的软件，怎么改进整个软件工程生命周期。这意味着我们在测试、在经过验证的代码、在 agent 如何验证自己的代码、在 computer use 这些方面投入了大量时间。如果你做的是硬核软件工程、需要写不能出问题的代码，我们真的很突出。这也是为什么我们在华尔街的银行和很多财富 100 强企业里有那么大的吸引力——他们的代码不能崩，他们做的不是 vibe coding，是真正的编码。我觉得一切都是从这里推导出来的。在此之上我们做了几个假设，我觉得很能体现差异。我们从第一天起就把 Devin 做成云端 agent，这意味着我们非常关注异步工作。工作大概有两种，我想大家都开始体会到了：同步工作——你坐在工位上做单一任务流，比如设计一个落地页，跟它来回互动；异步工作——你同时推进八件不同的事，管一下这个 agent，再回去管另一个 agent。我认为云端 agent 这种范式就是未来，大家也都开始认同这一点。它就是更快，而且解锁了 Devin 里很多我觉得相当独特的东西。我刚加入 Cognition 时，花了很多时间和每一位工程师聊：你怎么用 Devin？你的配置是什么？你有什么技巧？我听到的最棒的一条、对我而言最大的转变——也是待会我要演示给你看的——就是让一个 Devin 去启动一堆子 Devin，充当其他 agent 的指挥者（orchestrator）。妙的是每个子 Devin 都是独立的 VM：它有自己的电脑、自己的 computer use、有显示器、能测试自己的代码。然后这个主 agent 能拉取这些子 Devin 的进展、给它们发消息。所以云端 agent 是我们的第一个突出点——就是这个产品的成熟度。第二是生态：我们有新发布的桌面端产品 Devin Desktop，有 Devin CLI，有 Devin 云端，它们彼此互通。这是我们第二个突出的大点：它是一个多人协作（multiplayer）工具，是给整个团队用的工具，而不只是单个开发者手里的「瑞士军刀」。

[19:22] Peter

Oh, it's like a it's like multiplayer as in like actual other human multiplayer, not not just a bunch of agents, right?

哦，你说的 multiplayer 是指真的有其他人类参与的多人协作，而不只是一堆 agent，对吧？

[19:28] Jared

Yeah. Yeah. Yeah. I mean, it's agent agentic multiplayer with human multiplayer. And yeah, it it's super useful because you can save kind of macros and knowledge and skills for your whole team, but you can also inter Devons can interact with other people's Devons and you can you can see theirs. And there's a lot of kind of that like sharability enterprise features there. And honestly, the biggest surface people use Devon on is Slack and and automations. And uh actually I'll add one third thing and then and then we'll go into the demo. The third thing you should understand about like why Devon is different and it kind of goes with this async work but we really believe and it's already happened this month I think had a big switch uh more Devons are being launched from other Devons or programmatically than by humans meaning automation. So every time a data dog alert goes on you can pipe it to Devon or Slack just responding to customers. So, we believe in the same way that like during the self-driving car phase, uh it wasn't a not a phase, but during the self the beginning of self-driving cars, people had a lot of GPUs, and you wouldn't want the GPU to sit idle because you're losing money. In the same way, if I'm a developer, I'm going to sleep. I want my agents doing stuff. I don't want

对对对。是 agent 层面的多人协作加上人类的多人协作。这非常有用，因为你可以为整个团队保存类似宏的东西、知识和 skill；而且你的 Devin 可以和别人的 Devin 交互，你也能看到他们的。这里有很多可分享性和企业级功能。说实话，大家用 Devin 最多的入口其实是 Slack，还有各种自动化。我再补充第三点，然后我们进演示。理解 Devin 为什么不同的第三件事——它和异步工作一脉相承——是我们坚信、而且这个月已经发生了一个大转折：由其他 Devin 或程序自动启动的 Devin，已经比人类手动启动的多了，也就是自动化。每次 Datadog 告警响起，你可以把它接给 Devin；或者在 Slack 里直接回复客户。我们的信念是：就像自动驾驶汽车刚起步的那个阶段，大家手里有很多 GPU，你不会想让 GPU 闲着，因为那是在亏钱。同样地，我作为开发者，我去睡觉了，我也想让我的 agent 继续干活，我不想——

[20:52] Jared

I don't want the human to be the bottleneck anymore. And that's kind of this whole async cloud ecosystem comes together. Does that make sense?

我不想让人类再成为瓶颈了。这整个异步云端生态就是这样合到一起的。这样讲清楚了吗？

[21:01] Peter

Um, I think it makes sense if you're on these like all you can eat $200 plans, but like you know, if you're paying per API, do do you think it still makes sense? Like Uber is like capping spend of their engineers and stuff.

嗯，我觉得如果你用的是那种 200 美元「无限畅用」的套餐，这么干是说得通的。但如果你是按 API 用量付费，你觉得还划算吗？像 Uber 就在给工程师的花费设上限之类的。

[21:13] Jared

Yeah, it's a great question. Uh, the answer is yes to be honest. Uh, I two reasons. one, and this goes back to the beginning of our conversation, you should assume that we're going to be getting I think models are going to get both more expensive and cheaper, but the prices of regular tokens is probably going to go down over time. So, it's depreciating. But the second reason I think is more important, and this is kind of a c this is actually what really convinced me to join Cognition. I really like and Scott talks about this a lot, our view that we're not trying to replace coding. We're trying to build abundance of good software and really multiply an engineer. And for that reason, you want the engineer to bring taste and bring this like highlevel decision- making, but agents should be doing everything else and the agent shouldn't be bottlenecked on the engineer. So if you are bringing in ROI and bring and solving real problems, it's worth higher token spend, but it's not it's not worth it if the agents are doing stupid things and are not worth it. But like like I think

这是个好问题。说实话，答案是划算的，有两个原因。第一，这又回到我们对话开头说的：你应该默认模型会同时变得更贵和更便宜，但普通 token 的价格大概率会随时间下降，所以这块成本是在贬值的。第二个原因我觉得更重要，这其实也是真正说服我加入 Cognition 的一点。我很认同我们的理念——Scott 也经常讲这个——我们不是要取代写代码这件事，而是要创造优质软件的「富足」，把一个工程师的产出放大很多倍。正因为如此，你希望工程师带来的是品味和高层次的决策，其余一切都该交给 agent 去做，而且 agent 不应该被工程师卡住瓶颈。所以只要它在带来 ROI、在解决真实问题，花更多 token 就是值得的；但如果 agent 在干蠢事、产出不值钱，那就不值。不过我觉得……

[22:25] Jared

there are a lot of companies I think most companies in technology if they could pour more money into engineering and have outsized impact they would.

有很多公司——我觉得科技行业里大多数公司——如果能往工程上投更多钱并获得超额回报，他们是愿意投的。

[22:36] Peter

Yeah. But

对。不过——

[22:38] Peter

yeah, I think the key word there is good, right? Good. Good engineering because because you know like I I can also generate like a ton of slop that like a bunch of apps that are not good. And is that like use of tokens? I'm I'm not sure.

对，我觉得这里的关键词是「好」，对吧？好的工程。因为你知道，我也能生成一大堆垃圾——一堆质量不行的应用。那算是对 token 的合理使用吗？我不太确定。

[22:51] Jared

100% 100%. And as like as a a harness that's kind of independent from the model labs, the nice thing is we don't h we have the we don't have the incentive to make users spend more money because we don't have our like revenue tied to our next version of GPT or this or that. We just want users to use the best model for the best thing whether it's a cheap or the cheapest too. And it's kind of all trade-offs like how good is the model, how expensive is the model. And yeah, it's I think the incentives are aligned. I I but I but you're you're 100% right. You don't want to spend money on slop, but if you can spend money on real engineering work, it's worth it 10 out of 10 times.

百分之百同意。而作为一个独立于模型实验室的 harness，好处在于我们没有动机让用户多花钱，因为我们的营收不跟下一代 GPT 或者别的什么模型绑定。我们只希望用户用最合适的模型干最合适的事，哪怕是便宜的、甚至最便宜的那个。这里面全是权衡：模型有多好、模型有多贵。所以我觉得我们的激励是对齐的。但你说得完全对——钱不该花在垃圾产出上；可如果能花在真正的工程工作上，那十次里十次都值。

[23:36] Peter

And like, you know, we should get to the demo, but I I feel like inside companies, a lot of the real engineering work is not like zero to one fee features. is like, you know, refactoring and like, you know, trying to improve existing product like a really complicated messy code base like that kind of stuff, right? So, yeah.

对了，我们待会该进 demo 环节了。不过我感觉在公司内部，很多真正的工程工作并不是从零到一做新功能，而是重构，是在一个特别复杂、特别乱的老代码库上改进现有产品，这类事情，对吧？

[23:53] Jared

Yes. And I think I think Devon specifically as a harness is really shines on brownfield tasks as opposed to green field. So like our biggest users are using us for large scale code migrations or bringing more like coverage, test coverage or kind of just these these developer grunt work you can kind of call it and the stuff that developers like if it's

没错。而且我认为 Devin 作为一个 harness，恰恰是在 brownfield（存量项目）任务上大放异彩，而不是 greenfield（全新项目）。我们最大的那些用户都在用它做大规模代码迁移、补测试覆盖率，或者说就是那些开发者眼里的「苦力活」——那些开发者自己……

[24:24] Peter

Yeah.

对。

[24:25] Jared

Yeah. It's not it's not not the most like you're not designing the parallax landing page and it can do that too, but it can do this hard boring stuff that is a blocker for your team. And there's a lot of you'd be surprised how many big companies in the country have about 10 people who understand their codebase out of thousands. and you you it's because it's in a cobalt or it's in a mainframe or it's in this old technology and you need to modernize it. Yeah.

对，这不是那种最光鲜的活——你不是在设计带视差效果的落地页（虽然它也能做），而是在啃那些又难又无聊、却卡着整个团队的硬骨头。你可能想不到，这个国家有多少大公司，几千名员工里真正懂自家代码库的只有大概十个人。原因就是代码写在 COBOL 里、跑在大型机上，或者用的是某种老掉牙的技术，你必须把它现代化。对。

[24:58] Peter

Cool. All right, dude. Well, should we get through that demo then? Yeah,

酷。好了老兄，那我们开始看 demo 吧？

[25:01] Jared

let's do it. Let's do it. So, this is a project I'm actually working on. I haven't released it yet, but I'll probably hopefully we'll see who releases this first, right? But this is just something super simple. I wanted it for myself. you know when you do the space on Mac and it gives you the quick look. It it didn't doesn't work with markdown by default. So I just had Devon build a simple project for that but I think it would be good to illustrate how Devon works. So this is a Devon session. You could think of it as just kind of your cloud agent. And in this case it built it. We have an actual browser to see what's going on. So let's wake this up because this was I was working on this a bit ago. But you could see what's going on in the actual VM that the computer has. I could see the changes made and this is kind of this whole agent interface. So now you can see it's awaiting instructions, but I love this test app feature. So Devon test, I waited to click this button so we could do it together. This sometimes takes a little time, but Devon Devon's now spinning back up the VM and it's going to actually do an integration test and click on stuff and make sure all the links work in this landing page that I'm building. So, you can see it's entering test mode and it's clicking. We can kind of

来吧来吧。这是我正在做的一个项目，还没发布——到时候看咱俩谁先把它发出去，哈哈。就是个特别简单的东西，我自己想用：你知道在 Mac 上按空格键会弹出「快速预览」吧？它默认不支持 Markdown。所以我就让 Devin 给我做了个小项目来解决这个，不过我觉得正好可以用它来演示 Devin 是怎么工作的。这就是一个 Devin session，你可以把它理解成你的云端 agent。这个项目就是它做的。我们有一个真实的浏览器可以看到发生了什么。我先把它唤醒，因为我之前搁置了一会儿。你可以看到它那台虚拟机里实际在发生什么，能看到做了哪些改动——这就是整个 agent 界面。现在它显示「等待指令」，但我特别喜欢这个「测试应用」的功能。我特意没点这个按钮，就是想留着咱们一起点。这有时会花点时间，Devin 现在正在重新拉起 VM，接下来它会实际跑一个集成测试，在我正在做的这个落地页上到处点，确保所有链接都能用。你看，它进入测试模式了，正在点击。我们可以……

[26:19] Jared

see what's going on, what's being clicked. But this is kind of as you see, I think a lot of people understand what an agent harness is, but this is an agent harness in the cloud that has a browser and has a full computer and has full computer use and has a bunch of other stuff. Uh so, uh does that make sense?

看到正在发生什么、在点哪里。就像你看到的——我觉得很多人已经理解什么是 agent harness 了，但这是一个跑在云端的 agent harness：自带浏览器、有一台完整的电脑、有完整的 computer use 能力，还有一堆别的东西。这样讲清楚了吗？

[26:43] Peter

Why don't why don't you pitch to the viewers here what's so great about being in the cloud versus local, you know, like

要不你给观众们讲讲，跑在云端比跑在本地到底好在哪？

[26:49] Jared

Totally. Well, first of all,

当然可以。首先——

[26:51]

what's great about

好在哪呢——

[26:52] Jared

you can close your laptop.

你可以合上笔记本电脑。

[26:54] Peter

Yeah. Yeah.

哈哈，对对。

[26:56] Jared

If it's in the cloud, you know, this is kind of a a meme these days that everyone's leaving their laptop open all day. With cloud, you you can just set it and forget it and it's running on a different system. But what's even better is if I'm doing something really big, like let's say I want to redesign the landing page. Can you spin up let's say 10 child devon sessions? Each one should create its own redesign. And you can see it's testing it out over here is the live view. But and I can also control it if I want to. But

如果它在云端——你知道最近有个梗，说大家为了跑 agent 一整天都不敢合笔记本。有了云端，你设置好就可以不管了，它在另一套系统上跑着。但更妙的是，如果我要干一件大事，比如说我想重新设计这个落地页——「能不能起 10 个子 Devin session？每个各自做一版重新设计。」你可以看到它在这边测试，这是实时画面，我想接管的话也可以手动控制。但是——

[27:39] Jared

so what I'm asking this this is kind of a trivial example, but I'm I'm saying maybe I don't like this design of this landing page. I'm asking it to spin up 10 other VMs or 10 other child devans. And child devon is like you could think of it as a full software engineer. And in this case,

我刚才提的这个请求其实是个很小儿科的例子，但意思是：假设我不喜欢这个落地页的设计，我就让它起 10 台别的 VM，也就是 10 个子 Devin。子 Devin 你可以把它当成一个完整的软件工程师。在这个例子里——

[27:57] Jared

I can't really you can imagine I've done projects where I've had 30 different devins at the same time. You can imagine locally this is not so fun. My computer probably can't handle this especially with multiple VMs. But even more so it's it's just asyncing. I I can spin it up from my phone. I can spin it up by messaging it on Slack. I've actually talked to startup teams where the CTO told me they do most of their work just kicking off Devon's on the commute to work and then once they get to work, they pull down the repo, they click around, make sure it's good. Sometimes they just watch the video from Devon and they push it. And I got it. I'm very bullish on cloud agents. I would say also I found myself also doing a hybrid approach where I have a local agent that spins up its cloud agents and hands off sessions to the cloud and then pulls it down and kind of monitors them so I can start to work synchronously when I want to do that.

你可以想象——我做过同时跑 30 个 Devin 的项目。你想想这要在本地跑得多难受，我的电脑多半扛不住，尤其还要开多台 VM。但更重要的是异步性：我可以在手机上把它拉起来，可以在 Slack 上发条消息就把它拉起来。我真的跟一些创业团队聊过，有位 CTO 告诉我，他们大部分工作就是在通勤路上把一堆 Devin 任务派出去，到了公司再把仓库拉下来，点一点确认没问题——有时候干脆只看 Devin 录的视频，然后就直接 push 了。所以我非常看好云端 agent。另外我发现我自己还会用一种混合模式：本地跑一个 agent，由它去拉起云端 agent、把 session 交接到云上，然后再拉回来盯着它们，这样我想同步干活的时候随时可以切进去。

[28:57] Peter

Okay. So basically like uh this whole project is in GitHub somewhere and then you basically spun up a bunch of work trees like clones to redesign.

OK，所以基本上就是：整个项目放在 GitHub 上，然后你拉起了一堆 worktree——相当于克隆——去做重新设计。

[29:05] Jared

Well, in this case it's not even work trees because they're on their own VM. So they're just pulling it down and they're making a new branch and making a new PR and then the Devon can kind of talk to these subdevons. But this is a I think this is a Devon specific paradigm of how to do real AI pill work and it's by doing this kind of agent fan out. I've talked to people on our engineering team who are doing kind of like the model training and they tell me they spin up a hundred Devons to look for things because

这个例子里其实都不用 worktree，因为它们各自在自己的 VM 上。它们就是把代码拉下来、开一个新分支、提一个新 PR，然后主 Devin 可以跟这些子 Devin 对话。我觉得这是 Devin 特有的一种范式，是真正「All in AI」式工作的做法——靠这种 agent 扇出（fan out）。我跟我们工程团队里做模型训练的人聊过，他们说会一次拉起一百个 Devin 去排查问题，因为——

[29:38] Jared

agents are much better when they're doing one thing specifically and kind of really focus just like humans are to be honest.

agent 在专注只干一件具体的事时表现会好得多——说实话，人也一样。

[29:47] Peter

So basically the master Devon can actually go back and forth with some of these other devons to make the landing page good, right? It's not

所以基本上，主 Devin 其实可以跟这些子 Devin 来回沟通，把落地页打磨好，对吧？它不是——

[29:53] Jared

like they're going to go through this whole checklist but it can go back and forth. Yeah,

倒不是说它们会走完一整套 checklist，但确实可以来回沟通。对。

[29:56]

the master Devon if I say um check or we could say make sure each child Devon responds with a screenshot. Send this message. It'll now like a human would to the agent. It'll prompt each of these Devans with a new message that'll say something like this. So it you see it has some way to interact with them. Okay, this is actually this is actually really really good, dude. So, I've I've been using, you know, these AI coin tools and I've been chatting with them across different threads, but it's it's me chatting with them. I haven't really kind of adopted this pattern of doing a master child thing. I mean, this is kind of a trivial example, but like in reality, like what are some good use cases for this master child thing?

（Jared）主 Devin 这边，如果我说——比如我们输入「确保每个子 Devin 都回一张截图」，发送这条消息。它现在就会像人跟 agent 对话那样，给每个子 Devin 发一条新的 prompt，内容大概就是这样。所以你看，它有办法跟它们互动。（Peter）OK，这个真的非常非常棒，老兄。我一直在用这些 AI 编程工具，也会在不同的对话线程里跟它们聊，但都是「我」在跟它们聊，我还真没用过这种主-子（master-child）模式。这个例子算是比较小儿科，那在实际中，这种主-子模式有哪些好的用例？

[30:47] Jared

Yeah, you know,

嗯，你知道——

[30:49] Jared

so a migration is one of the most obvious ones. Well, before I explain it, let me let me show you one of these message sends. So, we can we could go back to our agents and we can open one of these. I think it's sent to this one. And you could see here it it impersonated this is my like username on the the org, but it sent this message over and it's working on this new redesign. But

代码迁移是最典型的用例之一。不过在展开讲之前，先给你看一条它发出去的消息。我们回到 agent 列表，打开其中一个——我记得是发给这个的。你可以看到，它是以我在这个组织里的用户名的身份发的，把这条消息发了过去，而这个子 Devin 正在做新的重设计。但是——

[31:16]

it yeah

它……对。

[31:17] Jared

but uh but yeah anyway to answer your question the the the real world use cases of this ancient fanout paradigm is when you're doing something that is a very large project for example a huge migration of a codebase from let's say cobalt or for from honestly from JavaScript to TypeScript or from uh or React Native to Swift. And this is really useful because you want you want to break up the work. Well, it's useful for two reasons. One, you could parallelize things so it's just way faster to do it.

不过回到你的问题，这种 agent fanout（扇出并发）范式在现实里的用武之地，是那种规模非常大的项目，比如整个代码库的大迁移——从 COBOL 迁出来，或者说实话从 JavaScript 迁到 TypeScript，从 React Native 迁到 Swift。这时候它特别有用，因为你需要把工作拆开。好处有两个：第一，可以并行处理，速度快得多。

[32:01] Jared

But two, you could break up the work to keep the context window small so that each agent

第二，拆分工作可以让每个 agent 的 context window 保持在很小的范围内，

[32:06] Jared

has a very specific focus, can work on testing that thing. And it's part of this thing that kind of I started explaining of Devon and Cognition as a company. We care about agents writing verifiable code and writing code that works. And part of that is breaking it down into chunks that are verifiable and testable and not just these agents. These the all these 10 a 10 Devon agents are not just writing code and returning it, they're writing code, they're running it, they're taking screenshots, they're looking at it. It's really a full-fledged teammate

让每个 agent 都有非常明确的专注点，可以针对自己负责的那块做测试。这也是我前面开始讲的 Devin 和 Cognition 这家公司的理念的一部分：我们在意的是让 agent 写出可验证的代码、写出真正能跑的代码。其中一环就是把任务拆成可验证、可测试的小块。这 10 个 Devin agent 不是写完代码一交了事——它们写代码、运行代码、截图、自己检查结果。它真的是一个全能的队友。

[32:47] Peter

and and that is already baked into like the just the product, right? I don't have to do some crazy skills or prompts to make it work work that way.

而且这些能力已经内置在产品里了对吧？我不需要搞什么复杂的 skills 或 prompt 才能让它这样工作。

[32:54] Jared

Correct. This is a core feature and uh then you could get really cool when you're using Devon desktop locally or Devon CLI locally and it's interacting with these cloud agents and and it's handing off work to the cloud and parallelizing that way. And I think this is I'd say where the world is going. I I'd say this is probably like the advanced usage of coding agents. don't have to do all this stuff, but uh

没错，这是核心功能。更酷的玩法是，你在本地用 Devin 桌面版或 Devin CLI，让它跟这些云端 agent 交互，把工作移交给云端并行处理。我觉得这就是整个行业的走向——这算是 coding agent 的进阶用法吧，你不必非得这么玩，但……

[33:19] Jared

yeah,

对，

[33:19] Jared

it's it's helped me a lot and it's it's been it's been a big learning as I moved away from all the other agent products into Devon when I started at Cognition.

它帮了我很大的忙。我加入 Cognition 后，从其他所有 agent 产品切换到 Devin，这是我最大的收获之一。

[33:31] Peter

Interesting. Yeah. So, I was going to make a joke that this is how you token max, but may maybe actually save tokens because like each each one only has like a smaller context window than than like trying to do it all in one thread.

有意思。我本来想开个玩笑，说这就是「token 消耗拉满」的玩法——但其实这样可能反而省 token，因为每个 agent 的 context window 都比在一个线程里干完所有事要小。

[33:42] Jared

Totally. I I mean I'm I'm still a proto token maxer for what it's worth, but yeah, you do save and I think we we're working on so we have our own model that we build called suite and it's specifically designed for software engineering and it's designed to be much quicker and much cheaper. So the the the other pretty cool thing about Devon that I like a lot is I don't have to keep track of which is the best model that exists because we have a pretty cracked engineering team and they're spending a lot of time on eval. spending a lot of time on this sort of work and they route it to the best one. And we're starting to think a lot more about how do we do the trade-off between between cost and time and efficiency and and intelligence such that maybe some of these could be traded spun up with a smaller models. And this is I mean you see there's no model selector here. So, this is something that you let Devon handle.

完全正确。说实话我自己还是个 token 消耗大户，但确实能省。而且我们正在做的一件事是：我们有自研的 SWE 系列模型，专门为软件工程设计，目标是更快、更便宜。Devin 还有一点我特别喜欢——我不用自己操心「现在哪个模型最强」，因为我们有一支非常猛的工程团队，他们花大量时间做 eval、做这类工作，然后自动把任务路由到最合适的模型。我们也在深入思考成本、时间、效率和智能之间怎么权衡，比如某些任务也许可以交给更小的模型跑。你看，这里根本没有模型选择器——这件事你直接交给 Devin 处理就行。

[34:48] Peter

Oh, interesting. You know, I always get kind of suspicious some sometimes when companies do this because like I I just I I just think they're trying to save cost, but um maybe maybe that's not true. Maybe maybe it's

哦，有意思。说实话，公司这么做的时候我有时会有点怀疑——总觉得他们是想省成本。不过也许不是这样，也许……

[34:59] Jared

Well,

这么说吧，

[35:00] Jared

I'll tell you today, by default, it's using the best models for everything. We we have features that are currently in development and should be out soon that in this kind of in the for cloud agents specifically can be handed off to more light versions for simpler work to save money again because we're independent from the labs. We don't make like our goal is not for you to token max. Our goal is for you to ROI max and uh so so it's for I think it's not for us to save money. And then also if you're using Devon, like if you're using a local agent, you can choose the model yourself.

我可以告诉你：今天的默认设置是所有任务都用最好的模型。我们有一些正在开发、很快上线的功能，专门针对云端 agent，可以把简单的活儿交给更轻量的版本来省钱——省的是你的钱，因为我们独立于各大 lab，我们的目标不是让你「token 拉满」，而是让你「ROI 拉满」。所以这不是为了我们自己省钱。另外如果你用的是本地 agent，模型也可以完全自己选。

[35:43] Peter

Got it. This is this is this super helpful. Um and so so so then going back to the keeping things simple thing when when you build stuff with Devon, it looks like you just give it like pretty casual commands, right? Like do you have like a bunch of like agent MD and like you know like a bunch of custom skills set up or you just kind of

明白了，这些非常有帮助。那我们回到「保持简单」这个话题：你用 Devin 做东西的时候，看起来就是给它下一些挺随意的指令对吧？你有没有配一堆 agent.md、一堆自定义 skills 之类的，还是就……

[36:02] Jared

Yeah. So, we have a

嗯，我们有一些……

[36:05] Jared

So, we have a few other features. I didn't actually quite show you that closely. We have some things you can add here, like we have some knowledge, playbooks, macros, skills, that sort of thing. And what what those are are uh the goal of all those is and I'm just getting distracted by these screenshots coming in, but the goal of all those is to like share knowledge amongst the team. So by default, I think the best thing Devon does is it remembers how to set up environments. So if you have this whole local environment set up to run your app, it it'll do it once and then it'll save what it learned. So it could be much quicker next time. We also have kind of ask. So ask I can jump to this one. This was a test I just ran for you guys on OpenClaw repo. It's kind of we we pre-indexed the repo to make it easier to just ask questions. And this is where you can kind of do plan mode and ask things and have it write a plan that you can hand off to a real devon session. Uh but yeah, for skills macro I mean I use them all is the short version.

我们还有几个功能刚才没细展示。你可以在这里添加一些东西，比如 knowledge（知识）、playbooks、宏、skills 之类的。这些功能的目标——我有点被这些不断弹进来的截图分心了——这些功能的目标都是在团队内部共享知识。默认情况下，我觉得 Devin 做得最好的一点是它会记住怎么搭环境：如果你有一整套本地环境才能跑起你的应用，它做过一次之后就会把学到的东西存下来，下次就快多了。我们还有 Ask 功能——我跳到这个例子，这是我刚为你们跑的一个测试，用的是 OpenClaw 这个仓库。我们把仓库预先索引好，方便你直接提问。你可以在这里进行类似 plan mode 的操作，提问、让它写一份计划，然后把计划交给一个真正的 Devin session 去执行。至于 skills 和宏——简短版的回答是：我全都在用。

[37:16] Peter

Okay, got it.

好，明白了。

[37:17] Jared

I think uh this is like obviously a trivial example because this is not actually like this is just a landing page. Doesn't need to run anything. Just needs to make sure links work. But

当然这显然是个很简单的例子，因为这只是个落地页，不需要跑什么东西，只要确保链接都能用就行。但是……

[37:27]

yeah,

对，

[37:27] Jared

I think where Devon really shines is like on real engineering and here's their automations section where you can hook it up to the other work and you can unblock it on humans and we have kind of a wiki deep wiki which is also something people love where it's creating and creating docs for your repo and I think if you're in a company with a really large codebase some of these companies don't have docs for their repo and it's just an instant win. Um,

我觉得 Devin 真正大放异彩的地方是真刀真枪的工程工作。这是它的 automations（自动化）板块，你可以把它接到其他工作流上，让它不被人卡住。我们还有 DeepWiki，这也是大家很喜欢的功能——它会为你的仓库自动生成文档。如果你在一家代码库很庞大的公司，很多公司的仓库压根没有文档，这就是立竿见影的收益。

[37:56] Peter

oh, just quiz docs automatically.

哦，自动生成文档。

[37:58] Jared

Yeah. Yeah. And that's part of the indexing process on a repo. So maybe let me let me open one. Let's do this is a little repo I made. I'm having Devon look for an apartment for me in San Francisco. So, but it's creating docs of how this process works. And this is obviously a trivial example, too, but you can see kind of

对对，这是仓库索引流程的一部分。我打开一个给你看——这是我自己做的一个小仓库，我在让 Devin 帮我在旧金山找公寓。它会把这个流程的运作方式写成文档。这当然也是个很简单的例子，但你能看出来……

[38:20] Jared

how that would

这在实际中会是什么样子。

[38:21] Peter

All right, dude. Well, let me ask you a couple of questions to close our interview. So I think um yeah I do think like you know obviously the space is very competitive. I I would love to get your take on you know like for example codex and clock code they're compete against each other. They're both trying to expand into knowledge work beyond coding. Uh then you have this whole class of like more beginner friendly like lovable replet and so on. And then you have more hardcore tools for coding like Devon and maybe Factory and some of these other tools, right? Like obviously you work at Devon, so so maybe you're a little bit biased, but like how do you think this whole thing is going to play out, dude? Like do you think all these players will continue to exist or do you think this will be a consolidation or Yeah,

好的老兄，那我问几个问题来收尾。我觉得这个赛道显然竞争非常激烈。我很想听听你的看法，比如 Codex 和 Claude Code 在互相竞争，两者都想从编程扩展到更广泛的知识工作；然后还有一整类对新手更友好的产品，比如 Lovable、Replit 等等；再往上是更硬核的编程工具，比如 Devin，可能还有 Factory 和其他一些工具。你在 Devin 工作，可能会有点立场，但你觉得这整个格局会怎么演化？这些玩家都会继续存在，还是会走向整合？

[39:05] Jared

it's a good question. First of all, I think I think the market's bigger than people realize. And this is not to say it won't be a winner or take all, but it's just I'll tell you the reason I joined Cognition was I kind of realized that there's one problem that matters today and it's building a coding agent and all the biggest company all the labs are working on it. We're we're we consider ourselves an agent lab. we're working on it and just the it's just the power of this is such a big problem in such a big market and if you solve solve but if you build the a great coding agent it's it's just the 80% it's a power law when it comes to basically any other problem and basically every other problem is downstream of that so that's why I think everybody's rushing for it and that's why I think there's so much competition pred Predictions are hard. I'm really bullish on non-technical work with agents. I I I think one on one hand it's very smart that we're focusing on software, but I I use agents for a lot of nontechnical stuff, too. So, I don't think that's going away. I think uh there's a lot of great work by the labs doing it and by non-labs. If I had to say something I'm more doubtful of is kind of these beginner first beginner only build version one project products the kind of like lovables and and and the wixes of the world. I I think it I'm I love the idea of dem democratizing coding and letting more people to do it. I I think there's probably going to be a bifurcation where we're going to have hardcore engineering, real engineering, and then like play engineering. And I don't mean that in a derogatory word way either. I mean it to say single-use software short my my friend is a founder and he's kind of started coining the term short form software. So kind of something that exists just for one time or a few times and it's not a business in the same way and I think it's a completely different paradigm. So I think I I haven't seen anything there's not really anything built for that in the same way they're building for hardcore engineering and end up there. I do think if anything like these tools are going to eat the market of like more traditional UI focused tools like you know like like a PowerPoint or like some of these more traditional tools right because like you just tell the agent to do the work and then the last mile maybe the agent can't help with too or like you can you can manually tweak tweak stuff. Oh yeah. I think I think there's a lot of artifacts that are going to be second tier artifacts that we don't care about anymore to be honest. Code this is a little more controversial, but I don't I internally we have debates on this to be honest, but I think code is going to be a second order artifact where you're going to be interacting with something higher level than that. PowerPoint, same thing for sure. I don't I don't I create all my slides with a with agents now. video editing. I think there's a lot of point andclick. I mean, if you want my radical opinion,

好问题。首先，我认为这个市场比人们想象的大得多。这不是说它不会赢家通吃，但我可以告诉你我加入 Cognition 的原因：我意识到当下只有一个真正重要的问题，就是打造 coding agent，所有最大的公司、所有的 lab 都在做这件事。我们把自己定位成一个 agent lab，也在攻这个问题。它的威力在于：这是一个巨大市场里的巨大问题，如果你造出了一个顶级的 coding agent，那就是幂律分布里的头部——基本上其他所有问题都是它的下游。所以所有人都在抢，竞争才这么激烈。预测很难。我非常看好 agent 在非技术工作上的应用——一方面大家聚焦软件是很聪明的，但我自己也用 agent 干很多非技术的活，所以这个方向不会消失，各家 lab 和非 lab 公司都在做很棒的工作。要说我更怀疑的东西，是那些「只面向新手、只做 1.0 版项目」的产品，就是 Lovable、Wix 这类。我很喜欢「让编程平民化、让更多人能写代码」这个理念，但我觉得未来大概率会分叉：一边是硬核工程、真正的工程，另一边是「玩票式工程」——我没有贬义，我指的是一次性软件。我一个朋友是创业者，他开始用「短内容软件（short form software）」这个词：只用一次或几次的软件，它不构成传统意义上的生意，是一个完全不同的范式。目前还没有什么产品是真正为这个场景打造的——大家都在为硬核工程构建，最后却落到了那里。（Peter：我倒觉得这些工具会吃掉更传统的 UI 型工具的市场，比如 PowerPoint 这类——你直接让 agent 干活，最后一公里再手动微调就行。）没错，说实话我认为很多产物会降级成我们不再关心的「二等产物」。代码——这个更有争议，我们内部也在争论——但我认为代码会变成二阶产物，你将会和比代码更高层的东西交互。PowerPoint 肯定也一样，我现在所有幻灯片都是用 agent 做的。视频剪辑也是，很多点击拖拽的操作都会被替代。如果你想听我最激进的观点——

[42:26] Jared

uh, and I'll give it quickly, is

我很快讲一下——

[42:28] Jared

Yeah. Is I think I could see a I think humans are going to return to the world where we just have a wooden desk and we don't have this whole monitor setup and keyboard setup and mouse. I think the mouse is kind of antiquated. I think the keyboard is antiquated. I don't think we need that. I just want a button and a whisper flow. And I think that's all I need.

我认为人类会回到只有一张木头桌子的世界，不再需要这一整套显示器、键盘、鼠标。我觉得鼠标已经过时了，键盘也过时了，我们不需要这些。我只要一个按钮加一个 Wispr Flow（语音输入），这就够了。

[42:51] Peter

Yeah. Okay. Got it. Interesting. Button with Well, you have to see what the product is, right? Don't you have to see the UI or you don't think there will be a UI anymore? Just

好，明白，有意思。一个按钮……但你总得看到产品长什么样吧？你不需要看 UI 吗，还是说你觉得以后根本不会有 UI 了？

[42:59] Jared

Yeah, that's the hard part.

对，这正是难点。

[43:00] Jared

I don't know. Yeah, there might not be a UI anymore. I think there's I think design and aesthetics matter a lot. So, there's probably still a world where you need some sort of monitor. But maybe I could say, what if it just prints it out and I could look at it there and then I don't need it. I I think there's a lot of unique stuff to be made here. But I think this whole concept of the ergonomic keyboard and the mouse, if your job is managing a fleet of agents all day and

我不知道，也许以后真的没有 UI 了。我认为设计和美学仍然非常重要，所以大概率还是需要某种显示器的世界。但也可以设想：直接打印出来，我在纸上看，那我就不需要屏幕了。这里有很多独特的东西可以做。但「人体工学键盘加鼠标」这整套概念——如果你的工作是整天管理一支 agent 舰队……

[43:26]

yeah,

对，

[43:27] Jared

you don't need that. And you're like, I barely type. I type a little. Uh, okay. I type half a day, but I'm typing less and less and I'm talking way more to agents. And I that's natural. I'm a human. I like to talk. I mean, not everybody likes whisper phobia. Some people are more yappers than others, but I just think the desk setup is a little unnatural. I don't like this two monitor thing I have going on. I'd rather just have

你其实不需要那些。然后你会发现：我几乎不打字了。打一点吧——好吧，半天在打字，但打字越来越少，跟 agent 说话越来越多。这对我来说很自然，我是人嘛，我喜欢说话。当然不是每个人都爱对着机器讲话，有些人就是比别人更能唠。但我就是觉得现在这种桌面配置有点别扭，我不喜欢我这套双显示器的搞法，我宁愿只要——

[43:52] Peter

I see. That's funny. That's funny because like, you know, like for example, Peter Steinberger, the the open cloud guy, like he has six monitors or something, right? But like I guess it makes sense because like if you're a manager, you're not looking at the monitors of six of your reports all all the time. You know, you just kind of give them directions and hopefully they do the work right.

明白，这挺有意思的。有意思在于，比如 Peter Steinberger——就是搞那些开源 Claude 项目的那位——他好像摆了六块显示器还是多少块，对吧？但想想也说得通：如果你是个管理者，你不会一直盯着六个下属的屏幕看，你只是给他们指方向，然后指望他们把活儿干好。

[44:07] Jared

Yeah.

对。

[44:08] Peter

Right. There's a lot of different workflows. So, I think that's something that maybe some workflows won't need it anymore, but I want them I want us to return to the madman style desk. Um, okay, let let me get like another maybe a hot tech and then we can wrap up. So, I've been struggling with this a lot, man. Like, I feel like so I' I've hooked up APIs and, you know, MCPS and whatever to, you know, all all these agents. So, now I I don't even go to the website of these apps anymore. I just like, you know, query my agents. And I feel like if I own one of these other product like these SAS products, it feels like I'm losing the relationship with the customer, right? Because like the customer is just talking to me through like cloud or some or or something, you know? It's almost like how do you how do you design a product to be agent first versus human first? It's like a pretty radical shift, you know, if you think about it that way.

是啊，工作流本来就五花八门。所以我觉得也许有些工作流以后就用不上多屏了，但我希望我们能回到《广告狂人》那种桌子——桌上一块屏幕都没有。嗯，好，我再抛一个尖锐点的话题，然后我们收尾。这事最近一直让我很纠结。是这样：我已经把 API、MCP 什么的全接到这些 agent 上了，所以现在我根本不去这些应用的网站了，直接问我的 agent 就行。那我就在想，如果我是这些 SaaS 产品的老板，感觉我正在失去和客户的关系，对吧？因为客户只是通过 Claude 之类的东西在跟我打交道。这几乎等于在问：产品该怎么设计成 agent 优先，而不是人类优先？你这么一想，这其实是个相当激进的转变。

[44:57] Jared

Yeah. Yeah. 100%. I think uh I think we're still trying to I think as a society we're figuring that out. I I guess my my version of my take in that world is that I think everything can be reduced to an API endpoint or API call. Meaning there's no reason an agent can't build a house because what does it take to build a house? You need to you need to probably buy the property. You need to zone it. You need to buy the supplies. You need to hire the people. You need to tell the people what to do with the supplies. you need to take pictures of it after the these can all be discrete steps that an agent calls out to some API somewhere to do and there's yeah there's I I can see agents building homes and building towns and and running businesses autonomously and the everything's an API endpoint and I I love Rent to Human I think was kind of a little bit of a like tongue and cheek it felt like but I'm unironic ally believe that that's a really good idea.

对，百分之百同意。我觉得整个社会都还在摸索这件事。我在这个问题上的看法是：一切最终都可以归结为一个 API endpoint、一次 API 调用。换句话说，没有理由说 agent 不能盖房子——盖房子需要什么？你得买地，得办规划许可，得买建材，得雇人，得告诉工人拿这些材料干什么，最后还得拍照验收。这些都可以拆成一个个独立步骤，由 agent 去调用某个 API 来完成。所以是的，我完全能想象 agent 自主盖房、建镇、经营公司——一切皆 API endpoint。还有那个 Rent a Human（租个人类），我感觉它当时多少带点玩梗的意思，但我是毫无反讽、完全认真地觉得那是个特别好的主意。

[46:06] Peter

It's hard as a company though because like if if I just make everything API endpoint and then you know clock and just get all my data the human doesn't have to talk to me.

但站在公司角度这就难了：如果我把一切都做成 API endpoint，然后 Claude 直接把我的数据都取走，人类用户就再也不用跟我打交道了。

[46:13] Peter

It's like what kind of like you know do I even have a brand left or is it just a bunch of APIs I'm giving away?

那就变成：我到底还有没有品牌可言？还是说我就只是在白送一堆 API？

[46:18] Jared

No, it's a great question. What what is brand? But maybe the brand is something for agents. Maybe it's you're writing it in markdown and like it's you find a way to convince agents and AEO and but yeah I don't

不，这问题问得特别好。品牌到底是什么？也许以后品牌是做给 agent 看的。也许你得用 markdown 把它写出来，想办法去说服 agent，做 AEO（面向 AI 引擎的优化）之类的。不过说实话，我——

[46:32] Jared

we'll figure it out. Yeah,

总会摸索明白的。是啊——

[46:33] Jared

I don't have a good answer. That's a good question though.

我没有一个好答案。但这真是个好问题。

[46:36] Peter

Cool, Jared. Well well I mean where can people find you and learn more about Devon?

好，Jared。那大家可以去哪里找到你、进一步了解 Devin？

[46:40] Jared

Yeah, you can find me. You can find me on X at X Twitter at I am Jaredz I am Jre DZ. Uh find Devon at Devon.ai cognition.ai AI is the company that builds Devon. And yeah, just start using it. You can sign up. You can try it out for free. You can use it for free. And let me know if you have any questions using it. I'm always eager to help help agent build people.

好，你可以在 X（Twitter）上找到我，账号是 @imjaredz——i、m、j、a、r、e、d、z。Devin 的话去 devin.ai，做 Devin 的公司是 Cognition（cognition.ai）。然后就直接上手用吧——可以注册，可以免费试用。用的过程中有任何问题随时来找我，我一直很乐意帮想构建 agent 的人。

[47:06] Peter

Cool. All right, Jed. Well, I'm gonna go off and spin up some agents now, but it was great chatting with you.

好的。行，Jared，那我这就去开几个 agent 跑起来了。跟你聊得非常开心。

[47:10] Jared

I love it. Well, thanks for having me.

太好了。谢谢你邀请我。