What product looks like when coding is solved | Geoff Charles (Ramp CPO)

频道: Lenny's Podcast
视频: https://www.youtube.com/watch?v=ZG8Mf3P9xzI
原文语言: en
统计: 共 18 轮 · Geoff 15

[0:06] Geoff

Hello everyone. First off, it's been absolutely amazing to see the speakers before me. I want to give one more round of applause for the incredible amount of content and preparation. I'm inspired. I've rewritten my speech three times. I learned how to do storytelling. And so here's a story for you. I learned from Ron about velocity. We love speed. I love speed. I love speed so much that I once signed up to race cars at Le Mans. I'm just kidding. Lemons. 24 hours of lemons. It's a race where you buy a $500 car and you race around for 24 hours hoping to win. Unfortunately for me, my car did not make it. I crashed. I crashed hard. I was okay. The car unfortunately was not. Now, this crash was entirely my fault as the driver. But in real professional racing, the driver is rarely the problem. In fact, the driver is only 15% of the impact on the race. The real impact is the interaction between the driver and the car, between the car and the team. In other words, winning isn't just about driving faster. The best drivers in the world can only perform up to the level of the system. Winning the race is about removing the bottlenecks around the driving. That's the key point I want to drive today. Take for example the pit stop. In the 1950s, it took 67 seconds to change tires.

大家好。首先，前面几位讲者的分享真的太精彩了。我想请大家再为他们鼓一次掌——内容量这么大，准备得这么充分。我深受启发，演讲稿已经重写了三遍。我学会了怎么讲故事，所以先给大家讲个故事。我从 Ron 那里学到了速度（velocity）。我们都爱速度，我也爱速度。我爱速度爱到什么程度？我曾经报名去勒芒（Le Mans）赛车。开玩笑的——是 Lemons，24 Hours of Lemons（恶搞勒芒 24 小时耐力赛的业余赛事，lemon 在英语俚语里指破车）。这个比赛是你花 500 美元买一辆车，然后连续跑 24 个小时，盼着能赢。可惜我的车没撑到最后。我撞车了，撞得很惨。人没事，车可就没那么走运了。这次撞车完全是我这个车手的错。但在真正的职业赛车里，问题很少出在车手身上。事实上，车手对比赛结果的影响只占 15%。真正起作用的，是车手与赛车之间、赛车与团队之间的配合。换句话说，赢比赛不只是开得更快。世界上最好的车手，也只能发挥到整个系统允许的水平。赢得比赛，靠的是清除驾驶之外的那些瓶颈。这就是我今天最想讲透的一点。拿进站换胎来说。在 1950 年代，换一次轮胎要 67 秒。

[1:48] Geoff

Today, 1.8 seconds. How did 67 seconds turn into 1.8? Certainly not by asking the mechanic to work 37 times harder. All you leaders, you know what I'm talking about. They found the bottleneck and they iterated to remove them. Specialized functions, better technology, more practice. And F1 is relentless. 90% of the parts of a car change every year. Only 10% of the 16,000 parts of an F1 vehicle carry over to the next year. Now, I want you to also think about what if 90% of your code changed every year? Because that is the bar. Why am I telling you all this? We're in a race. Speed is everything. Execution is everything. And we're in a race to build with AI. For the race car, it starts when the car comes in for new tires and the clock stops when it leaves. For us, the clock starts when there's pain and it leaves when the customer has a product that solves it. AI simply removes the bottleneck, but moves it. And the best team, the winning team, is the team that can find the bottleneck faster, remove it, and move on to the next one. Now, we all know the product development life cycle. We identify, we define, we spend a lot of time building, and then we improve. Now, the engineers, fantastic engineers, they've been able to automate a lot of their work and coding is now much easier.

而今天只要 1.8 秒。67 秒是怎么变成 1.8 秒的？当然不是靠让维修技师卖力 37 倍。在座各位管理者，你们都懂我在说什么。他们找到瓶颈，然后不断迭代，把瓶颈一个个消除：分工专业化、更好的技术、更多的练习。而且 F1 迭代起来毫不手软。一辆赛车 90% 的零件每年都会换掉——一台 F1 赛车有 16,000 个零件，只有 10% 会沿用到下一年。那也请大家想一想：如果你们 90% 的代码每年都换一遍，会怎样？因为这就是标杆。我为什么要讲这些？因为我们正身处一场比赛。速度就是一切，执行就是一切。而这是一场用 AI 构建产品的比赛。对赛车来说，赛车进站换胎时开始计时，出站时停表。对我们来说，客户一出现痛点就开始计时，直到客户拿到解决这个痛点的产品才停表。AI 只是把瓶颈消除了——但同时也把它挪到了别处。最好的团队、能赢的团队，就是能更快找到瓶颈、消除它、再转向下一个瓶颈的团队。我们都熟悉产品开发生命周期：识别问题、定义方案、花大量时间构建，然后持续改进。如今，工程师们——了不起的工程师们——已经把自己的很多工作自动化了，写代码变得容易多了。

[3:24] Geoff

So, here's a tip. Be more like engineers. Be as lazy as engineers. Because the reality is that the bottleneck has now shifted to us. We need more things to define. We need more things to collaborate. We need more things to coordinate, more things to ship, more things to test, more things to release. The bottleneck has shifted to us and so we need to invest in our own factory. The speed of this loop determines the outcome of the race. And the question becomes, how quickly can you identify the bottleneck and redesign the factory around it? What I'm going to show you today are five of these steps and how Ramp has gone far in iterating and automating through it. The first step is identify. Identify the problem. Identify the pain, right? The issue is that that pain lives somewhere and there's a lot of pain, a lot of data. Gong, Zendesk, LogRocket, a survey, very angry email to an executive. The problem is that context lives all over the place. Too many silos, too many opinions. And so the first bottleneck to identify is how to sift through the noise. And LLMs are great, but a 1 million token window, that's less than 0.5% of Gong transcripts at Ramp. So we start small. We created a hate channel and every day it posts all the lovely things that our customers say. And yes, these are real quotes.

所以给大家一个建议：多向工程师学学，像工程师一样「懒」。因为现实是，瓶颈现在已经转移到我们身上了。要定义的东西更多了，要协作的事更多了，要协调、要交付、要测试、要发布的东西也都更多了。瓶颈转移到了我们这儿，所以我们得投资建设自己的工厂。这个循环转得多快，决定了这场比赛的胜负。于是问题就变成：你能多快找到瓶颈，并围绕它重新设计你的工厂？今天我要给大家展示其中的五个环节，以及 Ramp 在这些环节上一路迭代、自动化，已经走了多远。第一步是识别：识别问题，识别痛点，对吧？麻烦在于，痛点藏在各个地方，而且痛点很多、数据也很多：Gong、Zendesk、LogRocket、一份调研问卷、一封发给高管的暴怒邮件。问题是上下文散落在各处：孤岛太多，意见也太多。所以要找出的第一个瓶颈，是怎么从噪音里筛出有用的信号。LLM 很强，但 100 万 token 的上下文窗口，连 Ramp 全部 Gong 通话转写的 0.5% 都不到。所以我们从小处着手。我们建了一个「吐槽频道」（hate channel），每天把客户说的各种「甜言蜜语」都贴出来。没错，这些都是客户的原话。

[4:45]

[snorts]

（笑）

[4:47] Geoff

That got out of hand really quickly.

这事很快就玩脱了。

[4:51]

[gasps]

（倒吸一口气）

[4:51] Geoff

Yes, I know. I am not proud of it. No one's perfect. And so we had to redesign from scratch. We built an actual customer insight agent. That agent pulls from all the data sources at a company. It uses traditional ETL and pipelines. It uses actual vector search. It has clustering around the same context, understands our product, and understands our teams, and understands the features that we have, and it's accessible to the rest of the organization. Once we had this working, we asked ourselves, "How do we get this to as many hands as possible?" And so we again experimented. A Slack agent where you can ask questions, a nice HTML dashboard that people can log in and read. My favorite one was the podcast. The hate podcast. Great way to start your day. You listen in, 100 customers yelling at you about all the ways your product is broken. The concept is simple, right? Get people to access to this data as fast as possible. That doesn't mean don't talk to customers, it means you actually identify exactly which customers to talk to. Because the data is traceable. Okay, once you've found what to focus on, the next step is to define your product. How do you go from you know, concept to a fully scoped idea?

是的，我知道，我并不以此为荣。人无完人嘛。所以我们只能推倒重来。我们做了一个真正的客户洞察 agent。这个 agent 会从公司所有的数据源拉取数据。它用了传统的 ETL 和数据管道，也用了真正的向量检索。它能把同一上下文的内容聚类到一起，懂我们的产品，懂我们的团队，懂我们有哪些功能，而且公司里其他人都能用。跑通之后，我们问自己：「怎么才能让它到尽可能多的人手里？」于是我们又做了一轮实验：一个可以直接提问的 Slack agent；一个做得挺漂亮、大家登录就能看的 HTML 看板。我最喜欢的是播客——吐槽播客。开启一天的绝佳方式：你点开一听，100 个客户冲你吼，数落你的产品有多少种坏法。思路很简单，对吧？就是让大家尽快接触到这些数据。这并不是说不用跟客户聊了，而是说你能精准找出到底该跟哪些客户聊。因为数据是可追溯的。好，一旦找到了要聚焦的点，下一步就是定义你的产品。怎么从一个概念，走到一个范围完整界定清楚的想法？

[6:11] Geoff

And so, we turn to AI, right? And the AIs all kind of look the same. They ask you, "What do you want to build?" And you stare. You have this super intelligent being asking you what you want to build. Is that really the best question to ask? Is that really the best starting point? Right? The goal with AI is to connect it to your systems. That is how you're going to make AI extremely extremely actionable. And so, we did that. We built our own AI agent called Glass, and we gave it all the context it needs to do the work. It connects to all our systems so that it understands both the data with Snowflake, it understands our user research, and it can nail exactly the job to be done that our customers are asking for. That gives us specificity with both qualitative and quantitative data. Glass also understands our product strategy. It understands how we define product specs. It understands our code base. And so, instead of a PM bothering an engineer asking them, "Is this possible? Would this break something? What am I missing?" you ask AI. AI can be your best thought partner. So, now AI is your tech lead. And finally, you can build a prototype. Because it understands our product principles, it understands our design system, it understands our code base, you can actually define and build a prototype that actually works within your product. Your engineering team don't want any of these individual things. They don't want your prototype.

于是我们求助于 AI，对吧？可各家 AI 看起来都差不多。它们问你：「你想做点什么？」然后你只能干瞪眼。一个超级智能的存在在问你想做什么——这真的是最该问的问题吗？这真的是最好的起点吗？对吧？用 AI 的目标，是把它接进你的各个系统。只有这样，AI 才能变得真正、真正地可落地。我们就是这么做的。我们自建了一个 AI agent，叫 Glass，把它干活所需的上下文全都给了它。它接入了我们所有的系统，所以既能借助 Snowflake 读懂数据，也能读懂我们的用户研究，从而精准锁定客户想让我们帮他们完成的那件事（job to be done）。这让我们在定性和定量数据上都能做到具体。Glass 还懂我们的产品战略，懂我们怎么定义产品 spec（需求规格），也懂我们的代码库。所以，PM（产品经理）不用再去烦工程师，问他们「这能做吗？会不会把什么搞坏？我漏了什么？」——你直接问 AI 就行。AI 可以成为你最好的思考伙伴。所以现在，AI 就是你的技术负责人（tech lead）。最后，你还能做原型。因为它懂我们的产品原则、懂我们的设计系统、懂我们的代码库，你真的可以定义并做出一个能在你们产品里实际跑通的原型。但单拎出来的任何一样，你的工程团队都不想要。他们不要你的原型。

[7:37] Geoff

It's useless. They don't want your long spec. That's useless. They want the combination of qualitative and quantitative data to convince them that this is a problem, the actual requirements that they can use for their coding agents, and a prototype that they can actually get inspired by. That is the next contract. So, now you have a strong foundation for which to go. The next step is to build. I mentioned that building is no longer the bottleneck. Obviously, it's a bit more nuanced than that. When we focus on one bottleneck, the next one appears. The first bottleneck was coding. Everyone should have a coding agent at this point. We built Inspect. Inspect is our own coding agent, and the reason why we built it ourselves was because we wanted to have a strong harness, and we wanted to make sure that our coding agent really understood our codebase. It works inside of Slack. It's fully provisioned. It can run under 5 seconds, and it can return an actual product in a deployed preview, so that the actual product manager can actually use and interact with the code that's being built. Anyone at Ramp can now code. Inspect has a million sessions. 75% of our PRs is actually built by Inspect, and a thousand of those PRs over the last month was submitted by a non-engineer.

没用。他们也不要你那份长长的 spec，那也没用。他们要的是这几样的组合：能说服他们「这确实是个问题」的定性加定量数据；能直接交给他们的 coding agent 用的真实需求；再加一个真正能给他们启发的原型。这就是下一代的交付契约。这样一来，你就有了一个扎实的出发基础。下一步是构建。我前面说过，构建已经不再是瓶颈了。当然，实际情况要更微妙一些：当我们集中解决一个瓶颈，下一个就会冒出来。第一个瓶颈是写代码。到了今天，每个人都应该有一个 coding agent 了。我们做了 Inspect。Inspect 是我们自己的 coding agent。之所以自己做，是因为我们想要一套强大的 harness，也想确保我们的 coding agent 真正吃透我们的代码库。它在 Slack 里就能用，环境全都配好了，不到 5 秒就能跑起来，还能把一个真实的产品以已部署预览的形式交回来，让产品经理本人可以亲手使用、操作正在构建的代码。现在 Ramp 的任何人都能写代码。Inspect 已经跑了 100 万个 session。我们 75% 的 PR 其实都是 Inspect 构建的，而且过去一个月里，其中有一千个 PR 是非工程师提交的。

[8:52] Geoff

One tip here, though, is that coding agents are really good when you have a strong architecture and a strong codebase. This expands how much of your company can actually use AI. You now are shipping a lot of code. The next step is code reviews. Becomes the next bottleneck. Engineers are just constantly combing through a ton of code. And so, we focused on that. We built Review Buddy. Review Buddy fully understands our codebase, fully understands our actual quality checks, our security concerns. It finds the right human reviewer to loop in, and it also understands the context, all the prompts that led you to build that code, so it can actually audit how you used AI in the first place. So, the review and the collaboration is now grounded in visible context. 93% of our PRs are now automatically handled by Review Buddy, so that the highest quality engineers are spending time on the 7% of PRs that truly matter. That have a lot of questions around are we building this correctly? Are there any risks? The next bottleneck is testing. Previously PMs, I'm sure you're familiar with this, we spin up a QA environments, we try to tweak different fixtures, we click through the screens, we make sure that the engineer didn't skip out on that one feature we really liked. We automated that. We built Testo. Testo is a browser-based QA agent. It spins up the product in 100 different combinations based on the actual production data that we have access to, and it runs in the product like a user.

不过这里有个提醒：只有当你的架构扎实、代码库扎实时，coding agent 才真正好用。这会扩大你们公司里真正能用上 AI 的范围。现在你们交付的代码量很大了。下一步是代码审查（code review），它成了下一个瓶颈。工程师们一直在海量代码里逐行梳理。所以我们把精力放在了这上面，做了 Review Buddy。Review Buddy 完全吃透了我们的代码库，也完全懂我们实际的质量检查项和安全关切。它会找到合适的人类审查者拉进来，还理解上下文——也就是促使你写出这段代码的所有 prompt——所以它还能回头审计你当初是怎么用 AI 的。这样一来，审查和协作都建立在看得见的上下文之上。现在我们 93% 的 PR 都由 Review Buddy 自动处理，这样最顶尖的工程师就能把时间花在真正要紧的那 7% 的 PR 上——那些疑问很多的 PR：我们这么做对吗？有没有风险？下一个瓶颈是测试。以前——各位 PM 肯定都熟悉这套——我们要搭一个 QA 环境，调各种测试数据（fixture），一屏一屏点过去，确认工程师没有漏掉我们特别喜欢的那个功能。我们把这件事自动化了，做了 Testo。Testo 是一个基于浏览器的 QA agent。它会根据我们能拿到的真实生产数据，把产品按 100 种不同组合跑起来，然后像用户一样在产品里操作。

[10:20] Geoff

We give it instructions. We say, "Hey, pay an invoice, but amortize it." And it clicks around and actually gives us feedback, both blocking feedback, bugs, issues, blockers, and honestly, extremely thoughtful design and quality feedback around things that were confusing, things that break our design system or break our design language. And that goes back into the product development life cycle. Testo in the last 30 days has caught 425 bugs. I think these are bugs better caught by us than our customers. And they send it back through the loop to fix. So, now you're shipping a lot of product. The next step is to coordinate. How do you make sure that everyone in your company is in the loop? Things are moving so fast, you're shipping so many different things. The next bottleneck becomes human attention. We've all been there. PMs being inundated with notifications. And so, our job becomes coordination. I see a few people coordinating right now. Too much process, right? And slows down the builders. Too little process and everyone is confused. And so, human attention becomes that bottleneck. How do you solve this? Well, every question is an API. Every question is an API. And you architect a system. We build gadgets.

我们给它下指令，比如：「嘿，付一张发票，但要做摊销。」它就会到处点一点，然后真的给我们反馈：既有阻塞性的反馈——bug、问题、卡点；也有——说实话——非常有见地的设计和质量反馈，比如哪些地方让人困惑、哪些地方破坏了我们的设计系统或设计语言。这些反馈又会流回产品开发生命周期。过去 30 天，Testo 抓到了 425 个 bug。我觉得这些 bug 由我们自己抓到，总比让客户撞上要好。然后它们会被送回循环里去修复。所以现在，你们交付的产品很多了。下一步是协调：怎么确保公司里每个人都同步知情？事情推进得这么快，你们同时在交付这么多不同的东西，下一个瓶颈就变成了人的注意力。我们都经历过：PM 被各种通知淹没。于是我们的工作就变成了协调。我看台下现在就有几位正在「协调」呢。流程太多，对吧？就会拖慢构建者。流程太少，大家又一头雾水。所以，人的注意力就成了那个瓶颈。怎么解决？其实，每个问题都是一个 API。每个问题都是一个 API。然后你围绕它设计一套系统。我们做了 Gadget。

[11:38] Geoff

A gadget essentially understands the intent of these questions and connects it to the formal record, the road map in Notion, the specs in Notion, the customer calls in Notion, Slack, Linear, our tickets. The principle here is simple. For you to empower an agent, the agent needs to be able to understand and read the organization. And the organization needs to be legible to your agents. I'll give you an example. A question is like, "What's the status of this project? What's the status of this launch? Are we on track?" An agent handles that. It understands the full picture. It understands who owns the next steps and answers with evidence. But not only that, the agent updates our road map. The The agent gives status updates. The agent pings people who are late on their deliverables. There's a lot of fun ways you can actually scale. Or another question, sales, asking questions about, you know, "What is this feature? How do I sell it? Is it available in Brazil? What's the use case? What's the price?" All of these things are fully understood in gadget. If it does not understand, routes it to the right person. But we can take a step further. A launch. Gadget understands the product and so it can write the help center article.

Gadget 本质上能理解这些问题背后的意图，并把它们对接到正式记录上：Notion 里的路线图、Notion 里的 spec、Notion 里的客户通话记录，还有 Slack、Linear、我们的工单。这里的原则很简单：要想给 agent 赋能，agent 就得能读懂、理解这个组织；而组织也得让你的 agent 读得懂。举个例子。比如有人问：「这个项目进展如何？这次发布进展如何？我们还在正轨上吗？」这种问题交给 agent 处理。它掌握全貌，知道下一步由谁负责，回答时有理有据。不仅如此，agent 还会更新我们的路线图，发进度通报，还会去催那些交付物拖期的人。能把这件事规模化的有趣玩法有很多。再比如另一类问题：销售会问，「这个功能是什么？我该怎么卖？巴西能用吗？使用场景是什么？价格多少？」这些 Gadget 全都门儿清。要是它不懂，就把问题转给对的人。但我们还能再往前走一步：发布。Gadget 懂产品，所以它能写帮助中心的文章。

[12:51] Geoff

It can help write the blog post. It can help write the customer email. So on and so forth. Again, start one place and you'll see how far it expands. 85% of questions that are being asked to PMs now are fully answered with AI. And those that are not are answered and fed back into the system. So you're shipping a lot of products now. Everyone's fully aware and coordinated. Everything is perfect. But that's just the start, right? You are constantly improving on the product and the loop starts again. Now, PMs, we we naturally gravitate towards the small reactive things, the things that we have high confidence in, the things that are easy, that are visible, the dopamine hit, oh we've solved something. I think that is a huge mistake. Claire, I think had a great talk around how to be ambitious. You don't be You're not ambitious because you're focusing on small things. You need to automate your way out of these small loops. And so we did just that. For most small things, an AI fully runs the loop. It routes it to the right team. It matches it with your backlog in linear. It dedupes. It counts. It ranks. It runs the plan. It has a human in loop through Slack around, "Hey, is this ready to go? I'm ready to code." It writes the code. It runs through test and CICD just like we talked about. And when it launches, it runs through the knowledge base. These are small loops.

它能帮忙写博客文章，能帮忙写给客户的邮件，诸如此类。还是那句话：从一个点开始，你会看到它能延展到多远。现在，大家问 PM 的问题里，85% 都完全由 AI 回答了。没被 AI 答上的，答复之后也会回灌进系统。所以现在，你们交付了大量产品，每个人都充分知情、步调一致，一切都很完美。但这只是开始，对吧？你要不断改进产品，循环又从头开始。说到这儿，我们 PM 天生容易被那些小的、被动应对型的事情吸引：我们很有把握的事、容易做的事、看得见的事——多巴胺一上来：「哦，我们解决了一个问题！」我觉得这是个巨大的错误。我觉得 Claire 那场讲怎么保持雄心的演讲很精彩。你之所以没有雄心，正是因为你老盯着小事。你得靠自动化把自己从这些小循环里解放出来。我们就是这么做的。对大多数小事来说，AI 会完整地跑完整个循环：把事情分派给对的团队，和你在 Linear 里的 backlog 做匹配，去重、计数、排序，执行计划。它会通过 Slack 让人参与把关（human in the loop）：「嘿，这个可以开工了吗？我准备写代码了。」然后它写代码，跑测试和 CI/CD，就像我们前面讲的那样。等上线时，它还会把知识库那一环也走一遍。这些都是小循环。

[14:11] Geoff

And humans are in the loop only just barely. Thousands of little loops with thousands of little things, so you can focus on the big ones. Thanks to these autonomous loops, 60% 60% of UX issues that are identified by either a customer or a salesperson or sales person or ourselves are fixed within 24 hours. So at this page, you're probably asking yourselves three questions. The first question is, okay, how do we know we're moving faster? Right? How do we move moving faster with quality? After all, you know, we talk about product velocity, how do you measure it? It's certainly harder than, you know, a lap time at a racetrack. Well, some things are not easily quantifiable. Niki Lauda in 1974 took a lap with a Ferrari car and told Enzo, "This car is a piece of It drives poorly. It handles poorly. It brakes poorly." Got out of the car, went to the engineer, fixed it. That is not dissimilar to the culture at Ramp. I'll start there. But the point I'm trying to make here is that you you will know if you're moving fast enough if you actually hire a driver that knows what speed looks like. And you put them in charge and you actually allow them to challenge you. That might be hard for some leaders, but that's what we need to do to be challenged by people who know what great speed looks like.

而人只是勉强参与其中。成千上万个小循环处理着成千上万件小事，这样你就能专注于大事。多亏了这些自主运行的循环，无论是客户、销售还是我们自己发现的 UX 问题，60%——整整 60%——都能在 24 小时内修好。讲到这儿，你们大概会问自己三个问题。第一个问题是：好吧，我们怎么知道自己真的变快了？对吧？怎么在保证质量的前提下变快？毕竟，我们总说产品速度（product velocity），可它要怎么衡量？这肯定比测赛道上的单圈时间难多了。确实，有些东西没那么容易量化。1974 年，Niki Lauda 开着一辆 Ferrari 跑了一圈，然后对 Enzo 说：「这车就是一坨……开起来很差，操控很差，刹车也很差。」说完就下车，找到工程师，把车改好了。这和 Ramp 的文化其实没什么两样。这个就点到为止吧。但我想说的重点是：要想知道自己够不够快，你得真的去请一位知道「快」是什么样的车手，让他来掌舵，而且真的允许他挑战你。这对有些领导者来说可能很难，但我们就是需要被那些懂得什么才是真正的快的人挑战。

[15:33] Geoff

The second question might be, what about our budget? We have limited resources. We have limited resources in terms of engineers or simply with tokens. How do we compete with companies with no constraints? In 2006, Audi asked Audi had a bad car, a very slow car. And so they asked themselves, how can we win if we can't go faster than anyone else? Their answer? Fuel efficiency. Fewer stops, more time on the track. And they won them all three years in a row. So constraints force you to choose a dimension in which you can be world-class. So embrace your constraints, but not your bottlenecks. Find the one thing that you're really, really good at. Find that one bottleneck that you think can 10x your company and just start there. And the last question, something that we've talked a lot about today, is the future role of product. Are we automating our jobs? What's going to happen if these loops are running without PMs? Well, the first thing I'll say is I think it's amazing for for product managers to be out of the loop for a lot of things that engineers can drive. This helps us get a ton of leverage. And I don't think we should be scared about that. Our jobs, though, I think will evolve in three ways. There are three tracks that I see for product managers. The first track is the technical track and something that I don't think we are investing nearly enough. It's the PM that's building the actual factory, the technical product manager that is identifying bottlenecks and removing the drag in your organization. They're not shipping products for the customer, they're shipping the product that helps you build the product for the customer or helps AI do so. The second role, arguably one of the most important roles, certainly the one that gets the most limelight, is a tastemaker. It's the driver.

第二个问题可能是：那预算怎么办？我们资源有限——不管是工程师人手，还是单纯的 token，都有限。我们怎么跟那些不受任何约束的公司竞争？2006 年，Audi……Audi 当时的车很差，速度非常慢。所以他们问自己：如果我们没法比别人跑得更快，怎么才能赢？他们的答案是什么？燃油效率。进站次数更少，在赛道上跑的时间更多。结果他们连续三年全赢了。所以，约束会逼着你选出一个能做到世界级的维度。拥抱你的约束，但别拥抱你的瓶颈。找到那件你真正、真正擅长的事，找到那个你认为能让公司实现 10 倍增长的瓶颈，然后就从那儿开始。最后一个问题，也是我们今天聊了很多的：产品岗位的未来。我们是在把自己的工作自动化掉吗？如果这些循环没有 PM 也照样跑，会怎么样？首先我想说，很多事情工程师自己就能推动，产品经理不必再参与其中，我觉得这太棒了。这能给我们带来巨大的杠杆，我不认为我们该为此害怕。不过，我认为我们的工作会朝三个方向演变。在我看来，产品经理有三条路径。第一条是技术路径，我觉得我们在这上面的投入还远远不够。这是在搭建那座工厂本身的 PM——技术型产品经理，负责找出瓶颈、清除组织里的阻力。他们交付的不是给客户的产品，而是帮你为客户打造产品的那个产品——或者说，帮 AI 去打造。第二个角色，可以说是最重要的角色之一，也肯定是最受瞩目的，是品味定义者（tastemaker）。也就是车手。

[17:24] Geoff

Ultimately, you need someone who's holding the steering wheel and doing things that AI cannot do and holding the bar for what great product, great taste looks like. And finally, the GM. I think PMs will absolutely evolve and expand their role to not just the product organization, but the marketing organization, the sales organization, the growth organizations, the operational organizations. PMs will become GMs and they will own the actual business outcome and lead the entire function. So, we walked through a lot. I'll recap three main takeaways. The first is that speed is not about just coding faster, it's identifying and removing the bottleneck in your build process. And everyone has one. Scientifically, there is always one. Two, as you as you actually solve that first bottleneck, the next one appears. And your job is to move through them as fast as possible. And lastly, us as product leaders, we need to obsess a little bit less about the product that we're delivering and a little bit more about the factory that helps us build products faster. Now, I just walked through a lot here. A lot of great things that we built. And I did so because the reality is that everything you've seen here is outdated.

说到底，你需要有人握着方向盘，去做 AI 做不了的事，守住「好产品、好品味是什么样」的那条标准。最后是 GM（总经理）。我认为 PM 一定会进化，职责会扩展到不只是产品团队，还有市场团队、销售团队、增长团队、运营团队。PM 会变成 GM，对真正的业务结果负责，统领整个职能。好，我们讲了很多，我来总结三个要点。第一，速度不只是写代码更快，而是找出并清除你构建流程中的瓶颈。每个人都有瓶颈。从科学上讲，瓶颈永远存在。第二，当你真正解决了第一个瓶颈，下一个就会冒出来。你的任务就是尽快一个个闯过去。最后，作为产品领导者，我们要少执着一点于我们交付的产品，多执着一点于那座帮我们更快打造产品的工厂。刚才我讲了很多，展示了很多我们做出来的好东西。我之所以这么做，是因为现实是：你们在这里看到的一切，都已经过时了。

[18:41] Geoff

Just like F1, teams will copy each other. I want you to copy us. I want you to actually hopefully surpass many of the things you see today. And I'm sure people here have gone much further. The reality is that it's all about the next season. And so, I'll leave with this. Enzo Ferrari said, "The best Ferrari ever built is the next one." The best product you'll ever build is the next product that you'll launch. That begins in the factory. That begins in the software factory that you build today. I hope you copy us. I hope we get to copy you. Let's share more together as a community around how we are building these products so we can deliver more for our customers. And I hope you all stay stay safe out there. Thank you.

就像 F1 一样，车队之间会互相抄。我希望你们来抄我们，更希望你们能超越今天看到的很多东西。我相信在座有些人已经走得远得多了。现实是，一切都是为了下一个赛季。所以最后我想留给大家这句话。Enzo Ferrari 说过：「史上最好的 Ferrari，就是下一辆。」你能做出的最好的产品，就是你即将发布的下一个产品。它始于工厂——始于你今天搭建的软件工厂。我希望你们来抄我们，也希望我们有机会抄你们。让我们作为一个社区，多分享彼此是怎么打造这些产品的，这样就能为客户交付更多。也希望大家在外面都注意安全。谢谢。

[19:27]

[applause]

（掌声）