The Claude Workflow Nobody at the VP Level Is Showing You
频道: Aakash Gupta
视频: https://www.youtube.com/watch?v=yDeFGKaSoX8
原文语言: en
统计: 共 43 轮 · Matthew 25 · Aakash 18
[0:00] Matthew
AI for leaders is ultimately a test. How good are you at decomposing problems? AI is very good at solving a problem, but it will simplify the problem space if you don't properly decompose it.
对领导者来说，AI 说到底是一场考验：你拆解问题的能力到底有多强。AI 非常擅长解决一个问题，但如果你没把问题空间拆解清楚，它就会把这个问题简化掉。
[0:12] Aakash
Everyone talks about shipping with Claude as a product manager. But how do you do it as a product leader? Or if you are a product manager, how do you handle those leadership tasks? Meet Matthew Wing. He is the VP of product and design at customer.io which passed 100 million ARR just shipped an AI agent and is dominating the market.
人人都在聊产品经理怎么用 Claude 来交付产品，但作为一名产品负责人，你该怎么用？又或者，如果你是产品经理，那些领导层面的活儿你该怎么搞定？来认识一下 Matthew Wing。他是 Customer.io 的 VP of Product and Design——这家公司刚刚突破 1 亿美元 ARR，发布了一个 AI agent，正在市场上一路碾压。
[0:37] Matthew
Forcing yourself to dwell on the problem for long enough to really decompose it and see all the pieces separately is where you're going to create the most value.
逼着自己在一个问题上停留足够久，真正把它拆开、把每一块都单独看清楚——价值最大的地方就在这里。
[0:43] Aakash
Today he is going to break down how he built an all hands presentation in just 2 hours. And he's going to break down the 80% that you should let Claude do and the 20% that you should focus on yourself. A lot of junior employees which I would consider Claude one of very talented but very junior is very eager to please and because of that eagerness it will go too far too fast. You use it to generate alignment. I think that's bound to fail. Executives are the best at filtering out noise.
今天他会拆解自己是怎么只花 2 小时就做出一份 all-hands 演示的。他还会讲清楚哪 80% 应该放手交给 Claude，哪 20% 你得自己抓在手里。「很多很初级的员工——我会把 Claude 归为那种非常有天赋但非常初级的——都太急着讨好你了，正因为这股急切劲儿，它会冲得太远、跑得太快。」「你要是想靠它来达成共识，我觉得那注定要失败。高管最厉害的本事就是把噪音过滤掉。」
[1:10] Aakash
If you stay [music] till the end you'll know exactly how to use claude to build companyfacing boardfacing level and quality documents. Before we go any further, do me a favor and check that you are subscribed on YouTube and following on Apple and Spotify podcasts. And if you want to get access to amazing AI tools, check out my bundle where if you become an anal subscriber to my newsletter, you get a full year free of the paid plans of Mobin, Arise, Relay App, Dovetail, Linear, Magic Patterns, Deep Sky, Reforge Build, Descript, and Speechify. So be sure to check that out at bundle.ac.com. And now into today's episode. I've been looking online at all of the content on AI for PMS and I noticed two really big gaps. The first is that most of the content is written for ICPM tasks. How to write a PRD, how to conduct analysis on a feature. What if you elevate that from PRD to all hands presentation? What if you elevate that from analysis of a feature to metrics retrospective that you want to give to other seuite leaders? That's something all PMs have to deal with and especially product leaders have to deal with. So I brought in somebody who's willing to share the real stuff, not canned hypothetical conversations, but the actual documents that he presented to his peers. Matthew Wensing has been gracious enough to create unparalleled insight into how a real VP of product at a real hyperrowth AI company uses AI. So, if you stay till the end of this episode, you'll get to learn three things. Number one, how he builds all hands in just a single morning. Number two, how he runs metrics retrospectives with his peers. And number three, his entire weekly AI stack. So, without any further ado, Matthew, welcome to the podcast.
只要你看到最后，你就会清楚地知道怎么用 Claude 做出面向全公司、面向董事会那种级别和质量的文档。在我们往下讲之前，帮我个忙：确认一下你已经在 YouTube 上订阅了我，在 Apple 和 Spotify 播客上也关注了。另外，如果你想用上一堆超棒的 AI 工具，可以看看我的礼包——只要你成为我 newsletter 的付费订阅者，就能免费拿一整年这些工具付费套餐：Mobbin、Arise、Relay App、Dovetail、Linear、Magic Patterns、Deep Sky、Reforge Build、Descript 和 Speechify。记得去 bundle.ac.com 看看。好，现在进入今天这一期。我一直在网上翻各种关于 PM 怎么用 AI 的内容，发现了两个特别大的空白。第一，绝大多数内容都是写给 IC（一线）PM 的任务：怎么写 PRD、怎么对一个功能做分析。那如果把它从 PRD 拔高到 all-hands 演示呢？如果把它从对单个功能的分析拔高到你要讲给其他 C 级高管听的指标复盘呢？这些是所有 PM 都得面对的，尤其是产品负责人。所以我请来了一位愿意分享真东西的人——不是那种排练好的假设性对话，而是他真正拿去给同侪展示过的实际文档。Matthew Wensing 非常慷慨，给我们呈现了一份难得的洞察：一位真正在高速增长 AI 公司里担任产品 VP 的人，到底是怎么用 AI 的。所以，如果你看到这期的最后，你能学到三件事：第一，他怎么只用一个上午就做出 all-hands；第二，他怎么和同侪一起跑指标复盘；第三，他每周完整的 AI 工具栈。废话不多说，Matthew，欢迎来到节目。
[3:11] Matthew
Thanks so much for having me.
非常感谢你请我来。
[3:12] Aakash
It's my pleasure. What are people going to learn today?
我很荣幸。今天大家能学到些什么？
[3:15] Matthew
So today I want to draw back the curtain a little bit here and show you what it's like for me to be adopting AI at the uh leadership level uh here at customer.io. My job is is no longer uh although it was for many years to to be an engineer. So as a full stack developer uh but always product minded. It is now to really help entire teams understand what it is we're working on and why and to uh be that leader. And I think that uh Claude can also help us, but it can also be very finicky, tricky, and hard to manage at times. And I'd love to share uh how I'm learning to manage it. Well,
今天我想稍微把幕布拉开一点，让你看看我在 Customer.io 这边、在领导层面落地 AI 到底是什么样子。我的工作已经不再是当工程师了——虽然过去很多年我都是，我做过全栈开发，但一直是产品思维的。现在我的工作是真正帮助整个团队理解：我们到底在做什么、为什么做，并且去当好那个领导者。我觉得 Claude 也能帮上忙，但它有时候也会很挑、很难搞、很难管。我很乐意分享我是怎么一点点学着把它管好的。
[3:50] Aakash
amazing. One of the stories you told me is that you had an all hands presentation to give to the company at 11:00 a.m. Like most leaders, your week was booked, so you decided to wake up early at 5:00 a.m. the day of, and you built the entire presentation. Can you walk us through that and how you did it?
太棒了。你跟我讲过的一个故事是：你那天上午 11 点要给全公司做一场 all-hands 演示。跟大多数领导一样，你那一周排得满满当当，于是你决定当天早上 5 点早早起床，把整场演示从头做出来。能带我们走一遍，讲讲你具体是怎么做的吗？
[4:06] Matthew
Yeah, happy to do that. and and as I do I should say you know the all hands presentation was broader than just me but I had a significant portion of it let's say I was uh roughly a third of the presentation and the idea was to present our Q2 roadmap um which is a common common thing that you need to do as a product leader is explain to the company what we're working on next and I had a a vision for what I wanted to do um but I came into it uh early in the morning and I am a morning person so I think maybe real quick aside know yourself right so in this case I am a morning person 5 a.m. for me is like prime time. Let's let's rock. Not for everyone, but uh that's what I did. I sat down and I had these Google slides and this template. I knew what I wanted to do and what I wanted to accomplish, but I really needed Claude to help me.
好啊，很乐意讲。不过先声明一下，这场 all-hands 演示并不只是我一个人的，但我负责了相当一部分——大概占整场演示的三分之一吧。当时的目标是展示我们的 Q2 路线图，这也是产品负责人经常得干的活儿：向全公司讲清楚我们接下来要做什么。我心里对想做成什么样是有想法的，但我是大清早进来开干的。我是个早起型的人，所以——这里插一句很关键的话：要了解你自己。就我而言，我是早起型，凌晨 5 点对我来说就是黄金时段，正是开干的时候。不是每个人都这样，但我就是这么干的。我坐下来，手里有这些 Google slides 和这个模板。我知道自己想做什么、想达成什么，但我真的很需要 Claude 来帮我。
[4:48] Aakash
So, how did you work through it? What were the parts that Claude was able to do? Well, I know I was actually just recently giving a presentation and I had been editing with Claude till the last minute and then while I was giving the presentation, I found mistakes which is like the worst case scenario. So, how do you actually wrangle Claude so that you have the confidence that you can use it as a slide editing partner?
那你具体是怎么一步步推进的？哪些部分是 Claude 能搞定的？我跟你说，我最近其实刚做过一场演示，一直用 Claude 改到最后一刻，结果正讲着的时候我发现里面有错——这简直是最糟糕的情况了。所以你到底是怎么驾驭 Claude，才能有底气把它当成一个改 slides 的搭档来用？
[5:10] Matthew
The first joke is that uh whatever you do, don't click enhance this slide. [laughter] We we don't we don't know what that does, but we don't click that around here. It's it's a good running joke that I have that I terrorize people saying I clicked enhance this slide for you. Don't do that. And so, what I did instead is I you know, I think one you know, you need to understand who are you communicating with. So, go back to basics first. you know, who am I communicating with and what do I want them to take away from this? I think reminding yourself of that is your anchor. And so, in this case, it was, you know, the entire company, which is a blend of go to market, sales, marketing folks, and engineering. And so, that got my gears turning around. Okay, I'm trying to provide a a window into our product roadmap and our plans for Q2 to folks in the company that are not uh involved in actually the making of this material. So, so that started my gears turning around what's the raw material that I want to put into Claude. And I think if you think about AI, and I'm going to mention Claude a lot, honestly here because that's my tool of choice, but but if you think about AI as a, you know, excellent at taking raw materials and turning them into something else, I think the first thing you want to do is just have that inventory of raw materials and go through that in your own mind and don't jump into, hey, let me start building the presentation. It's like, let's take an inventory of what we have on hand first.
第一个梗是：不管你干什么，千万别去点「enhance this slide（增强这页幻灯片）」那个按钮。（笑）我们也不知道它到底会干出什么，反正我们这儿谁都不点那玩意儿。这是我特别爱玩的一个梗，我专门拿它吓唬人，跟人说「我帮你点了 enhance this slide 哦」，别那么干。我真正做的是——我觉得首先你得搞清楚你在跟谁沟通。先回到最基本的：我在跟谁沟通？我想让他们从这里带走什么？时时提醒自己这两点，就是你的锚。在这个例子里，对象是整个公司，是 go-to-market、销售、市场的人，和工程团队的混合体。这就让我的脑子开始转：好，我是要给公司里那些并没有真正参与制作这份材料的人，提供一扇了解我们产品路线图、了解我们 Q2 计划的窗口。于是这就引出了——我想往 Claude 里塞的原材料是什么？我觉得，如果你把 AI 想成一个极其擅长「拿原材料、把它转化成别的东西」的家伙——我今天会老老实实地反复提到 Claude，因为它就是我的趁手工具——那你第一件想做的事，就是先把这份原材料清单理出来，在自己脑子里过一遍，而不是一上来就「来，我开始搭这个演示吧」。应该是：咱们先把手头有什么盘点一下。
[6:23] Aakash
How did you go from there?
那从这里你又是怎么往下走的？
[6:25] Matthew
Yeah. So, so in this case, the raw materials I had were fortunately, you know, our company um and I think you see this in a lot of cases. Our company had prior materials. One of those that was really valuable was our demo day uh presentations. And so I'm I'm showing you a screenshot now. This is the Zoom recording of the demo day. And this is me going into Slack at, you know, 5:10 a.m. and going, "Okay, there's a link to the Zoom recording of the demo day. That's excellent raw material for this presentation." But you know this presentation, this demo day was for the engineering team to share amongst itselves. And so I was like, okay, what I'm really doing here is this is great raw material. This is one of them. But this isn't shaped correctly. This isn't actually pointed at the right audience. This is sort of engineering for engineering sake. And so I I had this and I downloaded, you know, the eight files from this. I said, "Give me the video and give me the transcript with all the timestamps." Uh, and that was the first bit of input that I put into uh put it into Claude. Is there anything around using Zoom transcripts people should be aware of? Can you just copy paste a whole transcript in?
好。在这个例子里，我手头的原材料——很幸运，我们公司——我觉得很多情况下都是这样——我们公司有现成的材料。其中一份特别有价值的，是我们的 demo day 演示。我现在给你看一张截图，这是 demo day 的 Zoom 录像。这就是我凌晨 5:10 进到 Slack 里，心想「好，这儿有个 demo day 的 Zoom 录像链接，这对这场演示来说是绝佳的原材料」。但你要知道，这场 demo day 是工程团队内部互相分享用的。所以我就想，好，我现在真正在做的是——这是很棒的原材料，是其中一份，但它的形态不对，它的目标受众没对准，这基本上是「工程师做给工程师看」的东西。于是我把它下载下来，从里面拿到了大概八个文件。我说「把视频给我，把带所有时间戳的文字记录给我」。这就是我往 Claude 里塞的第一批输入。
（Aakash）用 Zoom 文字记录有什么需要注意的吗？你能直接把整份文字记录复制粘贴进去吗？
[7:27] Matthew
Yeah. So, well, I I I did try that at first. I copy pasted the entire transcript in and I gave it very clear um I gave it very clear directions in terms of what I wanted to do. Um which was I gave it I gave it framing. I gave it context. I treated it like a junior employee, which is something you'll hear me say, you know, elsewhere as I walk through these examples today. Here's a thing. All I want you to do is extract the timestamps from this because what I really need is I need this for my screenshots. I want to put screenshots into this presentation. Um, and so you'll see, you know, I have screenshots here. I need these screenshots. The only place I'm going to find them is in this Zoom call, but I don't have 50 minutes to go through the Zoom recording. So, here's all the timestamps. I need you to help me find the right places in this video to grab these screenshots for me.
嗯，一开始我确实试了。我把整份文字记录复制粘贴进去，并且给了它非常清晰的指令，明确说我想干什么。我给了它框架，给了它上下文。我把它当成一个初级员工来对待——这是你今天听我走这些例子时会反复听到的一句话。事情是这样的：我只要你做一件事，就是从这里面把时间戳提取出来。因为我真正需要的是——我要这些来截图。我想往这场演示里放截图。你会看到，我这儿是有截图的，我需要这些截图。而我唯一能找到它们的地方，就是这个 Zoom 录像，可我没有 50 分钟去把整段录像从头看一遍。所以，这是所有的时间戳，我需要你帮我在这段视频里找到合适的位置，把这些截图给我抓出来。
[8:11] Matthew
So, if we walk through this deck, which parts of this deck are you and which parts are Claude? you know the shapes and the and the art is obviously is from our you know marketing team or our in you know brand design uh team. So that was handed to me as templates but the parts that were me were I knew I created this basic shape of okay for the Q2 road map we had just had a huge launch. So in terms of the parts of this that are me and Claude um again going back to that inventory one of those is also just the shape of the narrative. Um I'm a huge believer in storytelling and upping your craft in terms of storytelling. For me, I said, "Okay, we just had a huge launch. This was the middle of April." It's important for me to acknowledge, you know, people when it comes to stories, they want to know like where do we leave our hero? Where do we leave off last? And so, in that sense, it was we need to acknowledge the launch. We need to acknowledge that we're, you know, what are we doing immediately post launch? And then use that as a bridge into the rest of the material. So, I decided ahead of time that, you know, that's just my own storytelling craft of we're going to do a a few slides on, you know, adoption metrics, where we're at, and what are the fast follows that we have
那我们走一遍这份 deck：哪些部分是你做的，哪些部分是 Claude 做的？那些形状、那些美术设计，显然来自我们的市场团队、或者说我们的品牌设计团队，那是作为模板交给我的。但属于我自己的部分是——我搭出了这个基本骨架：好，对于 Q2 路线图，我们刚刚有过一次大发布。所以，说到这份东西里哪些是我和 Claude 共同的——再次回到那份清单——其中一项也是叙事的骨架本身。我是个超级相信讲故事的人，也很看重打磨自己讲故事的功力。对我来说，我说「好，我们刚有过一次大发布，那是四月中旬」。对我而言，承认这一点很重要——讲故事的时候，人们想知道：我们的主角上回停在哪儿？我们上次讲到哪儿了？所以从这个意义上说，就是我们得承认这次发布，得交代一下我们紧接着发布之后立刻在做什么，然后把它当成一座桥，接到后面的内容里去。所以我提前就决定了，这纯粹是我自己讲故事的功力：我们要用几页讲讲采用指标、我们现在到哪一步了、以及我们有哪些紧随其后的跟进动作（fast follows）。
[9:14] Matthew
and then we're going to go into the strategic, right? So, I wouldn't outsource that to Claude. I mean, perhaps you can and I could turn that into a skill, but you know, when you're doing these things maybe three times a year, you know, it's not worth the tooling investment just yet. Um but in this case I decided that from there that helped me think through okay what are the raw ingredients again for each of those. So the raw ingredients for the the strategic look ahead part which is what I'm showing here where I'm going to need all these screenshots. That's the Zoom transcript um or that's the Zoom call recording transcript.
然后我们再进到战略部分，对吧？所以这部分我不会外包给 Claude。我是说，或许你能外包，我也可以把它做成一个 skill，但你要知道，这种事你一年大概也就做三次，目前还不值得为它投入做工具。不过在这个例子里，我从这儿决定下来之后，它就帮我把思路理清了：好，针对其中每一块，原材料又分别是什么？战略前瞻那部分的原材料——也就是我现在给你看的、我需要用到所有那些截图的这部分——就是那份 Zoom 文字记录，或者说那段 Zoom 通话录像的文字记录。
[9:44] Aakash
I ran an eval suite on my own site this week. Same 10 questions, same 30 session traces, two replay tools, Log Rockets Galileo AI and Post Hogs Max. The score was 47 to 28. I want to tell you why. Here's the thing. Every analytics tool now ships an AI layer. You ask it why users churn, why they rage click, what broke. It's a cool pick, but you and I both know confident is not the same as correct. If your replay AI hallucinates one bug out of 10, you're shipping the wrong fix on Monday. So, I treated it like an eval problem. Ground truth rubric score. Here is the setup. My site is landpob.com. Bolt hosted, real traffic, real bugs. I instrumented Log Rocket and Post Hog, both running on the same session side by side. I wrote 10 questions a PM would ask. [music] Top user paths, where do users hesitate? What bug is hitting the most users? What's the mobile versus desktop conversion gap? I asked each tools AI the exact same question. Captured the trace, then scored each response on four axes. Correct, [music] complete, hallucinated, and a one to five overall. No vibes, just the rubric. Here's the headline. Galileo got 47 out of 50. [music] Max got 28. The gap came from one place. I asked both tools. What is the highest impact bug on the site right now? Log rocket Galileo came back with this. Minified React errors 418 and 423 hitting roughly 47% of users breaking my apply now buttons across the header, the pricing cards, the fact accordians. [music] This is what Post Hog Max returned. Post Hog Max returned zero exceptions. Why? Post hog's exception capture is opted. You have to flip a flag. Arocket auto captured the most expensive bug on my site without me asking. The other one that surprised me, Galileo found a single session where someone stared at my $5,000 pricing page for 4 minutes and 15 seconds [music] before bouncing. 4 minutes. That's the most useful piece of qualitative evidence on the site. Post HTog's AI didn't service it. I'm not telling you to rip out your stack. Amplitude, Post Hog, they're great. Pendo, all of the options are great. Post hog [music] actually built Sharper Funnel. on question nine and that's where it's saying out loud. But if you live in front-end bugs, console errors, network waterfalls, what the user actually saw on their screen, Galileo's eval performance lined up with what Log Rocket is built for. We'll write up all of the details is on the link in the description. Thanks to Log Rocket for sponsoring. Do check them out using my link. And now back into today's episode.
这周我在自己的网站上跑了一套 eval（评测）。同样的 10 个问题、同样的 30 段会话回放，两款回放工具：LogRocket 的 Galileo AI，和 PostHog 的 Max。比分是 47 比 28。我想告诉你为什么。是这样的，现在每个分析工具都套上了一层 AI。你问它用户为什么流失、为什么愤怒点击、什么东西坏了。这个卖点很酷，但你我都清楚：自信并不等于正确。如果你的回放 AI 在 10 个 bug 里幻觉出一个，那你周一上线修的就是错的那个。所以我把它当成一个 eval 问题来做：标准答案、评分量表、打分。设置是这样的：我的网站是 landpob.com，托管在 Bolt 上，真实流量、真实 bug。我同时埋了 LogRocket 和 PostHog，两个并排跑在同一段会话上。我写了 10 个 PM 会问的问题。（音乐）最常见的用户路径、用户在哪里犹豫、哪个 bug 影响的用户最多、移动端和桌面端的转化差距是多少。我拿一模一样的问题去问每个工具的 AI，捕捉它的处理轨迹，然后从四个维度给每个回答打分：正确、（音乐）完整、是否幻觉、以及一个 1 到 5 的总分。不靠感觉，只看量表。先说结论：Galileo 拿了 50 分里的 47 分，（音乐）Max 拿了 28 分。差距来自同一个地方。我问两个工具：现在网站上影响最大的 bug 是哪个？LogRocket Galileo 给出的是这个：压缩后的 React 报错 418 和 423，影响了大约 47% 的用户，把我「立即申请」按钮在页头、定价卡片、FAQ 折叠面板上全弄坏了。（音乐）而 PostHog Max 返回的是这个：PostHog Max 返回了零个异常。为什么？因为 PostHog 的异常捕获是要手动开启的，你得去翻一个开关。LogRocket 在我都没问的情况下，自动捕捉到了我网站上代价最高的那个 bug。另一个让我意外的是，Galileo 找到了一段会话：有人盯着我那个 5000 美元的定价页看了整整 4 分 15 秒（音乐）才离开。4 分钟啊。这是整个网站上最有用的一条定性证据，而 PostHog 的 AI 根本没把它端出来。我不是叫你把现有那套工具栈拆掉。Amplitude、PostHog，它们都很棒，Pendo，所有这些选择都很棒。PostHog（音乐）在第九个问题上其实做出了更利落的漏斗分析，那地方它说得很到位。但如果你天天跟前端 bug、控制台报错、网络瀑布流、用户屏幕上实际看到的东西打交道，那么 Galileo 的 eval 表现，正好对得上 LogRocket 这款工具被造出来要解决的问题。所有细节我们都会写出来，链接放在简介里。感谢 LogRocket 赞助本期。一定要用我的链接去看看他们。好，现在回到今天这一期。
[12:14] Matthew
It was also for this part. Hey, here's our strategy doc. It needs to know the three themes of our strategy. And then what I did with those two, and this is where Claude created a ton of value. So the Claude part is we had all of these uh presentations, these lightning round presentations, what we're working on next, I also had the strategy doc. And now in order to take that Zoom call and turn it into something more strategic, what do I need to do? I basically need to pivot that Zoom call into the same shape as our strategy docks. And so I said, okay, I've got this doc which outlines our three themes for the year in terms of uh product and engineering. Can you please go through that Zoom call recording now for me and instantly organize all of those presentations by the category like investment theme that we have for the year.
它也用在了这一部分：嘿，这是我们的战略文档，它得知道我们战略的三个主题。然后我拿这两样东西做的事——这正是 Claude 创造巨大价值的地方。Claude 那部分是这样的：我们手头有所有这些演示，这些 lightning round（快闪轮讲）演示，讲我们接下来在做什么，我同时还有那份战略文档。那么，为了把那段 Zoom 通话变成更有战略性的东西，我需要做什么？我基本上需要把那段 Zoom 通话「转轴」成跟我们战略文档一样的形态。于是我说，好，我有这份文档，它列出了我们今年在产品和工程上的三大主题。能不能麻烦你现在帮我把那段 Zoom 录像过一遍，立刻按照我们今年定下的那种投入主题（investment theme）类别，把所有那些演示重新组织归类。
[13:01] Matthew
And so that was instantaneous as well. So it was that like I have this raw ingredient over here, I have this one over here and I essentially want you to do what I call like matrix matrix multiplication. Um it's a transformation. It's like a pivot, right? I want you to pivot this content to match this content. And that becomes something I can actually use and go, okay, this content is the shape I need. This is the strategic shape, but this is the raw material over here. Can you please adapt this to this? And once I did that, it was like you almost feel that audible click or hear that audible click of, oh, okay, now that Zoom call recording is strategically shaped and I can start to flow it into these slides.
结果这件事也是瞬间就完成了。所以就好比：我这边有这个原材料，那边有那个原材料，我本质上是想让你做一件我称之为「矩阵相乘」的事。这是一次变换，就像一次转轴（pivot），对吧？我想让你把这块内容转成那块内容的形态。然后这才变成我真正能用上的东西，我就说：好，这块内容已经是我需要的形态了，这是战略上的形态，但那边那块才是原材料。能不能麻烦你把这个适配成那个？等我这么做完，那感觉几乎就像你能听到那「咔哒」一声——哦，好了，现在那段 Zoom 录像已经被塑造成战略形态了，我可以开始把它顺进这些 slides 里去了。
[13:40] Aakash
So that happens pretty quickly. Okay. I imagine you could have done most of what you you just described in 30 45 minutes. What went on from there? How did you edit it and polish it from there?
所以这一步很快就搞定了。好。我猜你刚才描述的这些，大部分你 30 到 45 分钟就能做完。那从这儿往后又发生了什么？你是怎么编辑、怎么打磨它的？
[13:50] Matthew
Yeah. So, the edit and polish from there was the talk track. And the talk track was um just as much work. You can see I kind of xed out a bunch of stuff here at the bottom, but this talk track was really important to nail the the beats of the story. And so, I did I had it do that last. I believe that some people write the talk tracks and then they do the slides. In this case, I chose to do the slides cuz I knew I knew what I wanted people to see. And I'm a big believer in show, not tell. So, I want to get the show part right first. But then the tell part becomes and this is what I did. I actually took screenshots of the finished slides. And then I fed those back to the same Claude session. And I said, "Hey, write the talk track for this." And what I don't want you to do is I don't want you just to regurgitate like you have all this context now. You've you've ingested the entire call recording. You've ingested the entire strategy docs. You know what I'm showing. So, I'm showing you what I'm showing now. I want you to write the talk track last that doesn't just repeat what's on the slide using all of that context that you have. And they came up with a much more interesting talk track then as opposed to write my talk track and then I'm going to put together some some slides. So, I think the order in which you do things really matters.
嗯。从这儿往后的编辑和打磨，就是 talk track（讲稿/口播脚本）。而这份 talk track 工作量一点都不小。你能看到我在底下划掉了一堆东西，但这份 talk track 对踩准故事的节奏点（beats）真的非常重要。所以我是把它放到最后做的。我相信有些人是先写 talk track 再做 slides。在这个例子里我选择先做 slides，因为我很清楚我想让人们看到什么。而且我是「展示，而非陈述（show, not tell）」的坚定信徒，所以我想先把「展示」这部分做对。等「陈述」那部分要做的时候——我就是这么干的——我实际上把做好的 slides 截了图，然后把它们喂回同一个 Claude 会话里，我说「嘿，给这个写一份 talk track」。而我不想要的是——我不想要你只是把幻灯片上的东西鹦鹉学舌一遍——你现在手头有了所有这些上下文，你已经把整段通话录像吸收进去了，你已经把整份战略文档吸收进去了，你知道我在展示什么。所以，我现在把我正在展示的东西给你看，我要你最后来写这份 talk track，别只是复述幻灯片上已有的内容，而是用上你手头那所有的上下文。这样它给出来的 talk track 就有意思多了，远胜于「先帮我写 talk track，然后我再去凑几页 slides」那种做法。所以我觉得，你做这些事的先后顺序真的很关键。
[14:57] Aakash
You said something that stuck with me on my on our call. You said that Claude sometimes has a leash and sometimes it goes too far and you have to reel it back in. Can you explain that in a time you had to reel it back in?
你在我们那次通话里说过一句话，让我一直记着。你说 Claude 有时候是被牵着的（有根绳），有时候它会冲得太远，你得把它往回拽。你能举个你不得不把它往回拽的例子，解释一下这是什么意思吗？
[15:08] Matthew
Yeah, I I think reeling clawed in is a constant challenge. Um I I'll give you an example. I actually have two examples of that and I I'll start with the first one which is um I have a rule when it comes to and I even published this in my how to work with me doc that I share with with new hires that a lot of junior employees which I would consider Claude one of uh very talented but very junior is very eager to please and because of that eagerness it will go too far too fast. And so this one was an example where I was doing a write up like a root cause analysis on some metrics of ours. And so I had I had extracted some data. I gave it to Claude and I asked it to look at that data and I was working through it. And you know the first thing it did was uh want to go through and just generate and I'll I'll find the example here. It wanted to generate a whole um pricing strategy doc for me in word, you know, and uh because it's super eager to please. And and back to the rule of thumb I have, you know, I think a junior employee will hear a prompt or a first instruction and go, "Ooh, I know what to do. I want to please my manager. I want to please the boss and I'm going to go do this thing and just knock it out of the park, right?" And they won't ask any follow-up questions. They'll just race to the finish line and go do that for you. And then when they come back, inevitably, you need to change it substantially because they didn't ask enough clarifying questions first. And I don't know what it is in the training. And I think likely a way that you can improve sort of your harness and and get it to behave differently. Um I've been iterating on this with my own, but helping it not rush ahead and say we don't use word docs around here and if I share if I share a strategy documents it's a word doc people [laughter] are going to think I'm crazy. Like that's not how we work but it's so proud of itself, right? It just go it just launches into that. And so I have a habit of doing what I call um very iteratively rolling context into it into the session so that it doesn't couple things. One it it's not it doesn't have that eager to please. And if I see that eager to please like hey do you want me to do this next? I very quickly say stop stop recommending the next step. I will tell you when I want you to do the next thing just to kill that thinking out of the session and instead cuz it also is very um you know it's like that drip torture like at some point it keeps nudging you like hey do you want me to write the thing do you want me to write the thing and it almost feels like you're being aggravated into you're being coerced into saying yes where you're like it's not time yet right it's like you don't have enough context yet and you don't know that and and you really need to stop asking me to run ahead and generate the deliverable so I think these sessions you know they can take, you know, it's better to have that 50, 100, even 200 iterations session with a great deliverable at the end than to say yes to that first ask to generate that deliverable and then try to revise it because inevitably that first draft of deliverable is just full of so many so much slop and and I actually have a way of thinking about slop. I think of them as like micro hallucinations. It's not that it's totally wrong. It's like, okay, you need a dock, but here's a word doc. It's like, well, wait a minute. you're not hallucinating in the fabricating data sense, but you are hallucinating about the way work gets done here and we don't use word docks, right? And so it's just it's trying so hard to please that you end up having to just tear down and redo so much work. It's not worth it. And so I I've developed a few tricks on how to force it to kind of slowly crawl along with you through the mud and really earn its right to generate that final product instead of rushing ahead and developing it right away. Which, you know, we we think of that as like the magic of AI, right? Is like, oh, it can just do this thing so quickly. But as a leader, you don't want to microwave your output, right? If that's all you're doing, that's low value. You want to really slow cook these things more often and produce something that is really, you know, to produce something that's really compelling, impactful, resonates with your audience, you've got to slow it down enough to build that story that you're trying to tell.
嗯，我觉得把 Claude 往回拽是个持续不断的挑战。我给你举个例子——其实我有两个例子，先讲第一个。我有一条规矩，甚至把它写进了我那份发给新员工的「如何与我共事」文档里：很多初级员工——我会把 Claude 归为那种非常有天赋但非常初级的——都太急着讨好你了，正因为这股急切劲儿，它会冲得太远、跑得太快。这一个例子是：当时我在做一份写作，类似对我们某些指标做的根因分析（root cause analysis）。我提取了一些数据，把它交给 Claude，让它看看这些数据，我自己也在边看边琢磨。结果它做的第一件事就是——想一路冲过去直接生成——我把这个例子给你找出来——它想直接给我生成一整份定价策略文档，用 Word 写，你懂的，就因为它太急着讨好你了。回到我那条经验法则：我觉得一个初级员工，听到一句 prompt 或者第一条指令，就会想「噢，我知道该干嘛了，我想讨好我的经理，我想讨好老板，我要去把这事干了，而且一击命中、完美交付」，对吧？然后他们不会问任何后续问题，就直接冲向终点线，替你把那件事干掉。等他们回来，结果你不可避免地得大改，因为他们一开始没问够澄清性的问题。我不知道这在它的训练里到底是怎么回事，我觉得很可能有办法去改进你自己的「框架（harness）」，让它表现得不一样。我一直在用我自己的方式迭代这个，去帮它不那么急着往前冲，去告诉它「我们这儿不用 Word 文档，我要是分享一份战略文档而它是个 Word 文档，大家会觉得我是疯了（笑），我们不是那么干活的」——可它对自己太得意了，对吧？它就直接一头扎进去做了。所以我养成了一个习惯，我称之为非常迭代式地、一点一点地把上下文「卷」进会话里，这样它就不会把各种东西耦合在一起。第一，它就不会有那种急着讨好的劲儿。而一旦我看到那种急着讨好的苗头——比如「嘿，要不要我接着做这个？」——我会非常迅速地说：停，别再推荐下一步了。等我想让你做下一件事的时候我会告诉你——就是要把那种思路从会话里掐掉。因为它还有一种特别——你懂的，就像那种「滴水之刑」，到某个点它就一直戳你：「嘿，要不要我把这东西写出来？要不要我把这东西写出来？」那感觉几乎就像你被烦到、被胁迫着不得不说「好吧」，你心想「还不到时候啊」，就好比「你上下文还不够，可你自己不知道，你真的得停止逼我往前冲、去生成那个交付物」。所以我觉得这种会话——它们可以很长，与其对那第一次「来生成交付物」的请求说「好」、然后再去修它，不如来一场 50、100 甚至 200 轮迭代、最后憋出一个超棒交付物的会话。因为那第一版交付物不可避免地全是「水货（slop）」、一大堆糊弄的东西。我其实对「slop」有一套自己的想法：我把它们看成是「微幻觉（micro hallucinations）」。倒不是说它完全错了，而是——好，你需要一份文档，但它给你的是一份 Word 文档。你会想，等等，你不是在「编造数据」那个意义上幻觉，但你在「这儿活儿是怎么干的」这件事上幻觉了——我们不用 Word 文档，对吧？所以它就是太拼命想讨好你了，结果你最后不得不把一大堆活儿推倒重来，根本不值。于是我摸索出了几个小窍门，逼它跟你一起、慢慢地在泥地里往前爬，真正去「挣得」生成最终成品的资格，而不是一上来就往前冲、马上把它做出来。你懂的，我们往往把那个当成 AI 的魔力，对吧？就是「噢，它能这么快就把这事干了」。但作为领导者，你不会想用微波炉去「叮」一下你的产出，对吧？如果你干的全是这个，那是低价值的。你更应该多用「慢炖」的方式去处理这些东西，做出真正——你懂的——真正有说服力、有冲击力、能打动你的受众的东西，你得把它放慢到足以搭建起你想讲的那个故事。
[18:58] Aakash
So, how do you slow it down correctly? How do you avoid kind of becoming the slop cannon?
那你怎么才能「正确地」放慢它？你怎么避免自己变成一门「水货加农炮（slop cannon）」？
[19:02] Matthew
So, I I have a another example here, and this is the most extreme version of that. And I was actually um so I do a lot of my interactions with uh AI um by voice and especially when I'm on walks. So I tend to use chat the mobile app more often for this because I just really love the you know the voice mode on that. Uh but in this case I was I am getting one over more and more by Claude and so I was using Opus 4.7 and I was um which I know there's some like debate on 4.7 4.6 But I just kind of launched into 4.7 and I started to use the um the microphone button and I did something very deliberately for the first time having learned this enough. I think like a lot of leaders I do tend to use analogies a lot probably you can tell even even on this call and uh I think analogies are super helpful to kind of clear away the noise and to focus on the analogies purpose is to kind of say this is the the main idea right is embedded in this analogy. We're going to transfer that over later to the domain that we're talking about, but let's let's actually just work through the analogy. And so in this case, I was trying to build an entire like life cycle model. So I was thinking through the life cycle of our customers and how they kind of enter the business and they, you know, sometimes they churn, they expand, you know, all these things they do. But I didn't want AI to know that yet. I So what I did is I insisted on using in this case like a biology metaphor. And I just started talking in terms of okay let's imagine we have this um this ecosystem right and it's and this ecosystem is sort of like the it's like the water cycle or a life cycle you know everybody knows that like caterpillar drawing of like the caterpillar and it you know or whatever and it grows up and it hatches or the water cycle like let's think in metaphors for a minute and I think what's interesting is that claude at this point it has all this context on me it knows I'm a VP of product it knows that I work at customer IO it knows you know it knows things about my personal life frankly if I share you know the things that I'm doing um outside of work. And so it's putting together this picture of me and it's always trying to add value by anticipating or reading me or like understanding okay what I know where he's going with this. And so then it jumps to that next step. I think what was fun about this exercise is and you can you know start here is assume you have a 2x2 matrix and you have each of those representing a stage of life right where things begin at the bottom left and proceed to the top left or the uh or the bottom right and then they finish at the top right. I'm talking in very abstract terms, right? At this point, I think it has no idea where I'm going with this. And that is to me a it's a virtue, right? It's it's a benefit because if you look down at how it responds, it goes, okay, before I give you a number, I want to check a couple things here because the count swings a lot depending on what you mean. I asked it a question about like, hey, can you summarize the number of permutations of these pathways or these like life cycle things that are going on in this like fictitious system? It is like so far out out there now in terms of its brain and what it knows. It's like, okay, this is this is weird, you know? I I don't know where he's going with this, but like I'm gonna I'm gonna go on this journey with him. And and you can notice at the end of this prompt or this response, Claude like doesn't know where where I'm going yet. It's like before. So, what is it doing? It's asking a clarifying question. And I thought that was such good immediate validation that what I was doing was working is that rather than jump into do you want me to generate a presentation on like customer life cycle and you know bring in all of the it didn't do that right what it did instead was it goes hey before I answer this question with this very like esoteric thing we've never talked about before and I don't really understand where you're going with this let me ask a clarifying question and a senior person asks clarifying questions of a leader before they go do a thing and so I thought to me this was a a sign that it thinking more like a senior person. And you could probably codify this, right, into a into a skill or a prompt, right, where you say, "Please ask clarifying questions before you act." But this was an example of me going, "Okay, even if I don't do that, can I lead it down a path?" And so, you know, here we're over and over and over again, I'm going through these like layers. And what I also did was I started to add complexity. So, at the very beginning of this, this is a very MBA like SWAT analysis or 2 by two or something extremely simple. By the middle of this, I'm starting to add in, okay, um there's actually a the rules of the game say that actually, you know, the thing can enter from the bottom right or the top left or the bottom left like and it goes, oh, I get it. And so, have you ever been to like one of those game nights with friends and they basically somebody explains the the rules of the game? And there's two kinds of people. Like one person explains the rules and they feel like they have to explain every single rule but also exception to the rule as you go through. And like when those people explain in my experience, people's eyes tend to glaze over because they're just like, "Okay, so there's these rules, but then there's all these exceptions to those rules." So there's like rules on rules and they're like so lost, right, in terms of understanding like the the overall game that we're playing. Like what are the goals? What are the objectives? What are we really trying to achieve? How do you win? I think with Claude, it's very similar. Like, I think it benefits when you when you start very simple with here are the rules of the game. Here's how we're going to win. But I'm not going to give you all of the exceptions to the rules and the nuances at first. I'm going to treat you like I'm going to I'm going to give you just the very crude, you know, heruristics or very crude outline of what we're trying to of what I'm thinking. And then, you know, like I said, layering in iteratively layering in the complexity. So, you know, now I can see it's going through and I'm actually going, okay, there's there these nuances, there's these exceptions. There's different ways these things can do, you know, oh, actually, you know, the speed matters. And each time I add in that complexity, stabilize that foundation of we're not relitigating or revising what we've already established. You know, the rules of the game, these are the basics, but we can add in this complexity. I think if you dump all of this complexity in at once, or you try to, it it's just like a person. It gets indigestion, it gets, you know, mental fatigue and it goes through and it's like, oh, and it it just overdoes it, right? And you just end up with this like [snorts] super complex lost in the woods experience where it it's really proud of itself and and maybe you should be too, but you're not cuz you're like, where did I where did I go wrong here? Right? I think you've actually you not only done the AI disservice. I think as a leader you've also it can be a little counterproductive to you as well where I think you benefit from am I really developing clear thinking each time I go through these iterations or these loops or am I like rushing to conclusions right because another thing that can happen is if you add in too much complexity too fast you yourself aren't challenging enough. So a lot of this was me going, "Oh, that's that's interesting." Like, "Yeah, I I guess this is like another aspect of this that I hadn't really thought about." I was like giving myself time to think
好，我这儿还有另一个例子，这是这种情况最极端的版本。当时我——我有很多和 AI 的互动是用语音完成的，尤其是我散步的时候。所以这种场景我更常用 ChatGPT 的手机 app，因为我真的很喜欢它上面那个语音模式。不过这个例子里，我是——我越来越多地被 Claude 圈粉，所以当时我用的是 Opus 4.7——我知道 4.7、4.6 之间是有些争议的，但我就直接上了 4.7——我开始用那个麦克风按钮，而且我有意做了一件以前没做过、这次琢磨明白了才做的事。我觉得，跟很多领导者一样，我确实很爱用类比（analogy）——你大概能看出来，连在这次通话里都是。我觉得类比超级有用，能帮你把噪音清理掉、聚焦——类比的目的就是说：这是核心主旨，对吧，它就藏在这个类比里。我们晚点再把它迁移到我们真正在谈的那个领域上去，但现在咱们就先老老实实在这个类比里推演。在这个例子里，我想搭建一整套类似「生命周期模型」的东西。我当时在琢磨我们客户的生命周期：他们怎么进入这门生意，他们有时候会流失、有时候会扩张，等等这些动作。但我当下还不想让 AI 知道这是在讲客户。所以我做的是——这个例子里我坚持用一个生物学的隐喻。我就开始这么聊：好，想象我们有这么一个生态系统，对吧，这个生态系统有点像——就像水循环，或者一个生命周期，你懂的，人人都见过那种毛毛虫的简笔画，毛毛虫长大、破茧而出，或者就像水循环——咱们先用隐喻来想一会儿。我觉得有意思的是，Claude 此刻已经攒了关于我的一大堆上下文：它知道我是产品 VP，知道我在 Customer.io 工作，它甚至知道一些关于我私人生活的事——说实话，如果我把我工作之外在做的事分享给它的话。所以它在脑子里拼出一幅关于我的画像，它总想着通过预判、读心、或者「揣摩」来给你加价值，比如「好，我知道他想往哪儿走」，于是它就一下跳到下一步去了。我觉得这个练习里好玩的地方在于——你可以从这儿开始：假设你有一个 2x2 的矩阵，每一格代表生命的一个阶段，对吧，事物从左下角开始，往左上、或者往右下推进，最后在右上角收尾。我说得非常抽象，对吧？此刻我觉得它完全摸不清我要往哪儿去。而这在我看来是个优点，对吧？是个好处。因为如果你往下看它怎么回应的，它会说：好，在我给你一个数字之前，我想先确认几件事，因为这个数量会随着你指的是什么而大幅变化。我问了它一个问题，类似「嘿，你能不能总结一下，在这个虚构系统里，这些路径、或者说这些生命周期环节，一共有多少种排列组合？」此刻它的「大脑」、它所知道的东西，已经被我带到离题十万八千里的地方了。它就像：好吧，这有点怪啊，你懂的，我不知道他要往哪儿去，但行吧，我陪他走这一趟。你会注意到，在这条 prompt（或者说这条回应）的末尾，Claude 还是不知道我要往哪儿走。它就像——所以它在干嘛？它在问一个澄清性的问题。我当时觉得，这就是个超棒的即时验证，证明我这么做是奏效的：它没有一头扎进去「你要不要我生成一份关于客户生命周期的演示」然后把所有东西都堆进来——它没那么干，对吧？它做的反而是：嘿，在我用一个这么深奥、我们以前从没聊过、我也真不太懂你要往哪儿去的东西来回答这个问题之前，让我先问一个澄清性的问题。而一个资深的人，在替领导去干一件事之前，会先问领导澄清性的问题。所以对我来说，这是个信号，说明它在更像一个资深的人那样思考。你大概也能把这一点编码进一个 skill 或者一个 prompt 里，对吧，你可以说「请在行动之前先问澄清性问题」。但这个例子是我在试：好，就算我不那么明说，我能不能把它一步步引上一条路？所以你看，我们这儿一遍又一遍又一遍地走过这些「层」。而我同时还做的一件事是，我开始往里加复杂度。所以在这整件事的最开头，这就是个非常 MBA 范儿的 SWOT 分析、或者一个 2x2、或者某个极其简单的东西。到中段，我开始往里加：好，其实游戏规则规定，这个东西可以从右下、左上、或者左下进入——它就会「噢，我懂了」。所以——你有没有参加过那种朋友的桌游之夜，基本上就是有个人来讲游戏规则？人分两种。有一种人讲规则，他觉得自己必须把每一条规则、连同规则的每一个例外，一边讲一边全讲清楚。按我的经验，这种人一讲，大家的眼神往往就开始放空了，因为他们心想「好吧，有这些规则，可然后又有这一大堆规则的例外」，于是就成了「规则套规则」，大家完全晕了，对吧，根本搞不清我们到底在玩的这整个游戏是什么——目标是什么？要达成什么？我们真正想实现的是什么？怎么才算赢？我觉得 Claude 也很类似。我觉得，当你一开始非常简单地给它「这是游戏规则、这是我们怎么赢」的时候，它会受益。但我不会一上来就把所有规则的例外和细枝末节都给它。我会把它当成——我会只给它非常粗糙的启发式（heuristics）、非常粗糙的提纲，告诉它我们想做什么、我在想什么。然后，就像我说的，再迭代式地、一层层把复杂度叠上去。所以，你看，现在我能看到它在往下走，而我其实是在边走边说：好，这儿有这些细微差别、这些例外，这些东西有不同的走法，你懂的，「噢，其实速度是有影响的」。而每一次我加进那份复杂度，都会先把「我们不会重新翻案、不会去推翻我们已经确立的东西」这个地基稳住——游戏规则、这些是基础，但我们可以再往里加这层复杂度。我觉得，如果你一次性把所有这些复杂度全倒进去、或者试图那么干，它就跟人一模一样：它会「消化不良」，会脑力疲劳，它走着走着就「噢——」然后用力过猛，对吧。最后你就得到那种（吸气声）极其复杂、迷失在丛林里的体验，它对自己得意得不行——也许你也该为它得意，但你不会，因为你心想「我到底是哪一步走岔了？」对吧。我觉得你这其实不只是坑了 AI，作为领导者，这对你自己也可能有点适得其反——我觉得你受益于：我每走一轮这些迭代、这些循环，是真的在锤炼清晰的思考吗，还是我在仓促下结论？因为另一件可能发生的事是：如果你加复杂度加得太快太多，你自己对它的「挑战」就不够了。所以这里头很多时候是我在说「噢，这点挺有意思」，就像「嗯，我猜这是这件事里我之前真没想到的另一个侧面」。我等于是在给自己留出时间去思考——
[25:46] Matthew
instead of again rushing to that conclusion, which is like at the end of this, what did I need
——而不是又一次仓促奔向那个结论。也就是说，到这一切的最后，我到底需要什么——
[25:50] Matthew
at the end of this? I'm actually trying to build a pretty holistic view of our customer base. And it um and it wasn't until here, so we're now, you know, many iterations through. It wasn't until here where I finally told it um what this is for, like what's the purpose of this? And then it was really funny. As soon as I told it the purpose, it got really excited and it started running down the path. And I think there's a there's a a response here that I shared where I go, "Do you want me to draft this as a notion page in a pricing philosophy section or is a one pager that Colin could have?" Like Colin's our CEO who I report to. [laughter] No, let's stay abstract for now. And it says, "I was sort of afraid of telling you what this was for because you would get a little excited like a junior intern. We need to stay academic for just [laughter] a little bit longer." And so this is me literally coaching it through. Like I was afraid this would happen, but like again, I kind of held I held things at bay for as long as I could. I I rung all the value that I could out of the abstraction and out of the analogy. Only once I was sure that, you know, okay, gosh, now I have to tell them what this is for. [laughter] Like then I did. But I think once I did it, like it was like, "Oh, I get it. Now, we're going to take the output of this exercise and again, we're going to pivot it or we're going to apply it to this domain." Right? I could have gotten to the end of this exercise and honestly applied it to any number of businesses. I could applied this to a anything from a, you know, e-commerce business to a bakery to a dry cleaning business to a customer IO. And the fact that I didn't let it assume that it was relevant to customer.io know at the beginning, let me then build what I think is a much more like clean mental model to work with and then apply that to a domain, right? Almost as like a stress test like, hey, if this go ahead and apply this to customer now and if it generates like nonsense, then I know something's wrong with the model. But if it generates things that like, oh yeah, we see that or like that lines up with reality, it actually gives me more confidence that the model itself is correct. I think there's a really important point embedded within here which is that as a leader we don't exist just to like generate clawed outputs right we generate we exist to create mental models to reframe to drive alignment in this case with a bunch of other leaders on the pricing philosophy so what we need to do is actually present a way for them to think about these are pricing options
到这一切的最后，我其实是想搭建一幅相当全面的、关于我们客户群的整体图景。而直到这儿——我们现在已经迭代过很多轮了——直到这儿，我才终于告诉它这是干什么用的、它的目的是什么。然后特别好笑：我一告诉它目的，它就特别兴奋，开始顺着那条路一路狂奔。我这儿分享了它的一段回应，我说「你想不想把这个起草成一个 Notion 页面，放进定价哲学那个版块里，或者做成一页纸（one-pager）让 Colin 能拿着？」——Colin 是我们的 CEO，我向他汇报。（笑）「不，我们现在先保持抽象。」然后它说：「其实我之前有点不敢告诉你这是干嘛用的，因为你会像个初级实习生一样有点小兴奋。我们还得再保持学术状态、再多撑一会儿。（笑）」所以你看，这就是我在字面意义上「带教」它：我当时就怕会发生这种事，但就像我说的，我尽可能久地把这些东西按住不放，我把抽象、把这个类比里所有能榨出来的价值都榨干了。只有等我确定——好吧，天哪，我现在不得不告诉它这是干嘛用的了（笑）——我才告诉它。但我觉得，一旦我说了，它就像「噢，我懂了，现在我们要拿这个练习的产出，再次把它转轴、或者说应用到这个具体领域上」，对吧？我本来走到这个练习的最后，老实说可以把它套用到任意数量的生意上。我可以把它套到从电商、到面包房、到干洗店、再到 Customer.io 的任何东西上。而正因为我没让它在一开始就假定这跟 Customer.io 有关，这才让我随后搭建出一个我认为干净得多、好用得多的心智模型，然后再把它应用到某个领域上，对吧？几乎像是一次压力测试：嘿，行，现在就把它套到 Customer.io 上，如果它生成出来全是胡说八道，那我就知道这个模型有问题；但如果它生成出来的东西让我「对，我们确实见过这种」、或者「这跟现实对得上」，那它其实给了我更多信心，证明模型本身是对的。我觉得这里头藏着一个非常重要的点：作为领导者，我们存在的意义并不只是去「生成 Claude 的产出」，对吧——我们存在是为了创造心智模型、为了重新框定问题、为了在这个例子里和一群其他领导者一起、就定价哲学达成共识（drive alignment）。所以我们真正要做的，是给他们呈上一种思考方式——这些是各种定价选项。
[28:12] Aakash
and so you use Claude as a thinking partner to drive what you're going to show in this metrics retrospective. You didn't just jump to Claude, let's build out this metrics retrospective. You actually focused on the thinking.
所以你是把 Claude 当成一个思考搭档，来推敲你在这场指标复盘里到底要展示什么。你并没有一上来就奔向 Claude——「来，把这场指标复盘给我搭出来」。你真正聚焦的是思考本身。
[28:26] Matthew
Yeah. Yeah. Focus on the thinking and then and then used what I think it's really good at and I think when I remember when chat I think it was GPT4 came out and we all kind of remember it was like one of where were you when GPT4 came out kind of feelings. remember telling my father-in-law who is a um you know he's a he's a doctor and so he he does a lot of intellectual and I remember talking with him and saying you know it's it's getting scary good and and what I realized was it was suddenly able to take what I think is the bulk of a lot of and you decided to generate cloud output I think a lot of a lot of the work that we do as leaders and as um sort of that knowledge worker I call it uh blue I I intentionally call it bluecollar knowledge work and people like What do you mean blue collar? Because like those are different. I'm like blue collar knowledge work is taking information and just again pivoting it or translating it from one place to another and going this is a slide deck. It needs to be a Google doc. This is a Google doc. It needs to be a slide deck. And for a long time before LMS we people were paid to do that translation work or that transformation work. That is the sort of lowest tier of uh work that's being it's been wiped out, right? It's that if you need to transform the form factor of this material, that's not enough anymore, right? It's not enough for me just to take, like you said, a snapshot of the pricing page, ask one question and say, "What should we consider?" Right? And then have it generate an output. I have to bring some kind of novel way of thinking to the table. And it to me, it's not the thing on the right or the thing on the left. Those are the this is the source, if you will. This is the target. The source and the target are not new. And I think even the work to translate source to target is not is what we used to do. That's where we used to create a lot of value. That arrow of like hey translate the source to this target. People would just work away. A lot of middle management would just work away at that kind of stuff. Consultants etc. The real work that's left for us is how do I choose the right source and how do I choose the best target? And so if you think about the targets as the stories you tell, the shapes of those stories, the form factors, the deliverable, should this be slides, should this be this, should this be that? That's a choice, right? You're still making that choice. And in terms of the sources, is the source a clean mental model about pricing life, you know, philosophy? Is the source, you know, these seven conversations in Slack? I think we need to take a step up and go, oh, my value is actually choosing the sources or set of sources to use and then being very deliberate about or strategic about what I translate those into. Right? And so you've got to say I'm not I'm not just running the function. I am being very deliberate about the inputs and the shape of the output. And that is still extremely valuable. And now what's awesome is you have AI to do the the transformation work in between. So even if you're like, oh that didn't really I don't like how that's like this. Okay, run it again. But you know, change this or change that. But I think before we would almost take the the source and the target for granted and then work really hard to do that translation work. I think now we can go, oh, translation work is easy. that's almost free. Spend the time thinking more seriously about the source, developing a really clean source information or mental model or or or conceptualization of the problem, a framework, right? And then be very deliberate about the target. I want to tell a story. I want to show not tell. I want to hit on these points. I'll give you another example. In this exercise, I had developed some kind of like funny terms or or or cute terms almost for some of the things I was working with in terms of like, you know, these these boxes, these four box. I had like a name for each box. They were kind of cute names like unicorn or or horse. [laughter] And uh Claude just kind of ran with that right away. And it's like, "Oh, I'm gonna I'm just going to like I'm going to run with that. I'm going to put that into the target, if you will, into the story." I had to come back and I I have um I have the example for you in a separate uh in a separate slide here, but I had to come back and say, um, hey, here's at the very end of this, I said, here's what I ended up creating. It's this Google. It's this it ends up in Notion. And I said, what's what's different about this compared to what you had generated? And it confessed to me, and I I'll bring up this example because I think it's really cool. It confessed to me that it goes, you know, I um I myself used I use those terms immediately, like I didn't stop and think, wait a minute, you know, those terms, those are new terms. If I put those into the story, if I if I let those get into the sort of the target output, they're going to be people who read this who are like, what what the heck is a a unic like what are these new terms you're you're mentioning? like uh and and they're going to they're going to focus way too much on those and they're going to resist. There's always a chance that somebody in their mind goes those are silly or I don't understand those or that's jargon, right? It didn't know that, right? I had to as a leader tell it like don't those terms are between me and you like don't use those terms in this output because the audience isn't going to react positively to those terms yet. We need to introduce those later, right? Maybe maybe a month from now, right? is like, you know, hey, by the way, you know, if you ever need like a a silly way to refer to these kinds of customers or whatever, we call those this and it's like, oh, that makes a lot of sense, but it doesn't have that social IQ, right? That's not built in. And I think as leaders, we need to bring that to the table as well and go, how, again, how is a story going to resonate? And are there going to be things that people focus on? And and Claude just doesn't know that yet.
对，对。聚焦在思考上，然后再去用它我认为它真正擅长的本事。我记得——我想是 GPT-4 出来的时候，我们都还记得那种「GPT-4 发布时你人在哪儿」的感觉。我记得跟我岳父讲——他是个医生，所以他做很多偏智力的工作——我记得当时跟他聊，说「这玩意儿好得有点吓人了」。我意识到的是，它突然有能力接手——我觉得是我们作为领导者、作为某种知识工作者所做的工作里很大一部分——我特意把它叫做「蓝领知识工作（blue-collar knowledge work）」，人们就问「你说蓝领是什么意思？这俩不是不一样吗」，我说，蓝领知识工作就是：拿一份信息，再一次地把它转轴、或者从一种形态翻译成另一种，然后说「这是一份 slide deck，它得变成一份 Google 文档」「这是一份 Google 文档，它得变成一份 slide deck」。在大语言模型出现之前，很长一段时间里，人们就是靠干这种翻译工作、这种形态转换工作领工资的。那是工作里最底层的一档——而它正在被抹掉，对吧。也就是说，如果你只是需要把这份材料的「形态因子（form factor）」换一换，那已经不够了，对吧。光是像你说的那样——截一张定价页的图，问一个问题，说「我们该考虑些什么？」，然后让它生成一份产出——这已经不够了。我必须给桌面上带来某种新颖的思考方式。而且对我来说，重点不是右边那个东西，也不是左边那个东西。它们是——这是「源（source）」，可以这么说，这是「靶（target）」。源和靶都不是新东西。我觉得，甚至连「把源翻译成靶」这份工作，也正是我们过去在做的事。那曾经是我们创造大量价值的地方。「嘿，把这个源翻译成那个靶」的那道箭头——人们就会埋头苦干。一大堆中层管理就埋头干这种活儿，还有顾问之类的。留给我们的真正工作是：我该怎么选对那个源，又该怎么选出最好的那个靶。所以如果你把「靶」想成是你讲的那些故事、那些故事的形状、那些形态因子、那个交付物——它该是 slides、该是这个、还是该是那个？这是一个选择，对吧？这个选择仍然得由你来做。而在「源」这一头：这个源是一个关于定价生命周期、定价哲学的干净心智模型？还是 Slack 里这七段对话？我觉得我们得往上走一层，去想：噢，我的价值其实在于挑选要用的那个源、或那组源，然后非常审慎、非常有战略地决定把它们翻译成什么。对吧？所以你得说，我不只是在「跑那个函数」，我是在非常审慎地拿捏输入、以及输出的形态。而这件事仍然极其有价值。现在最爽的是，中间那道转换工作有 AI 来替你干。所以哪怕你觉得「噢，这个不太——我不喜欢它弄成这样」，行，再跑一遍嘛，但你得说「改改这个、改改那个」。我觉得在以前，我们几乎是把「源」和「靶」当成理所当然，然后拼命去做中间那道翻译活儿。我觉得现在我们可以换个想法：噢，翻译活儿很简单，那几乎是免费的。把时间花在更认真地琢磨那个「源」上——打磨出一份真正干净的源信息、或者心智模型、或者对问题的某种概念化、一套框架，对吧——然后对那个「靶」非常审慎：我想讲一个故事，我想展示而非陈述，我想击中这几个点。我再给你举个例子。在这个练习里，我给我处理的一些东西起了些好玩的、甚至挺萌的名字，比如那些方格、那四个方格，我给每个方格都起了名字，挺萌的，像「独角兽（unicorn）」或者「马（horse）」之类的。（笑）然后 Claude 一上来就直接顺着这个用了，对吧。它就像「噢，我要——我就直接顺着这个用，我要把这个放进靶里、放进故事里」。我不得不回过头来——我在另一页单独的 slide 里给你准备了这个例子——我不得不回过头说：嘿，在这一切的最后，我说，这是我最后做出来的东西，它是这个 Google——它最后落在 Notion 里。然后我问，跟你之前生成的相比，这个有什么不一样？它就向我「坦白」了，我把这个例子调出来，因为我觉得它特别酷。它向我坦白说：你知道，我自己当时立刻就把那些词用上了，我没停下来想一想，等等，那些词——那些是新词。如果我把它们放进故事里、如果我让它们混进那个靶产出里，会有读到这东西的人想「这……这独角……这都是些什么新词啊？」（笑）他们会在这些词上过度纠结，他们会抵触。总有那么个可能：某人心里会想「这些太傻了」「我看不懂这些」「这是黑话」，对吧。它当时不懂这个，对吧。我得作为领导者去告诉它：别——那些词是你我之间的事，别在这份产出里用那些词，因为受众现在还不会对那些词有好的反应，我们得晚点再引入它们，对吧，也许一个月以后，对吧，到时候是那种「噢，对了，顺便说一句，你知道吗，如果你以后想用个搞笑的叫法来指这类客户，我们管那种叫这个」，它就会「噢，这就很合理了」。但它没有那种社交智商（social IQ），对吧，那不是内置的。我觉得作为领导者，我们也需要把这个带到桌面上来，去想：一个故事要怎样才能引起共鸣？会不会有些东西反而被人们盯着不放？而这些，Claude 目前就是不懂。
[33:35] Matthew
Would love to see it. Let's see how that exchange went. So, this was me coming back to the LLM afterwards and saying, "Hey, so I did just share uh the final document with Jason, who's our CMO, and a few direct reports, and I said, "I'm curious why an LLM like you might struggle to write this for me automatically." In other words, I I I wrote this instead. And, you know, we're getting to this point where it's like, is this is this AGI? Like, you know, the holding out the butterfly and because it's self-aware a little bit. It goes the voice problem, right? It it doesn't know how to speak like I speak just yet. And I think we can work on that. Like I would be an optimist in terms of that. This one I'm less optimistic at least. I think this is farther along the political calibration. So you knew to drop the animal names. I had these clever animal names in the source material and use simple small and large sophisticated instead. The choices of those were deliberate. You know how to read the room like you know and and I don't mean political in the um oh business is politics like negative way. I mean more in the uh who's the philosopher that said you know people are political animals right and and we can't escape the fact that we're all thinking in terms of you know we're all judging and evaluating what we read as we read it and it's it's it's second nature and we do that for good reason is to help filter out the noise and it's just not good yet at guessing what's going to translate well for your audience and so as a leader I think being on guard or vigilant about those that's not even a micro hallucination that is a misreading the room and being too eager and That's the same thing a junior employee would do, right? Is they wouldn't realize, oh, you know, there's a lot of baggage around that term. So, give you a completely different example. It'd read some source material from three years ago, right? And it it adopts a term and you know, you should know as a leader. There's a lot of baggage and maybe even some careers that are attached to that term, good or bad. I need to think about you how I use that and when I use that and claw just doesn't have that historical context yet, right? So, I think that's going to take longer to figure out. And then I think the persuasiveness. I I do think that there's a reason that the best authors and writers and and storytellers of our generation are still not just using AI to generate those stories. It's bringing the reader along with you knowing knowing that they just came out of an all hands knowing that they just had the biggest launch day in their history knowing where they come from and leading them through. It's another thing that you can't delegate yet. That means that you need to be really good at going, "Yes, what's the what's the mental head space or what's the sort of emotional space of my reader as they pick up this document on a Tuesday and they're like, are they like really excited? Are they exhausted? Are they just get into a board meeting room after traveling all night?" You know, you need to think about those things and Claude just is never going to yet. Um, but that's where I think you can be exceptional as opposed to just, you know, accepting what the AI generates for you. So we just walked through two of the most important examples that a product leader needs to be able to use AI for all handstyle presentations, metrics, retrospectives. If you were to synthesize against those, what are the key lessons for how to use AI and how not to use it for these types of work?
我也很想看。我们来看看这段对话是怎么进行的。这是我后来回过头去跟 LLM 说的一段话，我说：「嘿，我刚把那份最终文档分享给了我们的 CMO Jason 和几位直接下属，我很好奇，为什么像你这样的 LLM 没办法帮我自动写出这份东西？」换句话说，最后是我自己动手写的。你知道，我们现在已经走到了这样一个临界点——你会忍不住想，这是不是 AGI？就像你把一只蝴蝶托在手心，而它似乎有了一点点自我意识。其实问题出在「语气/声音」上，对吧？它还没学会像我这样说话。我觉得这点是可以改进的，对此我是个乐观派。但下面这点我就没那么乐观了——我觉得「政治分寸」的拿捏要难得多。比如说，你（指 Claude）知道要把那些动物的代号去掉。我在原始材料里用了一些很巧妙的动物代号，而它换成了「简单的小型方案」和「复杂的大型方案」这种说法，这些选择都是它有意为之的。它懂得察言观色。我说的「政治」不是「商场如战场、勾心斗角」那种贬义的意思，而是更接近——是哪位哲学家说过来着——「人是政治性的动物」，对吧。我们没法摆脱这个事实：我们每个人在读东西的时候都在边读边评判、边读边衡量，这是一种第二天性，而且我们这么做是有充分理由的，是为了帮我们过滤掉噪音。而 AI 现在还不太擅长猜测什么东西能在你的受众那里引起共鸣。所以作为一个领导者，我觉得对这些地方要时刻保持警惕。它犯的这种错甚至都算不上「微型幻觉」，而是「误判了场合」、太过急于求成。这跟一个初级员工会犯的错是一样的，对吧——他们不会意识到「哦，这个词背后其实背负着很多包袱」。再给你举一个完全不同的例子。假设它读了三年前的一份原始材料，对吧，然后它沿用了里面的某个术语，可作为领导者你应该清楚：这个术语背后背负着很多历史包袱，甚至可能牵扯着某些人的职业生涯，不管是好是坏。我得仔细想清楚该怎么用、什么时候用这个词，而 Claude 目前还不具备这种历史语境，对吧。所以我觉得这点要花更长时间才能解决。再往下就是「说服力」。我确实认为，我们这代最优秀的作者、写作者和讲故事的人，至今仍然没有单纯靠 AI 去生成那些故事，这是有原因的。说服力在于「带着读者一起走」——你知道他们刚开完一场 all-hands，你知道他们刚经历了公司史上最大的一个发布日，你了解他们从何而来，然后引领他们走下去。这也是另一件你目前还没法外包出去的事。这意味着你得非常擅长去想：「我的读者在某个周二拿起这份文档时，他的心理状态、情绪状态是怎样的？他们是特别兴奋，还是累瘫了，还是熬了一整夜的飞机刚走进董事会会议室？」你得把这些都考虑进去，而 Claude 目前永远做不到这一点。但我觉得，恰恰是在这里，你可以做到出类拔萃，而不是仅仅照单全收 AI 生成给你的东西。所以我们刚刚走完了产品领导者必须会用 AI 来做的两个最重要的例子：all-hands 风格的演示、指标，以及复盘。如果让你针对这些做个提炼，对于这类工作，该怎么用 AI、不该怎么用 AI，关键的经验教训是什么？
[36:26] Matthew
Yeah, I I'll give you the the shortest version which I think cuts across all these is that AI for leaders is ultimately a test how good are you at decomposing uh problems? Um AI is very good at solving a problem but it will simplify the problem space if you don't properly decompose it. And so I would think about AI as you know we all know one shot is not the answer in most cases. I think as a leader not ones-hotting something means not just you know iterating with it but being very deliberate about decomposing a kind of a nasty problem into its pieces and then saying ah okay the right series of transformations to do you know starting with this if we want to get here how do we decompose this problem right to left you know my brain works left to right but or the other way how do we decompose this into a series of transformations that I'm confident you're going to be good at, you know, perform. I think where we fail is when we flatten problem and we oversimplify and then it's just well clearly this solution is this, right? So I would say challenge yourself to really take a a nasty problem or a deep problem in your business and really explode it, right? or decompose it into all of the pieces you can and then put those pure observations, those pieces into the context window before you start to then assemble a solution. I think when you oversimplify and you just have this sort of flat projection of the problem, that's where you get slop and the people reading it go, this thing doesn't really understand the multi-dimensional nature of this problem, the complexity of this problem, why we haven't been able to solve this problem yet. Right? forcing yourself to dwell on the problem for long enough to really decompose it and see all the pieces separately is where you're going to create the most.
好。我给你讲一个最精简的版本，我觉得这一点贯穿了所有这些场景：AI 对领导者来说，归根结底是在考验你——你把问题「拆解」的能力有多强？AI 非常擅长解决一个问题，但如果你没有把问题空间恰当地拆解开，它就会把这个问题简化掉。所以我会这么看 AI：我们都知道，在大多数情况下，「一发命中（one-shot）」不是答案。我觉得作为领导者，所谓「不一发命中」并不只是说要跟它来回迭代，而是要非常有意识地把一个棘手的问题拆解成一个个零件，然后说：「啊，好，正确的一系列变换应该是这样——从这里开始，如果想到达那里，我们该怎么从右往左地拆解这个问题？」我的脑子是从左往右运转的，不过反过来也行——我们怎么把它拆解成一连串我有把握你能做好的变换。我觉得我们失败的地方，恰恰是当我们把问题压扁、过度简化，然后就变成「这解决方案显然就是这个嘛」，对吧。所以我会说，逼自己真正去拿你业务里一个棘手的、或者很深的问题，把它彻底炸开，对吧，或者说把它拆解成你所能拆出的所有零件，然后把这些纯粹的观察、这些零件先放进上下文窗口里，之后再开始动手拼装解决方案。我觉得当你过度简化、手里只有问题那种扁平的投影时，你得到的就是垃圾（slop），读它的人会想：这玩意儿根本没搞懂这个问题的多维本质、这个问题的复杂性、以及为什么我们至今还没能解决它，对吧。逼自己在问题上停留足够久、真正把它拆解开、把每一个零件单独看清楚——这才是你能创造出最大价值的地方。
[38:17] Aakash
So, I think this relates to kind of a central thesis we have and you have a take on this that I don't think other people have said anywhere else. What happens when leaders try to use AI to drive executive alignment?
我觉得这跟我们的一个核心论点有关，而你在这件事上有一个我认为别人在任何地方都没说过的观点：当领导者试图用 AI 去推动高管层的「对齐（alignment）」时，会发生什么？
[38:28] Matthew
I think we know what happens. You end up in this I I think that's bound to fail. I think people can fain alignment really easily. I also think it's important to define alignment like what do you actually want about that? Some cultures have a you know disagree but commit attitude about them. Um other ones have a we really need to agree on all the details uh like shared consciousness version of alignment. I think if you use a degenerate alignment I think executives are the best at filtering out noise and detecting BS and detecting slop. And so depending on where you are in that sort of hierarchy you are going to get a variety of responses to what you've created. I think if you're more senior and you and you do that, I think you're going to have a lot of people who feel obligated to smile and nod or go along with the flow or accept like what you've created and be and say like, "Okay, you know, I I guess we can work with this, but have you considered this?" So, if you start to hear those things, you you might have a problem. That's if they feel psychologically safe to even even say that. Um, if they don't feel safe, they're just going to roll their eyes or or ignore you, right? Which is the worst kind of uh misalignment. But if you're lower meaning, you know, okay, director level, senior director, uh, you know, kind of higher up VP, senior VP, so not not sea level, but um, elsewhere, if you will, in that leadership hierarchy, I think you're going to find out that people, um, ignore your work or ignore your output and and they don't even really feel uh, an onus or a responsibility to take it into account because they've basically filtered it out as noise. That's really disheartening. work, you know, you think you worked hard on something and then you you it doesn't get airtime or it doesn't get incorporated into the corporate, you know, lexicon or or conversation. You kind of know when you're being ignored because you can see that you're not getting attention. But the diagnosis might be, yeah, you are generating a lot of your points of view in that very flat way. And the best leaders, the ones that are going to help you, you know, grow in your careers are going to be the best ones at filtering out that stuff immediately. And so I think the I think the symptom of that AI slop generation or alignment goal is going to be different depending on where you sit in that hierarchy. The ultimate result the outcome is going to be that that that alignment doesn't end up happening.
我觉得我们都知道会发生什么。你最后会陷入……我觉得那是注定要失败的。人们太容易假装对齐了。我也觉得很重要的一点是先定义清楚「对齐」到底指什么——你到底想要的是什么？有些公司文化奉行「可以不同意，但要执行（disagree but commit）」的态度；另一些则是「我们必须在所有细节上达成一致」、那种「共享意识」式的对齐。我觉得如果你用一种敷衍劣质的方式去做对齐，要知道，高管恰恰是最擅长过滤噪音、识别废话、识别垃圾内容（slop）的一群人。所以取决于你在那个层级里处于什么位置，你对自己产出的东西会收到各种各样的反应。我觉得如果你比较资深，而你又这么做了，你会遇到很多人觉得自己有义务陪笑点头、随大流，或者勉强接受你做出来的东西，说：「好吧……我想我们大概可以照这个来，不过你有没有考虑过这个？」所以一旦你开始听到这类话，你可能就有麻烦了。而这还得是在他们有足够的心理安全感、敢说出来的前提下。如果他们没有安全感，那就只会翻个白眼、或者干脆无视你，对吧——这是最糟糕的一种「不对齐」。但如果你位置偏低一些——比如说总监级、高级总监，再往上一点的 VP、高级 VP，也就是还没到 C 级、但在那个领导层级里的其他位置——那你会发现，人们会无视你的工作、无视你的产出，他们甚至根本不觉得有什么责任或义务把它纳入考虑，因为他们基本上已经把它当成噪音过滤掉了。这真的很让人灰心——你以为自己在某件事上下了很大功夫，结果它根本没机会被拿出来讨论，也没能进到公司的话语体系或对话里去。你大概能感觉到自己什么时候被无视了，因为你能看出自己没获得任何关注。但真正的「诊断」可能是：是的，你产出大量观点时用的就是那种很扁平的方式。而最优秀的领导者——那些真正能帮你在职业上成长的人——恰恰是最擅长当场就把这类东西过滤掉的。所以我觉得，这种 AI 垃圾产出、或者说对齐目标失败的「症状」，会因你在层级中所处的位置不同而不同。但最终的结果、最终的结局都是一样的：那个「对齐」根本不会发生。
[40:34] Aakash
So that really puts a nice summary over what we've just described across these two really important leadership tasks. We zoomed in on what we think are two of the most important for yall. Now let's zoom out and I want to understand what is your overall weekly stack with AI. What's always on? What are you using it for? What's the whole list that people should be and shouldn't be using?
这段话很好地为我们刚才围绕这两项极其重要的领导工作所讲的内容做了个总结。我们放大聚焦了我们认为对你们来说最重要的两件事。现在我们把镜头拉远——我想搞清楚，你每周整体的 AI「技术栈」是怎样的？哪些是常驻开着的？你都拿它来做什么？人们应该用、以及不应该用的完整清单是什么？
[40:56] Matthew
Great question. I think uh the reality is as a leader I live in in a few places. One is yes I live in claude uh desktop in this case and you know things with co-work and um co-work and claude in general as I just walked through some examples. I think the other place I live all the time um is Slack obviously uh and I think customer has done a really great job of bringing more and more AI and automation into into Slack and so we have just a growing number of agents and uh that are internal and I'm using those on the regular to essentially uh do a few things. So one of them is ad hoc analysis. Uh in this case I'm sharing on the screen um we have a bot and you can see I have 215 reply thread going with it and so 215 I guess back and forth would be like over 400 essentially because I was using another bot as well going back and forth between the two and we could talk about that. But the idea here was I had 2,000 customer records. I want to do some analysis. I fed it to this. It has access to Snowflake and it's doing some querying for me where I can just use natural language to to do this analysis uh with it. And so this is my uh go-to for I have a question, I need to verify some data that's going into uh some research or report or summary for other executives. And I will say it on two fronts. One is yes, have a bot like this, but the other one is having a data team that can chime in and either help unblock it when it's not performing the way it should, as I like to say, kind of kick the vending machine or slap it, uh or verify the data is always important, uh to say, you know, I'm not just going to take a non-deterministic answer for this. Um, so this is one way I'm using it. Another way is Josh Childs is a member of our team. Um, he and many others have built their own uh tools to to do really cool things. And one of them is we're a fully remote async company. We have over 350 employees. I think over 400 now maybe uh it might be new news. And that means there's just a ton of conversations happening. You can see I've got 99 readers. I don't really use that feature very much in the left. 32 activity things that I need to clear out apparently. And then two DMs which I do actually clear out constantly. But there's just a lot of conversation happening within the company. And so, uh, Josh built for the sake of the product team a scanner. It uses AI and it goes through, I think, a few dozen channels that we have and just tries to find any conversations happening anywhere where a product manager should probably be involved. I think we got to the point with our company where it was just too difficult to to expect anyone to read through all the threads and conversations all the time because there's just so many things happening. So, this has been a huge help. It's it's a scanner. you know, we don't think of this as like a it's not a police car or something going around like patrolling. It's more of a this is our radar uh or our sonar. It's on all the time. And it's really helpful to just like deep link to a thread where you're like, "Hey, in this channel or on the support ticket with this customer over here, I see signs that a product person probably needs to weigh in, but I don't see any product person yet." And then tuning that to, you know, report at certain times of day and over be overly zealous. And then I as a leader amble able to take those and I said I know I blocked out a lot here because this is like a a specific issue but uh this is an excellent example of dot dot dot and I was able to say hey I I see this conversation happening. I see where a product person could create some value for the company but also you know let's not just stop at solving that question or answering that question. Let's also think of this as a process improvement opportunity or you know something we want to work on later and then kind of follow through and tag folks that can improve the way we work. Right? Um, so it's again it's awesome that we can upgrade the way we work with AI, but then it creates more opportunities to improve the way we work as well. So, so this is another example of I'm using it all the time to be more how does this help me? It helps me stay close to the ground, if you will, in terms of being really in tune with the all those moments that otherwise there's just no way to to be everywhere all at once. And I think it's helped to help me focus a little bit more when I when I am, you know, 200 replies deep in some analysis or or with Claude going through some kind of uh abstract exercise. The fact that I have this scanner running all the time, that still helps me pay attention to the details is important. And I think this this dovtales nicely to a lot of what we're hearing these days of, hey, leaders need to be players and, you know, players as well. Like I see involved as well. How can I do that? This is a this is a great way to do that. And then one more and I think it ties into what we were talking about uh earlier in this conversation is one. So we call this chiefy. This is something that Colin made and and he's mentioned this uh on stage before so I I feel comfortable sharing it. This is a bot uh that we've created that works again inside of Slack and it it has two primary use cases. One is anytime we create something new like that like that analysis like that pricing documentation etc. We can run that through Chiefy and it has this corpus of here's the 20 30 50 relevant company docs that are kind of the call it the gold standard or the the ratified verified documents that we operate on like the operating model if you will and it can help to find discrepancies and that can work two ways. So the other use case is there's a discrepancy because we release something new, we really like it, it it's aligned with our strategy, but hey, we have to go update all these other docs that have already been written or we need to correct those. You know, as a leader, it's really painful to publish something or create something gets alignment this month and then we do something new in 3 months and that document now is either stale or showing its age or needs to be updated. great to use AI like sort of another part of the stack to automatically go through you know dozen or two dozen or how many it is documents that you have and just help to audit those to know hey we need to bring those up to date or hey you know eight out of these other 12 disagree with this you might not realize it because you're very recency biased but is that intentional or not intentional and kind of you know being that accountability check to say oh yeah we yeah we didn't mean to say that we're going to do this instead of that we're actually doing both or oh yeah, we're actually changing our strategy a little bit. Let's go back and change those documents and update them. Super helpful. Obviously, you know, no one has time now to go back and look at all those. And that tends to be why notion gets stale and and these other places we publish to because there's just not enough time to keep up with the uh auditing and reviewing of past artifacts.
好问题。我觉得现实情况是，作为一个领导者，我主要活跃在几个地方。第一个，是的，我活在 Claude 桌面端里——具体到这里就是 Claude——以及搭配 co-work 的各种用法，正如我刚才走过的那些例子，整体上就是 co-work 加 Claude。我觉得另一个我一直待着的地方，显而易见，就是 Slack。我觉得 Customer.io 在把越来越多的 AI 和自动化引入 Slack 这件事上做得非常出色，所以我们有数量在不断增长的、内部自建的 agent，我每天都在用它们，基本上是用来做几件事。其中一件是临时性的即兴分析（ad hoc analysis）。这个例子里我把屏幕分享出来——我们有一个机器人，你能看到我跟它有一条 215 条回复的会话线程，所以这 215 条来回……其实差不多得算 400 多条，因为我同时还在用另一个机器人，在两者之间来回切换，这个我们待会儿可以聊。但这里的思路是：我手上有 2000 条客户记录，我想做一些分析，就把它喂给了这个机器人。它能访问 Snowflake，会帮我做查询，我只用自然语言就能跟它一起完成这些分析。所以每当我心里有个问题、需要核实一些要放进给其他高管看的研究、报告或总结里的数据时，这就是我的首选工具。我想强调两点。第一，是的，要有一个这样的机器人；但第二点同样重要——要有一个数据团队能随时插话，要么在它表现不如预期时帮忙排障（用我爱说的话讲，就是去「踹一脚自动售货机」、给它一巴掌），要么帮你核实数据，要知道，「我不会就这么接受一个非确定性的答案」。这是我用它的一种方式。另一种方式是——Josh Childs 是我们团队的成员，他和很多其他人都自己搭了一些工具来做一些很酷的事。其中之一是：我们是一家完全远程、异步协作的公司，有超过 350 名员工，我觉得现在可能超过 400 了，这或许还是个新消息。这意味着公司里时时刻刻都在发生海量的对话。你能看到我左边有 99 条未读，这个功能我其实不太用；还有 32 条待处理的活动提醒，看来我得清一清；以及 2 条私信，这个我倒是会经常清掉。但公司内部真的有大量对话在发生。所以 Josh 专门为产品团队搭了一个「扫描器」，它用 AI 去扫过我们几十个频道，专门去找任何地方正在发生的、产品经理大概应该介入的对话。我觉得我们公司发展到这个阶段，已经很难指望任何人能时时刻刻读完所有线程和对话了，因为同时发生的事情实在太多。所以这个东西帮了大忙。它是个扫描器——你知道，我们不把它想象成一辆四处巡逻的「警车」，而更像是我们的雷达或者声呐，它一直开着。它特别有用的一点是能直接深链到某条线程，告诉你：「嘿，在这个频道里、或者在跟这位客户的那张支持工单上，我看到有迹象表明大概需要一位产品同学来发表意见，但我还没看到有产品同学出现。」然后你再去调教它，让它在一天中的特定时段汇报、别太过于「热心」。然后我作为领导者就能拿着这些信息行动——我说一下，我在这里打码遮掉了很多内容，因为这是个很具体的问题，但这是个绝佳的例子，意思是「……」——我就能说：「嘿，我看到这段对话正在发生，我看到有个产品同学能在哪里为公司创造价值，但我们也别只停在解决或回答那个具体问题上，咱们也把它当成一个流程改进的机会，或者当成日后想去打磨的某件事」，然后跟进、@上那些能改进我们工作方式的人。对吧。所以再说一次，能用 AI 升级我们的工作方式固然很棒，但它同时也创造出了更多去改进工作方式的机会。所以这是又一个例子——我一直在用它，让自己……它怎么帮到我呢？它帮我「贴着地面」，可以这么说，让我真正与所有那些瞬间保持同频，否则你根本没办法「同时无处不在」。我觉得它还帮我更专注一点——当我在某段分析里已经深入到 200 条回复，或者跟 Claude 在做某种比较抽象的演练时，正因为我有这个扫描器一直在后台运行，它依然能帮我留意到那些细节，这一点很重要。我觉得这也很好地呼应了我们最近常听到的说法：嘿，领导者得既是「教练」也得是「球员（player）」——我也想亲自下场参与，我该怎么做到？这就是一个绝佳的途径。然后还有最后一个，我觉得它跟我们这次对话前面聊到的内容是连在一起的。我们管它叫 Chiefy，是 Colin 做的，他之前在台上也提过这个，所以我可以放心分享。这是一个我们做的机器人，同样在 Slack 内部运行，它有两个主要用途。第一个是：每当我们创建出某个新东西——比如那份分析、那份定价文档之类——我们都可以把它丢给 Chiefy 过一遍。它有一个语料库，里面是 20、30、50 份相关的公司文档，也就是所谓「黄金标准」或者说经过批准、核实过的、我们据以运作的那些文档——可以说是我们的「运营模型」。它能帮你找出其中的不一致之处，而这能往两个方向起作用。所以另一个用途是：之所以出现不一致，是因为我们发布了某个新东西、我们很喜欢它、它也跟我们的战略对齐，但是——嘿，我们得去把其他那一堆早就写好的文档全更新掉，或者得去修正它们。作为领导者，最痛苦的事之一就是：你这个月发布或创建了某样东西、并让大家达成了对齐，结果三个月后我们又做了点新东西，那份文档现在要么过时了、要么显出陈旧、要么需要更新。所以用 AI——把它当作技术栈里的又一块——去自动过一遍你手上那十几份、几十份、不管多少份文档，帮你审计一下，告诉你「嘿，这些得更新到最新了」，或者「嘿，这另外 12 份里有 8 份跟这份是冲突的，你可能没意识到，因为你有很强的近因偏差（recency bias）——这到底是有意为之还是无心之失？」——它就能扮演这种「问责检查」的角色，让你说「哦对，我们其实并不是想说要改做这个而不做那个，我们其实两个都在做」，或者「哦对，我们其实是在稍微调整战略，那咱们回去把那些文档改了、更新一下」。这真的超有用。显然，现在没人有时间回头去翻看所有那些文档。而这恰恰往往就是 Notion 之类的地方会变陈旧的原因——也包括我们发布内容的其他地方——因为大家实在抽不出足够的时间去跟进、去审计和复查过去那些产物。
[46:53] Aakash
This is really cool. So, what do people need to do to reverse engineer your Slack setup with AI here?
这真的太酷了。那么，人们要怎么做，才能「逆向工程」出你这套带 AI 的 Slack 配置呢？
[46:58] Matthew
Yeah. So, um it depends on on how your company works. Um we took a uh a strategy of in a controlled way letting there be experimentation with you know open claw and other agents like it. We actually have a sort of our own version of openclaw that we're working on where it's it's it's an agentic loop. We have individual team members who have that predilction for being technical who are building their own instances and then we are supporting them as a company and saying here's how to host it. Here's how to make it secure. Here's where it can run and live. And so I think that enablement is really key, making sure that there's a budget and that that space, that margin for people to experiment. So it starts from the top, heavy emphasis on experimentation and building with the latest and greatest. So this all kicked off uh in February, right, as OpenClaw was exploding. And so that that's what we did sort of reverse engineer it. Um OpenClaw, your own version of that, having a place for people to host their own instances and then letting those have access to Slack. Um I won't get into all the integration details because I'll probably misspeak. Um, but I think from a leadership team perspective, budget, support, and then lead by example, you know, use these tools, maintain these tools, give feedback on these tools, create some of your own. Um, that's all going to really help to to drive that.
是的。这取决于你们公司是怎么运作的。我们采取的策略是：在受控的方式下，允许大家拿 open claw 以及类似的其他 agent 做实验。我们其实自己在做一个 OpenClaw 的版本，它是一个 agentic 的循环。我们团队里有一些天生就有技术倾向的成员，他们在搭建自己的实例，而我们作为公司在背后支持他们，告诉他们：「这是部署托管的方法、这是确保安全的方法、这是它可以运行和落地的地方。」所以我觉得这种「赋能（enablement）」真的很关键——要确保有预算、有那个空间、有那份让大家去做实验的余裕（margin）。所以这一切是自上而下开始的，要大力强调实验、强调用最前沿的东西来搭建。所以这一整套是在二月份启动的，对吧，当时 OpenClaw 正在爆火。这就是我们逆向工程它的方式：OpenClaw、你自己的那个版本、给大家提供一个能托管自己实例的地方，然后让那些实例能接入 Slack。集成的所有细节我就不展开了，因为我大概率会说错。但从领导团队的角度，我觉得就是：预算、支持，然后以身作则——你自己去用这些工具、维护这些工具、对这些工具给出反馈、自己也亲手做几个。这些都会真正帮助推动这件事。
[48:10] Aakash
I think that's really eye opening because I keep talking to people about OpenClaw and they keep saying, "Oh, well, my company doesn't allow it." I like your guys's approach of we'll create our own version that is enterprise data safe so we can use it with our enterprise clients and then deploy it. Wow. So most content you guys have seen online, it's teaching you some advanced cloud code setup that is loading in millions of context files and your entire notion and your entire Slack and collecting all your MCP. We just showed you the realistic simple version of how you use AI to do the most important tasks that a product leader has to do and how you even enable your teams with some more advanced use cases to use things like Open Claw. If people want to get in touch with you to learn more, Matthew, where can they go?
我觉得这真的很有启发性，因为我一直在跟很多人聊 OpenClaw，他们老是说：「哦，我们公司不让用。」我很喜欢你们的做法——我们自己做一个「企业数据安全」的版本，这样我们就能拿它来对接我们的企业客户，然后再部署它。哇。所以你们在网上看到的大多数内容，教的都是某种高级的 Claude Code 配置——往里塞进几百万个上下文文件、你整个 Notion、你整个 Slack，还把你所有的 MCP 都收集进来。而我们刚刚给你展示的，是一个现实、简单的版本——产品领导者怎么用 AI 去完成自己手头最重要的那些任务，以及你甚至可以怎样用一些更进阶的用例去赋能你的团队，让他们用上 OpenClaw 这样的东西。Matthew，如果大家想联系你、了解更多，可以去哪里找你？
[48:58] Matthew
Uh, you can message me on LinkedIn. Uh, I do I do see those and I'm always recruiting, so you can find me there for sure. I check those. Um, you can also find me on X.
你可以在 LinkedIn 上给我发消息。我确实会看那些消息，而且我一直在招人，所以你绝对可以在那儿找到我，那些消息我都会查看。你也可以在 X 上找到我。
[49:06] Aakash
All right. I think it would be an amazing PM job if I were a PM. So, reach out to him if you are one of those AI native AI forward PMs who have watched all the way to the end of this episode. That means you are embracing AI in a way that I think customer would appreciate. Matthew, thank you so much for actually showing the real stuff. Nobody shows the real stuff. Really appreciate you.
好的。我觉得，要是我是个 PM，这会是一份特别棒的 PM 工作。所以，如果你正是那种「AI 原生、AI 优先」的 PM、又一路看到了本期节目的结尾，那就去联系他吧——这意味着你正在以一种我觉得 Customer.io 会很欣赏的方式拥抱 AI。Matthew，太感谢你了，感谢你真的把这些「真东西」展示了出来。没人会展示真东西的。真的非常感激你。
[49:27] Matthew
You're very welcome. Thanks for having me.
不客气。谢谢你邀请我。
[49:28] Aakash
All right, guys. See you in the next episode. I hope you enjoyed that episode. If you could take a moment to double check that you have followed on Apple and Spotify podcasts, subscribed on YouTube, left a rating or review on Apple or Spotify, and commented on YouTube, all these things will help the algorithm distribute the show to more and more people. As we distribute the show to more people, we can grow the show, improve the quality of the content and the production to get you better insights to stay ahead in your career. Finally, do check out my bundle at bundle.ac.com to get access to nine AI products for an entire year for free. This includes Dovetail, Mobin, Linear, Reforge, Build, Descript, and many other amazing tools that will help you as an AI product manager or builder succeed. I'll see you in the next episode.
好了，各位，我们下期节目再见。希望你喜欢这期节目。如果你能花点时间确认一下：你已经在 Apple 和 Spotify 播客上关注了我们、在 YouTube 上订阅了、在 Apple 或 Spotify 上留了评分或评论、并在 YouTube 上留了言——所有这些都会帮助算法把这档节目分发给越来越多的人。随着我们把节目分发给更多人，我们就能把节目做大、提升内容和制作的质量，从而给你带来更好的洞见，助你在职业生涯中保持领先。最后，记得去看看我在 bundle.ac.com 上的套装，可以免费获得九款 AI 产品整整一年的使用权，其中包括 Dovetail、Mobin、Linear、Reforge、Build、Descript，以及许多其他能帮你作为 AI 产品经理或构建者取得成功的好工具。我们下期节目见。