Context is now the product: Product leadership when software can build itself | Karri Saarinen

频道: Lenny's Podcast
视频: https://www.youtube.com/watch?v=Zn9NZ-r1-C4
原文语言: en
统计: 共 12 轮 · Karri 11

[0:06] Karri

All right, good afternoon everyone. Um, I'm here today to talk about great products, context, and the product work we do. But first, I want to talk about my summer. Um, I so I took a break this summer from paying attention to AI. I stopped following the news like what is the latest model? What can it do? what agents of do we have and what what are the new techniques we can use and when I was I was on this vacation I I thought like when I when I come back maybe I will feel left behind maybe things change a lot but it realized that when I came came back actually realized that nothing has changed much um there's new models there's new techniques there's new agents but I think like in the end that the making great products and building like increasing revenue or building the business like it's still hard so it made me wonder how much attention we're really paying to these tools. Like we are really watching the tools a lot all the time. There's always new things we can try and it's clear that these tools are getting better. They can do more. They can do it faster. They're more capable. But the I think more important question is like are we actually making better things with these tools? And are our teams getting better with these tools? And I I don't think that that answer is is clear clearly yes. Like if you look out there and like even in your inside your own companies, how do you even know like how are you making better things? Do you feel that way? Does your customers feel that way?

好，大家下午好。我今天来，是想聊聊好产品、上下文，还有我们做的产品工作。不过在这之前，我想先说说我的夏天。今年夏天我给自己放了个假，不去关注 AI 了。我不再追那些新闻——最新的模型是哪个？它能做什么？现在有哪些 agent？又有什么新技巧可以用？休假的时候我就在想，等我回来，也许会觉得自己被甩在后面了，也许很多东西都变了。结果回来以后我发现，其实没怎么变。是有新模型、新技巧、新 agent，但说到底，做出好产品、把收入做上去、把生意做起来，这件事还是很难。所以我就在想，我们到底在这些工具上放了多少注意力。我们真的是一直在盯着工具看，一刻不停。总有新东西可以试，而且这些工具显然在变好：能做的事更多，做得更快，能力更强。但我觉得更重要的问题是：我们真的用这些工具做出更好的东西了吗？我们的团队有没有因为这些工具变得更好？我不觉得这个答案是明明白白的「是」。你往外面看看，甚至就看你们自己公司内部——你到底怎么知道你们做出来的东西更好了？你自己有这种感觉吗？你的客户有这种感觉吗？

[1:35] Karri

So there's a lot of emphasis now on tool making and using tools and building systems. But I think like in the end like as a product people our job is to to make something good. And so in some ways like today I think there's been several talks about software factories and I think that the industry has been I think obsessed with this idea a long time like how do we scale and optimize the output our organizations do and I think it makes sense and it is it is important to to move fast and like do things but this started even before AI and I think before AI the the idea was that we need to hire more people we create very specialized roles so they work in their own ways and then we create because there's a lot of people we create more process and because we have process and a lot of people we actually don't even know anymore like what what people are doing or what they're supposed to do. So we run experiments. So experiments the data tell us like what what works and what doesn't and this is all to like scale like how do we get more output than like how do we get more um output from the organization but again like my philosophy as always it's more more [clears throat] output isn't better like I I think like what we want to build is like better experiences for the customers one of my messages today is that don't like you can build the software factories and you can make and you can automate things and that all makes sense but just like as an organization Don't become one. Don't become a software factory where you out like outsource all the work and thinking to somewhere else and like just focus on like how efficient can the output be because the output is not the product.

所以现在大家很强调造工具、用工具、搭系统。但我觉得说到底，作为做产品的人，我们的工作是做出好东西。从某种意义上说——今天已经有好几场演讲讲到软件工厂了——我觉得这个行业对这个想法着迷很久了：怎么把组织的产出规模化、做优化。我觉得这说得通，跑得快、把事情做出来确实重要。但这件事在 AI 之前就开始了。AI 之前的思路是：我们得招更多人，设出非常细分的专门岗位，让他们各干各的；然后因为人多，就加更多流程；又因为有流程、人又多，我们其实已经搞不清大家到底在做什么、应该做什么了。于是我们就跑实验，让实验、让数据来告诉我们什么管用、什么不管用。这一切都是为了规模化——怎么从组织里拿到更多产出。但还是那句话，我的理念一直是：产出更多不等于更好。我觉得我们想做的，是给客户更好的体验。我今天想传达的信息之一就是：你可以建软件工厂，可以把事情自动化，这些都说得通，但作为一个组织，别让自己变成软件工厂。别变成那种把所有的活儿和思考都外包到别处、只盯着产出效率能有多高的软件工厂，因为产出不是产品。

[3:11] Karri

Like customers are not buying the lines of code. They're not buying the experiments or they're not buying a lot of things. So in the end like if you want to sell something or make some good product you have to make something someone wants. When I think about making things I think about two things. So making products produces two things. The the product and the learning. Historically the building products or designing them it's always created this learning as well. Like when you are struggling like trying to think like what you should build, how you should build it, which way, what kind of shape does it take, you have all these questions you are asking and maybe you're answering them yourself. Maybe you go talk to someone, maybe you talk to the customers and try to find the answer. So the actual effort you put into building things, it's also like teaches you something about what the problem is, what the customers uh want or like what what is their view. So we're kind of like in this time now that there's there's a little bit this danger of losing that uh direct direct connection with the with the learning. And I think like if you think about any great company out there or companies that create great products, you probably respect their product, but you probably also respect their team. And I think like products are usually just like it's kind of obvious, but they're like they're produced by the team in the company. And the better the team, the better they understand the space they're in, the better skills or talents they have, the better taste they have, like the better they build things. And so like great companies they consistently be able to to build great things because they have this culture and context built around like how what how to think about things like what matters what doesn't matter. So this is like all the context around customers about the product space they're in the technologies they can use

客户买的不是一行行代码，不是那些实验，也不是别的一大堆东西。所以说到底，你要想卖出点什么、做出个好产品，就得做出有人想要的东西。说到做东西，我会想到两件事。做产品会产出两样东西：产品，和学习。一直以来，做产品、设计产品的过程本身也总是带来这种学习。当你在那儿挣扎，琢磨该做什么、怎么做、走哪条路、它该长成什么形态，你会问出一大堆问题，有的你自己回答，有的你去找人聊，有的你去找客户聊、想办法找到答案。所以你在做东西上实实在在花的那份力气，也在教你一些东西：问题到底是什么，客户想要什么，他们是怎么看的。而我们现在所处的这个时期，多少有点危险，就是可能丢掉和这种学习之间的直接联系。你想想外面任何一家伟大的公司，或者那些做出好产品的公司，你多半敬佩他们的产品，但你多半也敬佩他们的团队。产品嘛——这话有点理所当然——就是公司里的团队做出来的。团队越好，越懂自己所在的领域，技能和才华越强，品味越好，做出来的东西就越好。所以伟大的公司能持续不断地做出好东西，是因为他们围绕「怎么想问题、什么重要、什么不重要」建立起了一套文化和上下文。也就是所有这些上下文：关于客户的，关于他们所在的产品领域的，关于他们能用的技术的，

[5:01] Karri

the judgment the team uses to make decisions the taste they have to actually find what is good and all the history of the different things they've tried or or what they decided. And the image here is just a illustration of this kind of like compounding learning. It's a Formula 1 steering wheel which started with very kind of traditional form. But over the decades the the the kind of like the the team learned to how to really make it better and make it something that makes sense for this use case. Like Formula 1 cars, they don't need a lot of turning radius. So you need a lot of like small movements and like controlling the car in a high speed. It's very fine movements and then that's why the the actual steering wheel ended up different than what we typically see in cars. The danger really now with the product organization is that the more we automate stuff um to AI to do and more we automate it or tell teams to use AI, you kind of like create this separation from execution and from the learning. It's not necessarily bad thing like it's what I'm calling this out is that if this is the case and this is happening in your company like how are you going to fix it like eventually if you don't learn from the work you do I think eventually you would lose the advantage of what you have currently in your in your space or in your company one way to think about it is like well what is good to automate maybe what what is what is good for people to do I think there's definitely things that are repeatable and maybe not something you learn a lot about like even At winner we do now how to fix bugs. We have a we use it the winner loop to investigate the bug. It will connect to data do and sentry and other tools. It will try to look into the codebase and and try to understand where the bug is coming from and then it will write a fix. So it saves the time for the engineers like they don't have to investigate the

团队做决策时用的判断力，真正分辨出什么是好的那份品味，还有他们试过的各种东西、做过的各种决定的全部历史。这里这张图，就是用来示意这种复利式的学习积累的。这是 Formula 1 赛车的方向盘，一开始是很传统的样子。但几十年下来，车队学会了怎么把它真正做好，做成适合这个使用场景的东西。Formula 1 赛车不需要很大的转向幅度，需要的是大量小幅度的动作，在高速下控制车子，都是非常精细的动作，所以方向盘最后就长成了跟我们平常在车上见到的不一样的样子。现在产品组织真正的危险在于：我们越是把事情自动化、交给 AI 去做，越是自动化、越是让团队去用 AI，就越是把执行和学习分开了。这不一定是坏事——我把它点出来是想说，如果情况确实如此、你公司里正在发生这件事，那你打算怎么补？因为如果你不从自己做的工作里学习，我觉得你最终会丢掉你现在在这个领域、在这家公司里拥有的优势。可以这么想：什么适合自动化，什么适合让人来做。我觉得肯定有些事是重复性的，可能也从里面学不到太多东西。比如在 Linear，我们现在修 bug 就是这样。我们用 Linear 的这套 loop 去排查 bug：它会连上 Datadog、Sentry 和其他工具，会去翻代码库，想办法弄清楚这个 bug 是从哪儿来的，然后写出一个修复。这样就省了工程师的时间，他们不用自己去排查这些

[6:49] Karri

problems. They come and and verify the result and and sometimes they make little changes for those results. But so the idea is like how do you save time um now with these new new tools but without really like outsourcing everything. Um so where I think about like the team should spend more time in is in the customer space and the customer problems. From the beginning, we always told at of the beginning of the company, we we've been pushing the engineers and all the team members really go connect with the customers, join the Slack channels, join the shared Slack channels, answer their questions, ask them questions, and like kind of like stay close to the customer and kind of like build your customer intuition that way. And I think like now if engineering or other roles have some time savings I think there's areas they can do more in like they can spend more time with customers they can explore more things they can train their judgment or or make better quality uh work than they previously had time to. So the goal I think really for the product organization in the future is like how do you keep this learning loop happening like how do you have context and and and bring signals from that context and and bring those learnings into the whole team and the whole company and I think like actually now AI can be a great tool in this. So a lot of times people I think focus on what AI can do what it can execute but not what AI can teach you. So I think for example at linear what we do is that we use the AI to help us to build this context. So we collect all the customer information that we can. Um we have automations that pull uh customer feedback from sales calls and other other meetings uh as well as from support emails and and internal uh discussions and that's kind of like builds this like a place for the context and place for the the kind of like the customer context and then like some something I started you doing recently

问题。他们过来核对一下结果，有时候会在这些结果上做点小改动。所以思路就是：有了这些新工具，怎么把时间省下来，但又不是真的把所有事情都外包出去。那我觉得团队该多花时间的地方，是客户那边，是客户的问题。从一开始——从公司创立之初——我们就一直在推工程师和所有团队成员真的去跟客户接上：加入 Slack 频道，加入那些共享的 Slack 频道，回答客户的问题，也向客户提问，贴近客户，靠这种方式养出自己的客户直觉。我觉得现在如果工程或者其他岗位省出了一些时间，有些地方他们是可以多做的：可以多花时间和客户在一起，可以多做探索，可以锻炼自己的判断力，或者做出比以前有时间做的更高质量的工作。所以我觉得产品组织未来真正的目标是：怎么让这个学习循环一直转下去——怎么拥有上下文，从上下文里带出信号，再把这些学到的东西带给整个团队、整个公司。而且我觉得现在 AI 其实可以成为这件事上的好工具。很多时候大家盯着的是 AI 能做什么、能执行什么，而不是 AI 能教你什么。比如在 Linear，我们的做法是用 AI 来帮我们搭建这份上下文。我们把能收集到的客户信息全都收集起来。我们有一些自动化，会从销售电话和其他会议里把客户反馈拉出来，也从客服邮件和内部讨论里拉。这样就搭出了一个放上下文的地方，一个放客户上下文的地方。然后，我最近开始做的一件事

[8:54] Karri

is that the problem previously I think in a product organization especially it's a large one a lot of people start being kind of away from the customer problems on the day-to-day problems because there's so much of it and and you don't have the time to follow up with everyone or look at every single email or every request. But what I started doing is that we we collect all this context, but then I sent this like a watchers like an agent to watch it and see tell me when when there's something interesting for me. So one of these things I set up is this daily briefing about AI workflows. what are the customers saying about their AI workflows? I want to learn more from it because I've seen a lot of companies do it very differently. Uh today we I think there's no clear best practices. Every company does their stuff a little bit differently. And so that's why I'm trying to like learn more that like what are all the ways people are using these tools? What what do they want from us and how can we help them better? So this can it's it doesn't take a lot of time for my day to just read this. It gets few bullet points usually but I get this like a daily briefing basically saying like the these are the customer what they're saying about the AI workflows.

是这样——以前的问题在于，我觉得在产品组织里，尤其是规模大的组织，很多人会慢慢离客户的问题、离那些日常的问题有点远，因为量实在太大了，你没时间去跟每个人逐一跟进，也没法把每一封邮件、每一条需求都看一遍。但我开始做的是：我们把这些上下文都收集起来，然后我放了一个类似 watcher 的东西、就是一个 agent 去盯着它，一有我可能感兴趣的东西就告诉我。我设的其中一个，就是关于 AI 工作流的每日简报：客户都在怎么说他们的 AI 工作流？我想从这里面多学一点，因为我看到很多公司的做法差别非常大。呃，今天我觉得还没有什么明确的最佳实践，每家公司做事的方式都有点不一样。所以我才想多了解一些：大家用这些工具的方式到底有哪些？他们想从我们这儿得到什么？我们怎么才能更好地帮到他们？这个东西……读一下花不了我一天里多少时间，通常就那么几条要点，但我基本上每天都能拿到这么一份简报，告诉我客户对 AI 工作流都说了些什么。

[10:09] Karri

On the judgment side I wanted to highlight couple things we do. I think what I've also noticed that when companies start using AI more and agents more people start uh siloing themselves like a lot of people are working alone with their agents building things and I think it's efficient but then there's also the problems like we not learning from each other anymore um or sport as much so one of the practices we start doing is this quality Wednesday and this happened because we as We keep hiring people. We we notice that not everyone has the same standard or understanding what what the quality means. And so we task everyone to go every week, spend some time, look into the product, try to find one quality defect in the product and then fix it. It can be very small. It can be weird hover state like janky animation. It can be a copy error. It can be something else. So it doesn't mean that like it should be like a huge project. It it could be like a five minute fix. The bigger impact with this is not really that we do this fixes. We do like that too. But the bigger impact is that the team trains everyone trains themselves to look for this like they train their eye to notice the small mistakes and then because we do this in a meeting everyone shares their findings and their fixes. Everyone can learn from that and see how other people are finding these problems. And then the second thing we do for new features is this feature roast. Uh it's a little similar but a little different where it's this optional meeting. Anyone in a company can join the team who is building the feature hostess and then what they ask people to do is just critique the whole feature. Like you can be as nitpicky or confused or whatever you want. You just put your raw feedback and it's not personal. It's just like this this is how I see it. And the team can still like interact with the people and ask them questions if they don't

在判断力这一块，我想重点讲几件我们在做的事。我还注意到一点：当公司用 AI、用 agent 用得越来越多，大家就开始各干各的——很多人是一个人带着自己的 agent 在那儿做东西。我觉得这样是高效，但也带来了问题，比如我们不再互相学习了，嗯，也不像以前那样互相切磋了。所以我们开始做的其中一个实践，就是 Quality Wednesday（质量星期三）。之所以有这个，是因为随着我们不断招人，我们发现不是每个人对质量的标准、对「质量到底意味着什么」的理解都一样。所以我们给每个人布置了个任务：每周花点时间，进到产品里看一看，试着在产品里找出一个质量缺陷，然后把它修掉。可以非常小——可以是某个怪怪的 hover 状态，比如卡顿的动画；可以是一处文案错误；也可以是别的什么。所以这并不是说它得是个大项目，可能就是个五分钟的修复。这件事更大的影响，其实不在于我们做了这些修复——修掉我们当然也乐意——更大的影响在于，团队、每个人都在训练自己去找这类问题，训练自己的眼睛去注意到那些小毛病。而且因为我们是在一个会上做这件事，每个人都会分享自己发现了什么、怎么修的，大家都能从中学到东西，看到别人是怎么发现这些问题的。第二件事是针对新功能的，叫 feature roast（功能吐槽会）。呃，跟前面那个有点像，又有点不一样：这是个自愿参加的会，公司里任何人都可以来，由正在做这个功能的团队来主持，他们请大家做的就是把整个功能从头到尾评议一遍。你想多挑剔就多挑剔，想多困惑就多困惑，随你。你就把最原始的反馈丢出来，这不针对个人，就是「我看到的就是这样」。团队也还是可以跟这些人互动、向他们提问，如果他们没

[12:07] Karri

understand something. Again, like we're trying to train people that what matters and like what do they we care about and also like kind of train um give [clears throat] the team who is building the feature a little more like realistic way to see what maybe how the users would think about the feature because a lot of people in a company this is the first time for them seeing the feature. So they if they are confused it's likely that the the users are confused too. And then like after that the lead can synthesize the feedback and and make group it and make some kind of actionable issues out of it and then they can like fix it. But like I think the important part is that these are done together and there's this like discussion that happens with these meetings and that that kind of like gives everyone a little more everyone can learn from each other and they notice like when you're building on your own product or your own feature you remember one of these discussions and you know that oh those that person always cares about the onboarding or they care about this animation so I must I should check into that and like remember to do it so I don't get that feedback. So my like really the ask is that we we we have this idea that well AI can do a lot of stuff and like let's make everything more efficient. Let's produce more. But if if AI is really efficient and it's saving time, then maybe there's an opportunity to spend more of that time to something else.

听懂什么的话。还是那句话，我们是想训练大家明白什么才重要、我们在乎的是什么，同时也算是训练……嗯，让正在做这个功能的团队能用更贴近真实一点的方式，看到用户可能会怎么看这个功能——因为对公司里很多人来说，这是他们头一回见到这个功能。所以如果他们看得一头雾水，用户多半也会一头雾水。在那之后，负责人可以把这些反馈综合一下、归归类，整理成一些可以动手处理的 issue，然后他们就可以去修。但我觉得重要的是，这些事是大家一起做的，这些会上会有讨论发生，这就让每个人多了一点……每个人都能互相学习。然后他们会发现，当你在做自己的产品、自己的功能时，你会想起其中某一次讨论，心里知道：哦，那个人总是特别在意 onboarding，或者他们在意这个动画，所以我得……我应该去查一下那块，记得把它做了，免得又收到那条反馈。所以我真正想提的请求是这样：我们现在都有这么个想法——好，AI 能干很多事，那就把一切都变得更高效，多产出一些。但如果 AI 真的很高效、真的在省时间，那或许就有机会把这些时间更多地花到别的事情上。

[13:27] Karri

Like rather than focusing on the execution, we could sort of start focusing on the customers, the explorations, the the critiques, the quality or the even the reflection of what we do. I think like in the future the product organization maybe that the work we do changes and we move away from the spending so much time on the execution but actually going a little more abstract and like thinking about the the actual activities and like how do we become better as a team and how we can become better as a as a product organization. what I said earlier that the the I think companies are often compound understanding of the work and the the problems they they uh solve. So I think there's an important aspect that you should have some kind of place where you put this stuff like there's some place that's available for learning about the customer context or learning about your product thinking or learning about anything else you're doing and then I think that it's important for the people but it's also important for the agents. So I think now a lot of times I can now ask the agents like what do we have about this or teach me something about the customers like what are they saying and what are they doing and and like what how is the team responding to that. I think there is a value of like creating this like a shared space where this context lives um around the customers and the product and the the work that is happening around the product. I will end end this with this that I think that the tools will keep improving. And I think we will keep watching the tools and I think we will keep automating more and more stuff. But I think my view is that we should automate more of the known things like the things that are repeatable and maybe are not that impactful. It's not something we can learn a lot from. My hope is that like we can spend more time like how do we build the understanding of the what we're doing in this company

比如与其盯着执行，我们可以开始把注意力放到客户、探索、评议、质量上，甚至放到对我们所做之事的反思上。我觉得未来的产品组织，也许我们做的工作会变，我们不再把那么多时间花在执行上，而是往更抽象一点的层面走，去想那些实际的活动，想我们作为一个团队怎么变得更好，作为一个产品组织怎么变得更好。就像我前面说的，我觉得公司往往就是对所做的工作、对所解决的问题的理解，一点点复利累积出来的。所以我觉得有一点很重要：你得有个地方来放这些东西——有那么一个地方，可以让人去了解客户上下文、了解你们的产品思考，或者了解你们在做的任何其他事情。我觉得这对人很重要，对 agent 也同样重要。所以现在很多时候，我可以直接问 agent：关于这个我们手上有什么？或者，给我讲讲客户的情况——他们在说什么、在做什么，团队又是怎么回应的。我觉得打造这么一个共享空间是有价值的，让这些上下文住在里面——嗯，围绕客户、围绕产品，以及围绕产品正在发生的那些工作。我就用这一点来收尾：我觉得工具会不断进步，我们也会一直盯着这些工具，会不断把越来越多的事情自动化。但我的看法是，我们应该更多地去自动化那些已知的事——那些可重复、可能影响也没那么大的事，那些我们从中学不到太多东西的事。我的希望是，我们能把更多时间花在这上面：怎么建立起对我们在做的事的理解——在这家公司里

[15:11] Karri

or in this product organization like how do we make sure that the team is learning from something and from someone from somewhere and not just executing the next prototype or the next idea or something because I think I we we at linear we don't run experiments we we use we tell people to use their intuition but I often tell people that intuition is not some like magical force that like and Star Wars or something that it's happens to you. It it is something that you have learned while working on something or listening to customers or something. Intuition is basically your training in your brain that you have. And so the more you can learn and and listen and and see this context, I think it will compound your personal understanding and then eventually compounds the whole team's understanding. And so in the end I like what what this talk was about is like context becomes the product. Then like it's it's more that we we have this obsession to look at what is the output the software or the code but I think in the end it's it's I I look the other direction it's like what is actually creating the software like who is making the decisions and that's the people are doing that and the better context those people have and like better understanding they have about the what they're supposed to do I think the products are better and so at linear I always thought that hiring people is the first step building great products and maybe it's kind of obvious but I think sometimes people just think that I I need to hire to do something I need to increase the output but what you're really like trying to do is think about like what is the trajectory this person can create for the company like what kind of ideas they have like do they have the right judgment or taste how can they use it in this organization so in the end I think like more of the product organizations maybe comes around the it becomes the context like more of the

或者说在这个产品组织里——怎么确保团队是在从某件事、某个人、某个地方学到东西，而不只是在执行下一个原型、下一个想法之类的。因为我觉得……我们在 Linear 是不跑实验的，我们让大家用自己的直觉。但我常跟人说，直觉不是什么神奇的力量，像《星球大战》里那种，莫名其妙就降临到你身上。它是你在做某件事、听客户说话之类的过程中学来的东西。直觉基本上就是你大脑里已经有的那份训练。所以你能学到、听到、看到的上下文越多，我觉得它就会让你个人的理解复利式地累积，最终让整个团队的理解也复利式地累积起来。所以说到底，这场演讲讲的就是：上下文成了产品。更确切地说，我们总有一种执念，盯着产出看——软件也好、代码也好。但我觉得归根结底……我是往另一个方向看的：到底是什么在创造软件？是谁在做决定？是人在做这件事。这些人拥有的上下文越好，对自己该做什么理解得越透，我觉得产品就越好。所以在 Linear，我一直认为招人是做出好产品的第一步。这也许有点理所当然，但我觉得有时候人们只是想着：我得招人来干某件事，我得把产出提上去。可你真正要做的，是去想这个人能给公司带来怎样的轨迹——他们有什么样的想法？有没有对的判断力或品味？在这个组织里又能怎么把它用起来？所以最终，我觉得产品组织可能会有更多的部分围绕着……它会变成上下文，更多的

[17:01] Karri

work is about managing this context like re learning from it finding ways to to team to come together and learn learn from it versus the actual execution of the code or or building the thing so that's what I mean like context becomes the product thank

工作是在管理这些上下文——从中学习，想办法让团队聚到一起、从中学习，而不是真的去执行代码、或者把那个东西造出来。这就是我说的「上下文成了产品」的意思。谢谢

[17:19]

[applause]

（掌声）