How Forward Deployed Engineering is done at Ramp — Leo Mehr

频道: AI Engineer
视频: https://www.youtube.com/watch?v=ITMXwI6QL6A
原文语言: en
统计: 共 16 轮 · Leo Mehr 13

[0:01]

[music]

[音乐]

[0:13] Leo Mehr

Awesome. Thank you, guys. Awesome. It's great to to meet everyone. I mean, I hope that after the talk, you know, if you want to come out and we can chat, we'd love to. Um Cool. So, yeah, today my goal is to share with you guys the two most important principles from what we learned doing FDE at Ramp. So, just yeah, briefly a little bit about myself. Yeah, I'm a director of engineering at Ramp. Uh I joined the company 2 and 1/2 years ago when it was just you know, FDE was just two engineers at the time. And today, my org is about 30 engineers across four deployed developer API and our new AI services um business. So, I know this is kind of a running theme, but like no one knows what FDE is. So, I'm just going to spend a moment on that. So, yeah, I I I actually kind of like this meme. It's To me, it's kind of funny. Um I um but I I I actually think it's like totally wrong. I don't see this as the actual like true form of what FDE is. Um I don't see it as like the final evolution or like boss mode of technical go-to-market roles.

太好了，谢谢大家。很高兴见到各位。等会儿讲完，如果你们想过来聊两句，我们非常欢迎。好，那今天我的目标，就是把我们在 Ramp 做 FDE 学到的两条最重要的原则分享给大家。先简单说说我自己：我是 Ramp 的工程总监，两年半前加入公司，那会儿 FDE 就只有两个工程师。今天我这个组大概 30 号工程师，分成四块，涵盖 deployed（驻场交付）、developer API，还有我们新的 AI services 业务。我知道这好像已经成了今天的一个共同话题——没人真的说得清 FDE 到底是干什么的。所以我花一点点时间讲讲这个。这张 meme 我其实还挺喜欢的，我觉得挺好笑。但我也觉得它完全不对。我不认为这是 FDE 的真身，我不觉得 FDE 是技术型 go-to-market 岗位的「最终进化形态」或者「满级 boss 形态」。

[1:27] Leo Mehr

Um now, this might be true at some companies, but at least at Ramp, it's a little bit different. So, FDE at Ramp, uh we live within the engineering organization. And our goal is to help Ramp win upmarket. So, with that in mind, what we do is we basically work on the core product and our new agentic features and make them work really well for our largest enterprise customers. So, that's just a little bit of intro context. I want to dig in and and today, like I said, there's just two things I'm going to share with you. Literally two things. Very easy talk. And these are the principles that I would say have really guided us and I would say probably the two most important things that we have. Always be scoping and scale with tokens. So, let's get with start with the first one. On scoping. So, I would say there's this thing where like people many many people think that as an FDE, your job is to just say yes to the customer.

在有些公司这么说可能确实成立，但至少在 Ramp 不太一样。Ramp 的 FDE 是长在工程组织里的，我们的目标是帮 Ramp 打赢上探大客户（win upmarket）这场仗。所以我们干的事，基本就是在核心产品和我们新的 agent 化功能上做工作，让它们在我们最大的那批 enterprise 客户那里跑得特别顺。背景就先交代到这儿。下面进入正题，像我刚才说的，今天我只讲两件事，真的就两件，所以这场分享特别轻松。这两条原则是真正在指导我们的东西，我觉得也大概是我们手上最重要的两样：always be scoping（永远在划范围），以及 scale with tokens（靠 token 扩张产能）。我们先从第一条说起，也就是 scoping。有个很普遍的误解：很多很多人觉得，做 FDE 的活儿就是对客户一路说 yes。

[2:33] Leo Mehr

But that's wrong. If you were just to say yes, you know, instead of like beautiful Waymos that we have driving us around in San Francisco, you'd have something like this, you know. Yeah, horses with like rockets strapped to their legs. And the point is you you want to help the customer be successful. You want to try to figure out a way to say yes. But you actually want to deliver good software. You need to build the right thing. So, you don't just endlessly say yes to people. And I I do want to share an example of something that I would say happens somewhat regularly in one form or another at Ramp. So, it's Friday night and an enterprise sales rep comes to us with an urgent request that this super important strategic logo is only going to close if we build out an SAP S/4HANA integration. And I think that the default engineering reflex is like, "Shit. Like, where are the SAP API docs? Like, where do I find them and how do I build this integration?"

但这是错的。如果你只会说 yes，那今天在旧金山满街跑着载我们的，就不是漂亮的 Waymo 了，而是这种东西——马腿上绑几支火箭。重点在于：你当然想帮客户成功，你也确实要想办法找出一条能说 yes 的路径。但你真正要交付的是好软件，你得做对的东西。所以你不能无止境地对人点头。我想举个例子，这类事在 Ramp 换着花样、隔三差五就会上演一次。周五晚上，一个 enterprise 销售跑过来提了个紧急需求：有个特别重要的战略客户，只有我们把 SAP S/4HANA 的集成做出来才肯签。我觉得工程师的本能反应通常是：「完了，SAP 的 API 文档在哪儿？我上哪儿找，这集成到底怎么写？」

[3:42] Leo Mehr

But what an what a well-trained FDE would do is like pause for a second and say, "Okay. First of all, like, what's driving the urgency here?" Like, one thing I've seen is I've seen sales reps who like go kind of crazy because it's like the end of the quarter and they're trying to hit their quota and close the deal and not because the customer is the one driving the urgency. So, that's like one example.

但一个训练有素的 FDE 会先停一秒，然后问：「等等，首先，这个『急』到底是谁在急？」我见过的一种情况是，销售之所以火烧眉毛，是因为季度末到了、他要冲自己的 quota 把单子签掉，而并不是客户那边真的在催。这就是其中一种情形。

[4:05]

[clears throat]

[清嗓]

[4:06] Leo Mehr

But, you know, as an FDE, you're asking tons of questions to gather context about what's important um and what actually is the right thing to build. And so, you might ask like, "Who's using this integration? Have we exhausted all the different workarounds? Is there something manual that we can do in the meantime? Does the customer have like technical resources? Can they hit our API such that we don't have to build this thing?" But, I'd say the most important thing that an FDE does is also looks beyond this one request and looks at the other prospects that are coming down the pipeline and other customers to see if anyone else would benefit from this as well. And the point is that by gaining all this context, you can do a better job of building the right thing. So, I want to share another story that was really really painful for us in the early days. We had this large enterprise customer and they needed this reimbursement feature on mobile.

作为 FDE，你要问一大堆问题，去收集上下文：到底什么才是重要的，真正该做的又是什么。所以你可能会问：这个集成到底谁在用？各种变通办法我们都试遍了吗？眼下有没有什么人工的法子能先顶一阵？客户自己有没有技术资源？他们能不能直接调我们的 API，这样我们干脆不用做这个东西？但我觉得 FDE 最重要的一个动作，是把目光从这一个需求上挪开，去看 pipeline 里正在推进的其他潜在客户、以及别的现有客户，看看还有没有谁也能从这件事里受益。道理很简单：把这些上下文都拿到手，你才更有把握把对的东西做出来。我再讲一个我们早期特别惨痛的故事。当时有个大 enterprise 客户，他们需要在手机端做一个报销（reimbursement）功能。

[5:10] Leo Mehr

Unfortunately, our mobile team was totally swamped. So, we basically just had to roll up our sleeves as FDEs and just get find out a way to get things done. And we had two of the engineers on the team just like learn how to do iOS and Android development. And it was awesome. We were super excited. We're like, "Okay, we're going to ship this feature. It's going to be so good. Like, hell yeah." So, we grinded for a couple weeks, got the feature done on both platforms. And we go to the customer and we're like, "Awesome. Like, can you send us your list of, you know, beta users for Android?" And that's when they told us they only they they they require they mandate all of their employees to use iOS devices. So, you're like, what the [ __ ] Like Not not to the customer, you know, just internally. But like obviously it was super disappointing for us because we'd put all this effort in. And so, it was a a big lesson for us to remember the importance of scoping. Even some of the most basic assumptions like which you know mobile platform you build on it's it's super important um to validate them and and thus kind of emphasizes the importance of scoping up front.

不巧的是，我们的 mobile 团队当时忙得完全脱不开身。所以我们这些 FDE 只能自己撸起袖子上，想办法把事办成。我们让组里两个工程师现学 iOS 和 Android 开发。那感觉特别爽，我们都很兴奋，心想：好，这个功能我们要上线了，肯定特别棒，太带劲了。于是我们埋头肝了几周，两个平台都做完了。然后我们去找客户说：搞定了，能不能把你们 Android 这边的 beta 用户名单发给我们？这时候他们才告诉我们，公司强制规定所有员工只能用 iOS 设备。你当时的反应就是：什么鬼啊。当然不是当着客户面说，就是内部自己骂两句。但确实特别沮丧，我们投进去那么多功夫。所以这对我们是个很大的教训，让我们牢牢记住 scoping 有多重要。哪怕是最基本的假设——比如你到底要在哪个移动平台上做——都非常有必要去验证一遍。这也恰恰说明了，前期先把范围划清楚有多关键。

[6:20] Leo Mehr

Now, Okay, so let's say that you and your team have become masters of scoping. You know, you're you're amazing. In today's world, this is not enough. So, unless you are scaling with model capabilities, you are going to fall behind. Now, I'm not going to belabor this point too much. I think like every talk in this uh in in this conference is probably some flavor of this, but like the point is that we basically have to reinvent our jobs constantly now. So, whatever work we are doing today, you know, for the most part it's knowledge work, we have to figure out how to have models and agents do it for us. And so, that brings me to the second half of this talk and the the other point that I want to convey today, which is all of us have to figure out how to scale with tokens. And the way that I interpret scaling with tokens for FTE is take a look at the whole life cycle of what an FTE does. From gathering context to scoping out a request to writing out a spec and then implementing the feature, each stage of that pipeline can be replaced with agents.

好，那么假设你和你的团队已经把 scoping 练成了绝活，做得特别漂亮。但在今天这个世界里，光有这个还不够。除非你能跟着模型能力一起往上走，否则你就会被落下。这一点我不打算多展开，我觉得这次大会上几乎每一场演讲讲的都是这件事的某个版本。核心意思是：我们现在基本上得不停地重新发明自己的工作。我们今天在做的事，大部分是知识工作，那我们就必须想清楚怎么让模型和 agent 替我们把它做掉。这就引出了这场演讲的后半部分，也是我今天想传达的第二个观点：我们每个人都得想明白怎么跟着 token 一起扩张（scaling with tokens）。对 FDE 来说，我理解的“跟着 token 扩张”是这样：把 FDE 的整条工作链路摊开看——从收集上下文，到给一个需求划范围，到写出 spec，再到把功能实现出来，这条流水线上的每一个环节，都可以换成由 agent 来做。

[7:33] Leo Mehr

And at first it seems kind of daunting. You're like, like how are you going to go and approach and like solve that? But if you break the problem down and then make progress on it, it's it's actually pretty tractable. And so, I'll share share with you guys one example of something Oops. Something that we um that we've done at Ramp. So, we have this internal Slack channel called FDE requests, and this is where account managers, solutions, uh sales reps will post whenever there is a blocker for a prospect or customer that's large enough, basically. And so, we get these requests. In this case, actually uh one of the CSMs on our team, Greg, posted here. And um if you were to It's actually a Notion workflow. If any of you work at Notion, by the way, thank you. We like use Notion so much. Um if you were to click open in Notion, you'd see like a pretty long request that has all the details of what what exactly it is.

乍一听会觉得挺唬人的，你会想：这玩意儿要怎么下手、怎么解？但只要把问题拆开，一步步往前推，其实相当可做。我给大家分享一个我们在 Ramp 做过的例子。哎哟——我们内部有个 Slack 频道叫 FDE requests，客户经理、解决方案团队、销售代表，只要碰到某个潜在客户或现有客户身上足够大的卡点，就会发到这个频道里。需求就是这么进来的。这一条其实是我们团队的一位 CSM，Greg，发的。这背后是一套 Notion 工作流——顺便说一句，在座如果有 Notion 的同事，谢谢你们，我们真的太依赖 Notion 了。你要是点开“在 Notion 中打开”，就会看到一份挺长的需求，里面写清了这事到底是怎么回事的所有细节。

[8:31] Leo Mehr

And the problem is there's a super high variance. Like, some people will submit like really detailed, good requests from the customer. And others are just going to submit like one line, like, "We need uh you know, we need this SAP integration." And before what would happen is we would have FDEs manually kind of go through this request. We'd read the whole thing, understand it, figure out what exists in the product, do a bunch of back and forth with the customer. And this is like exactly what the first half of the talk was about, always be scoping. You know, we would spend a lot of time really digging in and validating what exactly was uh you know, absolutely necessary. And so, you can see here what we what we did then was we basically um used Notion uh Notion agents to build a V1, which literally just took the request and asked a couple of questions. That was it. And um after It was kind of astonishing. Literally, after a couple of weeks, we found that it was like saving us a lot of time because, first of all, immediate like the latency of replies went from like hours or days to like, you know, seconds. And immediately, like the account reps that the account managers, the reps would start kind of engaging with this agent. And one of the things that we did was because it went so well, this this is actually um the more recent iteration of it. It's very cute, you know, the little penguin actually helps make it seem a little more friendly and approachable.

问题在于，这些需求的质量方差极大。有的人会从客户那边带回非常详细、写得很到位的需求；有的人就只丢一行字——“我们需要那个 SAP 集成”。以前的做法是，FDE 得人工一条条去啃：整份读完、理解清楚、搞明白产品里现在到底有什么、再跟客户来回沟通好几轮。这正是演讲前半段讲的那件事——always be scoping。我们会花大量时间去深挖、去验证到底哪些是绝对必要的。所以你们看，我们后来做的事，就是用 Notion 的 agent 搭了个 V1，它干的活字面意义上就是：接过需求，问几个问题。就这么点。结果挺让人吃惊的——上线才几周，我们就发现它给我们省了很多时间。首先是响应延迟，从几小时甚至几天，直接变成几秒。然后客户经理和销售代表马上就开始主动跟这个 agent 互动起来了。因为效果太好，我们就继续往下做——屏幕上这个其实是更近期的版本。它还挺可爱的，那只小企鹅确实让整个东西显得更友好、更好接近一点。

[9:55] Leo Mehr

Um and what it does is it actually goes and does several rounds of back and forth questioning with the submitter until it deems that it's ready to create a lot a spec, basically. And it's actually been incredible how helpful this has been for us. I I would say it's probably saved us like a large percentage, I don't know, 20% of the time that we'd spend on scoping out these requests. So, you know, this is this is a great example. For us, this has been really helpful. It's I'm super excited about this. It's going to help automate a lot of the work that we've been doing manually. But, um it's really just the first stage of this pipeline that I was alluding to. So, if you look at the first part here, like we've been able to make some progress on it. The last step as well, going from a a well-shaped spec to like a working product, obviously like Frontier models can like one-shot medium-size features. And so, the last part is also is is a lot easier for us. It's this middle part that I would say is super like gnarly and like unformed and difficult.

它做的事是：跟提需求的人来回追问好几轮，直到它判断可以出一份 spec 了为止。这东西对我们的帮助大到不可思议。我估计它给我们省下了很大一块——具体我说不准，大概两成吧——就是我们原本花在给这些需求划范围上的时间。所以这是个很好的例子，对我们真的很管用，我为此特别兴奋，它会把我们过去手工做的很多活自动化掉。但它其实只是我前面说的那条流水线的第一段。你看流水线的开头这部分，我们已经推进了一些。最后一步也一样，从一份成型的 spec 到一个能跑的产品——现在的前沿模型基本上可以一把（one-shot）做出中等规模的功能，所以最后那段对我们来说也容易多了。真正难啃的是中间这一段，我觉得那部分特别棘手、特别没成型、特别难搞。

[11:03] Leo Mehr

And I'm I'm really excited about our team kind of investing a lot more and spending a lot more of our time just like building out this factory, building out agents to replace each one of these steps. And the thing is, if you look at if I were to say 6 to 12 months from now, like what does FDE at Ramp look like? Like these are the sorts of applied AI problems that we're going to be spending all of our time on, I think. Like, you know, making sure that the agent harness that's running each of those steps is running super smoothly. Um making sure that the the output quality of each of the outputs of the the pipeline is is actually good. That you know, with with evals, with rubrics, with human feedback. Um and there's of course like one of the biggest challenges, which is getting your agent the right context, you know, when you're making the alarm call, ensuring that it has the right context.

我特别期待我们团队在这件事上投入更多，把更多时间花在搭这座“工厂”上——为每一个环节都造出对应的 agent。而且，如果让我说 6 到 12 个月之后 Ramp 的 FDE 长什么样：我觉得我们会把全部时间都花在这一类应用 AI 的问题上。比如，确保跑每一步的那套 agent harness 运转得足够顺畅；比如，确保流水线每一步的产出质量真的过关——靠 evals、靠 rubric、靠人类反馈。当然还有最大的挑战之一：怎么给你的 agent 喂对上下文——在发起那次模型调用的时候，保证它拿到的上下文是对的。

[11:57] Leo Mehr

So, there's like a lot of historical data, data about the the product. Imagine like all the knowledge that a product manager has in their head about their product. Like, how do you get that into an agent? Like, Notion docs and all your existing knowledge base and help articles only give you so much of that. Um yeah, skills, memories, tools. I could go on for a bit, but ultimately the most important thing here is that as an FD, we still have the responsibility of taste and judgment over the final output. So, that's going to be like the underlying kind of throughline. Okay. So, let's say that you've done an amazing job building out this factory. But the problem isn't to tie this to the first half of the talk. If you don't do a good job of scoping out requests or or building upon the principles of scoping things well, you're going to get a token maxing slop cannon. And so, the whole point is that you have to do these both because the other way around is actually quite bad as well.

这里面有大量历史数据、关于产品本身的数据。你想想一个产品经理脑子里装着的那些关于自家产品的知识——怎么才能把这些塞进一个 agent 里？Notion 文档、你现有的知识库、帮助中心文章，能覆盖的只有那么一点点。还有 skills、memories、tools，我可以一直往下讲。但归根到底，这里最重要的一点是：作为 FDE，最终产出的品味和判断，责任仍然在我们身上。这会是贯穿始终的那条主线。好，那么再假设你已经把这座工厂搭得非常漂亮。但问题来了——把它跟演讲前半段接上——如果你在给需求划范围这件事上做得不好，或者没有把 scoping 那套原则打在底下，你最后得到的会是一门“token 拉满的垃圾产出加农炮”。所以整个要点是：这两件事你都得做，因为反过来那种情况也相当糟糕。

[13:04] Leo Mehr

If you are, you know, amazing at scoping, but don't invest in building out this, you know, agent factory, you know, it's going to be over for you. Like, uh your your agent native competitors are just going to overtake you and outcompete. And so, that's why um in the end here, I want to close with the the the most important thing is that if you have both of these, it can set you up for success in the future. Always be scoping and scaling with tokens. The future of FD needs both. That's all. Thank you, guys.

如果你 scoping 做得极好，却不肯在这套 agent 工厂上投入，那你也一样完蛋——那些原生就靠 agent 干活的竞争对手会直接超过你、把你打掉。所以最后我想这样收尾：最重要的是，这两样你都得有，才能撑起你未来的胜算。Always be scoping，同时 scaling with tokens。FDE 的未来，两样都需要。就讲到这儿，谢谢大家。

[13:45]

[applause] [music]

[掌声][音乐]