How I automate my own job at Hugging Face using agents — Niels Rogge, Hugging Face
频道: AI Engineer
视频: https://www.youtube.com/watch?v=FLUoowDJg4I
原文语言: en
统计: 共 27 轮 · Niels 26
[0:16] Niels
Okay. All right. Hello everyone. Thanks for coming by. Today I'll talk about how I automate my own job at Hugging Face using agents. Um short introduction. I'm just uh Niels from Belgium, the land of beer, fries, and chocolate. I studied at KU Leuven, and I'm a machine learning engineer at Hugging Face for 5 years now. Uh today I'll talk about the community science team at Hugging Face, which is the team I'm part of. Uh then I'll talk about how I automate large parts of the community science team. And finally, I'll also discuss some other efforts uh that we do at Hugging Face. So, let's start with the community science team at Hugging Face.
好，那我们开始。大家好，感谢大家过来。今天我要讲的是我如何用 agent 来自动化我自己在 Hugging Face 的工作。先简单自我介绍一下：我是 Niels，来自比利时——啤酒、薯条和巧克力之乡。我毕业于 KU Leuven（鲁汶大学），在 Hugging Face 做机器学习工程师已经五年了。今天我会先讲讲 Hugging Face 的 community science 团队，也就是我所在的团队；然后讲我是怎么把 community science 团队的大部分工作自动化的；最后我还会聊聊我们在 Hugging Face 做的其他一些事情。那就先从 Hugging Face 的 community science 团队说起。
[1:02] Niels
So, basically, this started when I was sent I was seeing like trending research uh passing by on GitHub. And a lot of times when I saw new interesting work, um the weights were not available on Hugging Face, sadly. Like, researchers use either Google Drive or they use GitHub releases. They use Dropbox, they use Zenodo, or other servers to put their um artifacts on. And this hurts uh discoverability of their work. It's like not easily uh visible or discoverable. And when I then open a GitHub issue to say like, "Actually, you could put your weights on Hugging Face for free." Most of the time, people replied to me like, "Yeah, migrating the weights uh from Google Drive to Hugging Face actually makes perfect sense."
事情的起因是这样的：我经常在 GitHub 上看到热门的研究成果刷屏。很多时候我看到有趣的新工作，遗憾的是它们的权重并没有放在 Hugging Face 上。研究者们要么用 Google Drive，要么用 GitHub releases，要么用 Dropbox、Zenodo 或者其他服务器来存放他们的产物。这其实很伤他们工作的可发现性——别人不容易看到、不容易找到。于是我就去开 GitHub issue 跟他们说：「其实你们可以把权重免费放到 Hugging Face 上。」大多数时候对方都会回复我：「对啊，把权重从 Google Drive 迁到 Hugging Face 确实很有道理。」
[1:46] Niels
So, yeah, the community science team can also uh be described as the Google Drive to the hub team. Um why? Because on Hugging Face, we have these paper pages, uh and every single paper is from archive. And then on the right side, you can basically list the linked artifacts, like the the linked models or data sets. So, people can easily reproduce your paper or find the models or data sets. So, yeah, you can see them on the right side. Um this improves the discoverability of your work because we have these metadata tags or filters on the app, so you can easily find, for example, depth estimation model, an LLM if you're interested. You can find them by language. You can tag them with the library they are compatible with and so on. So, this improves the discoverability of your work. So, these are, yeah, the metadata tags that you can add to every single model on Hugging Face or every single data set.
所以说，community science 团队也可以被叫做「Google Drive 搬运到 Hub」团队。为什么这么说？因为 Hugging Face 上有论文页面（paper pages），每一篇论文都来自 arXiv。页面右侧可以列出与这篇论文关联的产物，比如关联的模型或数据集，这样别人就能轻松复现你的论文、找到对应的模型和数据集——你们可以在右边看到它们。这提升了你工作的可发现性，因为我们平台上有各种元数据标签和筛选器，你可以很容易地找到比如一个深度估计模型，或者你感兴趣的 LLM；可以按语言找，可以标注它兼容哪个库，等等。这些都让你的工作更容易被发现。这些就是你可以给 Hugging Face 上每一个模型、每一个数据集加上的元数据标签。
[2:39] Niels
So, yeah, this is like the main problem that we saw, like, lots of people, lots of researchers are like using third-party services to publish their work. We have the Hugging Face platform, which is like a centralized place where people can find machine learning artifacts. It also improves with documentation because you can add a model card or a data set card. We have tooling, so you can easily upload or download stuff from Hugging Face. And it might also help reach researchers in promoting their work. So, it's basically a win-win both for researchers and then other people using the research. So, yeah, these are the typical GitHub issues that I was opening. I always had like the same template. I just asked, "Could you please release this checkpoint on Hugging Face? Could you please release this data set on Hugging Face?"
所以这就是我们看到的核心问题：很多人、很多研究者在用第三方服务发布他们的成果。而我们有 Hugging Face 这个平台，一个让大家集中查找机器学习产物的地方。它在文档方面也更好，因为你可以加 model card 或者 dataset card；我们有配套工具，上传下载都很方便；它还可能帮研究者更好地推广自己的工作。所以这基本上是双赢——对研究者是好事，对使用这些研究的人也是好事。这些就是我当时开的典型 GitHub issue，我一直用同一套模板，就是问：「能不能请你把这个 checkpoint 发布到 Hugging Face 上？能不能请你把这个数据集发布到 Hugging Face 上？」
[3:23] Niels
And [snorts] then I also opened PRs, pull requests on Hugging Face to add data set cards or model cards to improve the documentation of those artifacts. But, there's a problem. It's not really scalable for me to open all these GitHub issues or pull requests because every single day there are like hundreds of research papers coming out on archive, especially now with AI boom. Um yeah, also NeurIPS, for example, a major AI conference, they are seeing a massive amount of papers. So, can we automate this? Can we scale the community science team with agents? So, that's the second part of my talk. How can we, yeah, scale this uh to a massive amount of research papers?
然后我还会在 Hugging Face 上开 PR（pull request），去添加 dataset card 或 model card，改善这些产物的文档。但问题来了：让我一个人去开所有这些 GitHub issue 和 pull request 根本不可持续，因为每天 arXiv 上都有几百篇研究论文发出来，尤其是现在 AI 大爆发的时候。像 NeurIPS 这样的顶级 AI 会议，投稿量也是暴涨。那我们能不能把这件事自动化？能不能用 agent 来扩展 community science 团队的产能？这就是我演讲的第二部分：我们怎么把这件事规模化，去覆盖海量的研究论文。
[4:05] Niels
So, the idea is pretty simple. Uh we should have an AI agent which can help me do this outreach to all these researchers which publish uh models or data sets uh as part of their research work. And then, yeah, do the outreach in an automated way. So, this is the typical workflow that I was following. So, basically, whenever I saw a research paper, I first tried to find the GitHub URL uh of that paper, if it's available. Then, I I read the readme of that GitHub file. And then, I basically check if there's anything new interesting to be shared on Hugging Face. Uh it could be that it's on Hugging Face already. In that case, I check whether the model cards or data set cards already properly uh present, whether the metadata tags, for example, are there.
思路其实很简单：应该有一个 AI agent 来帮我做这种触达（outreach），去联系所有在研究工作中发布了模型或数据集的研究者，用自动化的方式完成这件事。这是我之前的典型工作流程：每当看到一篇研究论文，我先去找这篇论文的 GitHub 链接（如果有的话），然后读那个 GitHub 仓库的 readme，接着判断有没有什么新东西值得放到 Hugging Face 上分享。也可能它已经在 Hugging Face 上了，那我就检查 model card 或 dataset card 是否已经完善，比如元数据标签在不在。
[4:50] Niels
If uh not, then I will might open a pull request. Otherwise, if the artifacts are not yet on Hugging Face, I open a GitHub issue. And then, finally, I also follow up with the author. So, that's kind of the workflow that I had to automate uh with agents. And there are several ways to solve this. Uh you could uh go with a workflow. Uh these pictures are, by the way, taken from the blog post building effective agents by Anthropic, which is a really great read. Uh read. Um so, on the left side, you see, yeah, a workflow which is more deterministic. You basically use LLM APIs within steps of a predefined path or pipeline, uh which is more predictable. It's more deterministic. You have more control over it. Of course, it's less flex- flexible. And then, on the other hand, you could have a fully fledged auto- autonomous agent, which is an LLM in a loop that calls tools until it's done, which is more flexible, but also less uh predictable.
如果不完善，我可能就会开一个 pull request；如果产物还不在 Hugging Face 上，我就开一个 GitHub issue；最后我还要跟作者做后续跟进。这就是我要用 agent 自动化的工作流。解决方案有好几种路线。你可以走 workflow 路线——顺便说一下，这些图取自 Anthropic 的博客文章《Building Effective Agents》，非常值得一读。左边你看到的是 workflow，更偏确定性：你在一条预先定义好的路径或 pipeline 的各个步骤里调用 LLM API，更可预测、更确定，你的掌控力更强，当然灵活性差一些。另一边则是完全自主的 agent——一个在循环里不断调用工具直到任务完成的 LLM，更灵活，但可预测性也更低。
[5:44] Niels
Uh Uh, at the time, yeah, of course, it doesn't have to be a binary story. You can have a workflow on one hand, you can have a fully autonomous agent on the other hand, but you you could of course also mix and match these type of things uh, for your use case. In my case, I went for um, a pretty deterministic workflow. Uh, why? Because at the time that I was building this, this was in 2024, was at the time that Anthropic uh, wrote their blog post building effective agents. And there they actually said, "Try to avoid building agents if you really don't have to. Start simple, start with a single LLM API.
当然，这不一定是非此即彼的二选一。一边是 workflow，另一边是完全自主的 agent，但你完全可以根据自己的用例把这两类东西混搭起来。我自己选了一条相当确定性的 workflow 路线。为什么？因为我做这个东西是在 2024 年，正好是 Anthropic 发布《Building Effective Agents》那篇博客的时候。他们在文中明确说：「如果不是真的需要，尽量别做 agent。从简单开始，从单次 LLM API 调用开始。」
[6:19] Niels
Uh, avoid frameworks." Uh, and actually I think those were great tips. So, at the time I started building a workflow which basically replicated the workflow that I was doing when I was doing this outreach. So, yeah, this is the whole uh, pipeline. This is created using the Excalidraw MCP server in Cursor. It's pretty nice to create a visualization of your code. Uh, I'm not going to go into the details, but basically it just replicates um, the workflow that I was doing when doing the outreach. And I use LLM APIs in then each of the steps without any framework, without any agent framework. So, it made it quite uh, deterministic and I had a lot of control over uh, how this goes.
「避免用框架。」我觉得这些建议非常好。所以当时我开始搭一个 workflow，基本上就是把我自己人肉做 outreach 的流程复刻了一遍。这就是整条 pipeline，是用 Cursor 里的 Excalidraw MCP server 画的——拿它给代码做可视化挺不错的。细节我就不展开了，它基本上就是复刻了我做 outreach 时的那套流程。我在每个步骤里直接调用 LLM API，不用任何框架、任何 agent 框架。这样它就相当确定，我对整个流程的掌控力也很强。
[6:58] Niels
Um, in terms of deployment of this uh, workflow, it's a simple cron job. So, cron is just something that runs regularly. In my case, I run it once every night. So, when I'm sleeping, there is this agent, but technically it's just a cron job, a Python script with an LLM API, which is going to read all these hundreds of archive papers, uh, and then it might open GitHub issues or it might open pull requests on Hugging Face. I'm using GitHub Actions for this. Uh, I saw this very nice blog post free cron jobs with GitHub Actions, and actually it's probably the best entry point if you want to set up cron jobs, um, because GitHub has a pretty generous tier if you want to get started with like putting simple cron jobs uh, up there. And yeah, it makes it really easy for me in the UI to manage all these cron jobs.
部署方面，这个 workflow 就是一个简单的 cron job。cron 就是定时运行的任务，我的设定是每晚跑一次。所以我睡觉的时候，这个「agent」——技术上其实就是一个 cron job，一个带 LLM API 的 Python 脚本——会去读那几百篇 arXiv 论文，然后可能去开 GitHub issue，或者在 Hugging Face 上开 pull request。我用的是 GitHub Actions。我看到过一篇很不错的博客《用 GitHub Actions 白嫖 cron job》（free cron jobs with GitHub Actions），如果你想搭 cron job，这大概是最好的入门途径，因为 GitHub 的免费额度相当慷慨，很适合起步时挂一些简单的 cron job。而且在它的 UI 里管理这些 cron job 对我来说也非常方便。
[7:45] Niels
So yeah, every night I have hundreds of uh GitHub issues being created. For the tracing part, um I'm using LangFuse. Uh yeah, LangFuse also has a a booth here. Um LangFuse is pretty great. Um I use it mostly for the tracing part, the observability part, just to see what is the LLM doing, what are the inputs, what are the outputs, what are the prompts, how much does it cost, latency, and so on. Um so yeah, uh I definitely recommend it. Um but yeah, as my agents are opening so many GitHub issues every night, I then end up with a massive amount of unread GitHub notifications because people reply to those GitHub issues.
所以每天晚上都有几百个 GitHub issue 被创建出来。链路追踪（tracing）方面我用的是 LangFuse——对了，LangFuse 在这个会场也有展位。LangFuse 挺好用的，我主要用它做 tracing 和可观测性，就是看 LLM 在干什么：输入是什么、输出是什么、prompt 是什么、花了多少钱、延迟多少等等。所以我非常推荐它。不过，由于我的 agent 每晚开这么多 GitHub issue，我就攒下了海量的未读 GitHub 通知——因为大家会在那些 issue 下面回复。
[8:30] Niels
And that's a lot of work to then reply to all of those issues. It's kind of like going through your mailbox. So you could wonder, could we also um automate the follow-up to those GitHub uh issues? Because initially I was still the GitHub issue creation was done uh by an agent, but I was still the one involved in then doing the follow-up. Uh now a few months ago I also automated the the follow-up to those GitHub issues. Again, you could think, how should you solve this? Should you go for a more deterministic workflow or can you go for a fully autonomous agents, uh an LLM in a loop which runs with some tools and skills?
而挨个回复这些 issue 是一个巨大的工作量，有点像清理你的邮箱。所以你可能会想：能不能把对这些 GitHub issue 的后续跟进也自动化了？因为一开始，开 GitHub issue 这一步已经交给 agent 了，但做后续跟进的还是我本人。几个月前，我把对这些 GitHub issue 的跟进也自动化了。同样的问题又来了：这事该怎么解？是走更确定性的 workflow，还是上完全自主的 agent——也就是一个带着工具和 skill 在循环里跑的 LLM？
[9:08] Niels
Um well, here I went for kind of a fully autonomous agents, uh so it's kind of flexible. It's a bit less predictable, but it works quite well. Um I went for this because uh in November of last year at AI Engineer in New York, there was a pretty nice workshop by Anthropic on the Claude agent SDK. And there they were actually saying that agents might be better than workflows. So they So they were kind of contradicting themselves, but they he said that models have become so good that you might actually now start to work with fully autonomous agents rather than a workflow. So this is why I went with this approach and I actually am using the Claude agents SDK for this use case.
这一次我选择了完全自主的 agent 路线。它比较灵活，可预测性差一点，但效果相当不错。我这么选是因为去年 11 月在纽约的 AI Engineer 大会上，Anthropic 办了一场很不错的关于 Claude Agent SDK 的 workshop。他们在那场 workshop 里说，agent 可能比 workflow 更好——所以他们某种程度上是在自我打脸——但他们说模型已经变得如此之强，你现在或许真的可以直接上完全自主的 agent，而不是 workflow。所以我采用了这条路线，这个用例我实际用的就是 Claude Agent SDK。
[9:49] Niels
Uh there was another pretty nice talk by Cursor also at AI Engineer. This was in the European version in London a few months ago. There they talked about how they replaced 12,000 lines of custom code, pretty sophisticated workflow, with a very simple 200 lines of code skill. Uh actually it's pretty similar for me like I can replace a lot of custom codes, thousands of lines of code, with nowadays just a simple agent with maybe a CLI as a tool and a skill and that's it cuz the models have become so good. So yeah, in terms of the architecture, this is a bit what it looks like. Um so it's actually just the Claude agents SDK which is, I would say, a pretty good Python SDK for building an agent.
AI Engineer 上还有 Cursor 的一场很棒的演讲——那是几个月前在伦敦的欧洲场。他们讲了如何把 12,000 行自定义代码、一套相当复杂的 workflow，替换成一个非常简单的 200 行代码的 skill。我的情况其实很类似：如今我可以把大量自定义代码、成千上万行代码，替换成一个简单的 agent，配上一个 CLI 作为工具、再加一个 skill，就够了——因为模型已经变得足够强了。架构上大概就是这个样子。核心就是 Claude Agent SDK，我认为它是一个相当不错的用来构建 agent 的 Python SDK。
[10:37] Niels
Initially I was using the Claude models, but then I since actually this week I'm using the GLM 5.2 model via Hugging Face inference providers. So Hugging Face does offer a service which basically wraps a lot of inference providers like Together AI, Fireworks, Cerebras and so on. So you can use a lot of open models in a unified way. It's OpenAI compatible or Anthropic compatible and then I deploy this on Modal. Modal is also present here today. And it's mainly using Bash as a tool so the terminal to basically do Hugging Face commands because it's using the Hugging Face CLI quite a bit. So I combine it with the Hugging Face CLI skill which is actually all it needs. And then it might comment something on GitHub as a follow-up. And it also actually does the posting on Slack because eventually I also want to see the final results on our Slack channel uh from Hugging Face. So yeah, given that there's also a lot of hype on GLM 5.2 recently, for example, Cursor uh saw great performance on their Cursor bench.
最初我用的是 Claude 系列模型，但就从这周开始，我改用 GLM 5.2 了，通过 Hugging Face inference providers 来调用。Hugging Face 提供了一项服务，把 Together AI、Fireworks、Cerebras 等一大批推理服务商封装起来，让你能以统一的方式使用大量开源模型，接口兼容 OpenAI 或 Anthropic 格式。然后我把它部署在 Modal 上——Modal 今天也在现场。它主要用 Bash 作为工具，也就是用终端来执行 Hugging Face 命令，因为它大量使用 Hugging Face CLI。所以我给它配了 Hugging Face CLI 的 skill，其实它需要的就这一个。然后它可能会在 GitHub 上留言做跟进，还负责往 Slack 发消息——因为最终我也想在我们 Hugging Face 的 Slack 频道里看到结果。最近 GLM 5.2 的热度也很高，比如 Cursor 就在他们的 Cursor bench 上看到了很好的表现。
[11:44] Niels
Post-training bench is another one uh where it actually beats Opus 4.8 and it's cheaper. So, yeah, there's no reason not to use GLM 5.2 uh especially given that I work at Hugging Face. Um for the deployment, as I said before, I use Modal. It's pretty great if you want to deploy agents. Uh in my case, I'm using the batch processing feature. So, they allow you to spin up a massive amount of containers all in parallel. Every single container is basically one agent loop that is processing one GitHub issue. Uh it's super easy to use, I have to say. Um and the startups are also pretty fast. So, I definitely recommend it if you're building uh agents that are like, for example, running in the background, running overnight, for example.
Post-training bench 是另一个基准，在那上面它甚至打败了 Opus 4.8，而且还更便宜。所以没有理由不用 GLM 5.2，何况我还在 Hugging Face 工作。部署方面，前面说过我用 Modal，拿来部署 agent 非常好用。我用的是它的批处理（batch processing）功能：它可以让你一口气并行拉起海量容器，每个容器就是一个 agent 循环，处理一个 GitHub issue。不得不说它用起来超级简单，启动速度也很快。如果你在构建那种在后台运行、比如整夜跑的 agent，我非常推荐它。
[12:29] Niels
Um and then the way I invoke it, yeah, technically I could also just uh deploy this as a cron job. Modal, for example, has support for this. But typically, the follow-up on the GitHub issues, I still do that actually manually by invoking it as a skill. So, I created a skill for this in Cursor. Uh I call it process under it Modal. And then what it's going to do is it's actually going to invoke an agent, in this case, Composer 2.5, which is like the agent that I'm mostly using in Cursor, which is again going to invoke all the other agents. So, that's this is kind of the loop that people are talking about. And then finally, it's going to post uh all the results on our Slack channel.
至于调用方式——技术上我也可以把它直接部署成 cron job，Modal 也支持这么做。但 GitHub issue 的后续跟进，我目前其实还是手动触发的：把它作为一个 skill 来调用。我在 Cursor 里为此建了一个 skill，我把它叫 process_on_modal（process 下划线 Modal）。它做的事情是去调用一个 agent——这里是 Composer 2.5，也就是我在 Cursor 里最常用的那个 agent——然后由它再去调用其他所有 agent。这就是大家常说的那种「agent 套 agent」的循环。最后，它会把所有结果发到我们的 Slack 频道里。
[13:09] Niels
Uh so yeah, and this is actually what it just posts. So, what it does is it basically just posts a huge amount of Hugging Face papers, uh which are these research papers which people can uh make available on Hugging Face because every time someone mentions it in a model card or dataset card, we index it on the hub. And then it just posts all the artifacts that people have been uploading based on the outreach that we do via GitHub. Um so yeah, I do this still in a manual form. So I just invoke the skill and then after a few minutes, these messages appear on our Slack channel. Um yeah, I just included some fun results because to be honest, it's quite fun to see people interacting with the agents.
嗯，对，这就是它实际发出来的内容。它做的事情基本上就是发布大量 Hugging Face 上的论文——就是那些研究论文，大家可以把它们放到 Hugging Face 上，因为每当有人在 model card 或数据集卡片里提到某篇论文，我们就会在 hub 上给它建索引。然后它就会把大家上传的所有产出物都发出来，这些上传是基于我们通过 GitHub 做的外联触达。嗯，对，这一步我目前还是手动来做的：我调用一下那个 skill，几分钟之后，这些消息就会出现在我们的 Slack 频道里。嗯，我还放了一些有趣的结果进来，因为说实话，看大家跟这些 agent 互动挺好玩的。
[13:52] Niels
Um to be honest, I don't disclose that it's an agent. Why? Because I think if people know it's a bot, then they might quickly like close the issue. And to be honest, they post exactly the same stuff as I was doing before manually. So I don't actually see any reason to to do that. Um so and then you see replies like this. Hi Niels, thanks a lot for your suggestion and the clear guidance. I actually also often times see people using an agent to reply to my agents. So it's kind of the that internet nowadays. Um but people yeah, make all their artifacts available on Hugging Face. And out of the thousands of issues that are being created on Hugging Face, actually so far I've only had two negative comments. One guy saying yeah, please close this slop. So he closed the issue.
嗯，说实话，我不会公开说这是个 agent。为什么呢？因为我觉得如果大家知道这是个机器人，可能很快就会把 issue 关掉。而且说实话，它们发的内容跟我以前手动发的一模一样，所以我实在看不出有什么理由要披露。然后你会看到这样的回复：「你好 Niels，非常感谢你的建议和清晰的指引。」其实我也经常看到有人用 agent 来回复我的 agent，这就是如今的互联网了。不过大家确实都把他们的产出物放到 Hugging Face 上了。而且在 Hugging Face 上创建的几千个 issue 里，到目前为止我只收到过两条负面评论。有个人说：「请把这坨 slop 关掉吧」，然后他就把 issue 关了。
[14:38] Niels
And then another one. But most of the people they just say, yeah, actually it makes perfect sense to make my weights or my data sets available on Hugging Face. Like why didn't I think of this? Um so it's kind of a win-win I would say. Uh I often times also post fun results on our Slack channel. Like for example, one time someone a researcher from Apple sent me a DM like, I saw you reached out to me. Yeah, technically it's my agent just posting a GitHub issue regarding publishing a new Apple uh the artifacts of an Apple paper on Hugging Face. Or for example, it reaches out to Google DeepMind to publish um mathematics data sets.
然后还有另一条。但绝大多数人都会说：「对啊，把我的权重或者数据集放到 Hugging Face 上完全说得通，我怎么没想到呢？」所以我会说这是双赢。我也经常在我们的 Slack 频道里发一些有趣的结果。比如有一次，Apple 的一位研究员给我发私信说：「我看到你联系我了。」——严格来说那是我的 agent 在 GitHub 上开了个 issue，内容是关于把 Apple 一篇新论文的产出物发布到 Hugging Face 上。再比如，它还会去联系 Google DeepMind，让他们发布数学数据集。
[15:17] Niels
Um so a lot of times like I receive emails, the one on the the side, where yeah, they want to publish a 400 GB data set on Hugging Face, but this was also my agent just opening GitHub issues. Um Yeah, this is another fun result. So, Paddle OCR, it's like a Chinese company. They migrated all their OCR models to Hugging Face based on outreach by the agents that create issues for me. So, yeah, it's pretty nice. Another fun result is like when it when it completes the default template of model cards on Hugging Face. So, Mac Mitchell, who also works at Hugging Face, she has a famous paper called model cards for model reporting, making sure that anyone documents their models in a proper way. And so, we do provide this template, which you can see on the left side in the Git diff. And then, the agent is just completing that template based on the content that it finds based on the paper, like the GitHub readme,
嗯，所以很多时候我会收到邮件，就像旁边这封，对方说想在 Hugging Face 上发布一个 400 GB 的数据集——但这其实也是我的 agent 开 GitHub issue 触发的。嗯，这是另一个有趣的结果：PaddleOCR，一家中国公司的项目，他们把所有 OCR 模型都迁移到了 Hugging Face 上，起因就是替我创建 issue 的那些 agent 做的外联。所以真的挺不错的。还有一个好玩的结果，是它把 Hugging Face 上 model card 的默认模板给补全了。Margaret Mitchell——她也在 Hugging Face 工作——有一篇很有名的论文叫《Model Cards for Model Reporting》，目的是确保每个人都能规范地为自己的模型写文档。所以我们提供了这个模板，你可以在左边的 Git diff 里看到。然后 agent 就根据它找到的内容来补全这个模板，比如论文本身、GitHub 的 readme，
[16:17] Niels
the PDF itself, and so on. Um yeah, it's also quite funny to see, for example, in this case that it included me in the model card. It said, "Model card authors, Niels part of the Hugging Face community science team." I never prompted it this way, but it's pretty fun to see. Or people are replying, "Thank you for helping me fix my mistakes." So, those are all done by the agents. Uh I think the most popular GitHub issue that was created was this paper Tiny Recursive Models, which you might have seen, was quite trending both on Hugging Face, but also on Twitter. So, yeah, more than 60 people actually upvoted that issue so that the model was released on Hugging Face. So, this is again, I think, the win-win. So, it's both a win for the researcher, making their research more discoverable on Hugging Face, but it's also, yeah, better for the people then who want to build on top of that research and want to use them.
还有 PDF 原文等等。嗯，也挺好笑的，比如这个例子里它把我写进了 model card：「Model card 作者：Niels，Hugging Face 社区科学团队成员。」我从来没这样提示过它，但看到还是挺有意思的。还有人回复说：「谢谢你帮我修正了错误。」这些都是 agent 干的。我印象中最火的一个 GitHub issue，是关于 Tiny Recursive Models 这篇论文的——你们可能见过，它当时在 Hugging Face 和 Twitter 上都很热门。有超过 60 个人给那个 issue 点了赞成票，希望模型能发布到 Hugging Face 上。所以这又是我说的双赢：对研究者来说，他们的研究在 Hugging Face 上更容易被发现；同时对那些想在这些研究之上继续构建、想使用这些模型的人来说也更好。
[17:16] Niels
Uh so, yeah, I I hundreds of GitHub issues where I think I can show uh nice results um where people interact with the agents. You might also wonder, yeah, how to avoid slop because you might think, okay, you have an agent uh spamming the whole internet with your GitHub issues. Like, should you even do this? Again, I already talked about the win-win. Um, but a blog post that I highly recommend, if you want to avoid that your agent is just posting slop, is um the LLM Evils FAQ uh by Hamel Husain. Uh, I would say he's like the main expert when it comes to LLM evaluation. He also has like a a paid course, but he also publishes a lot of stuff for free online including this blog post. So, I highly recommend to go through it if you want to learn more about how to evaluate your agents.
嗯，所以，我有几百个 GitHub issue 都能展示出不错的结果，大家在里面跟 agent 互动。你可能也会想：那怎么避免产出 slop 呢？因为你可能会觉得，你这不就是放一个 agent 用 GitHub issue 刷屏整个互联网吗？这事到底该不该做？双赢的部分我前面已经讲过了。不过如果你想避免你的 agent 光发垃圾内容，我强烈推荐一篇博客：Hamel Husain 写的《LLM Evals FAQ》。我觉得他算是 LLM 评估领域的头号专家。他有付费课程，但也在网上免费发布了很多内容，包括这篇博客。所以如果你想深入了解怎么评估你的 agent，强烈建议通读一遍。
[18:06] Niels
So, my conclusion would be um that open models are actually getting great, especially now with GLM 5.2. You have Deep Seek V4 and so on. So, um yeah, we we are able to now replace closed-source models by open ones. Uh, for my use case, I would say agents are actually better than uh workflows. Uh, they only need a single CLI, which is the Hugging Face CLI. They need a single skill, the Hugging Face CLI skill, and a sandbox, and that's all they need to do their work. And finally, yeah, don't forget about evaluation. Um Finally, uh I can also discuss some other efforts that we do as part of the community science uh team.
所以我的结论是：开放模型现在真的越来越强了，尤其是有了 GLM 5.2，还有 DeepSeek V4 等等。所以我们现在已经能用开放模型替换掉闭源模型了。就我的使用场景而言，我认为 agent 确实比 workflow 更好。它们只需要一个 CLI，也就是 Hugging Face CLI；只需要一个 skill，就是 Hugging Face CLI 的 skill；再加一个 sandbox——有这些它们就能干活了。最后，别忘了评估这件事。嗯，最后我还可以聊聊我们社区科学团队在做的其他一些工作。
[18:44] Niels
Um, very shortly. Um, so, I have a Twitter account that I created. It's called Daily Papers. And it actually uses the exact same workflow as my agents behind the scenes to post interesting research papers on X. It uh recently crossed 90,000 followers without any involvement of me. I just deployed this uh and it posts interesting research papers and artifacts from Hugging Face every 4 hours or every time someone uh releases something cool on the Hugging Face. Um so, yeah. And I have like Gemini determining the best visual to tweet or to include in the tweet. Like for example, this recent tweet where it tweeted out that Nvidia released an optimized version of GLM 5.2 got more than 2,000 likes. So, that's pretty cool to see.
嗯，很快说一下。我创建了一个 Twitter 账号，叫 Daily Papers。它背后用的工作流跟我的 agent 完全一样，用来在 X 上发布有意思的研究论文。它最近粉丝数突破了 9 万，我完全没有参与——我只是把它部署上去，它就每 4 个小时发布一次 Hugging Face 上有意思的研究论文和产出物，或者每当有人在 Hugging Face 上发布了什么很酷的东西时也会发。嗯，对。而且我让 Gemini 来决定推文里配哪张图最合适。比如最近这条推文，说 Nvidia 发布了 GLM 5.2 的优化版本，拿到了 2000 多个赞。看到这些还是挺酷的。
[19:32] Niels
And a final effort that I'm working on right now is a revival of Papers With Code, which is a website that once existed and then was acquired by Meta and then sadly it died. So, I'm I'm trying to re- revive it and making a research and state-of-the-art easier accessible. Um for now it lives at paperswithcode.co. Uh so, yeah. You can find benchmarks over there. For example, for OCR models, all OCR benches like popular benchmark. But I'm also making it an educational resource so that people can learn about technical terms like mixed training uh on policy distillation and so on. So, yeah. That was it for my talk. I hope you learned something. Thanks all of you for your attention.
我现在正在做的最后一项工作，是复活 Papers With Code——这个网站曾经存在过，后来被 Meta 收购，然后很遗憾地死掉了。所以我在尝试把它复活，让研究成果和 state-of-the-art 更容易被获取。目前它放在 paperswithcode.co 上。嗯，对。你可以在那里找到各种 benchmark，比如 OCR 模型的，像 OCRBench 这种热门基准。同时我也在把它做成一个教育资源，让大家可以学习一些技术术语，比如混合训练、on-policy 蒸馏等等。好了，我的演讲就到这里。希望你们有所收获，谢谢大家的聆听。
[20:18]
[applause]
（掌声）