AI & ML interests

The AI community building the future.

Recent Activity

system  updated a dataset about 6 hours ago
huggingface/diffusers-metadata
tarekziade  updated a bucket about 9 hours ago
huggingface/transformers-ci-telemetry
hlarcher  updated a bucket about 9 hours ago
huggingface/cdn-live
View all activity

Articles

hlarcher 
updated a bucket about 9 hours ago
burtenshaw 
published a bucket about 20 hours ago
sergiopaniego 
posted an update 2 days ago
sergiopaniego 
posted an update 8 days ago
view post
Post
223
AsyncGRPOTrainer is now stable in TRL v1.18.0!

generation and training run at the same time on separate GPUs, instead of taking turns

we wrote up how to run it, what you can train with it and how it compares with GRPOTrainer: trl-lib/async-grpo-stable
sergiopaniego 
posted an update 11 days ago
view post
Post
6,868
New post-training example: SFT on agent traces with TRL

→ A large model works as a coding agent in OpenCode, run through OpenEnv
→ The tasks it solves become the training data
→ A 2B model goes from 5.5% to 34.4% on held-out tasks

About 500 traces get most of the gain. Five times more add only 3 to 4 points, so SFT gets the agent working and RL has to do the rest

Full guide, with the models, traces and every rollout open:
https://github.com/huggingface/trl/tree/main/examples/sft_agent_traces
  • 1 reply
·
sergiopaniego 
posted an update 18 days ago
view post
Post
4,753
TRL now supports MOPD (Multi-Teacher On-Policy Distillation) natively❗️

Many frontier open models use it. labs train one RL expert per domain and then distill them all into one model

Blog with what it is, who uses it and an experiment you can run on HF Jobs with one command

https://huggingface.co/blog/sergiopaniego/mopd-trl
sergiopaniego 
posted an update 22 days ago
view post
Post
1,246
if you want to deep dive into how the new @MistralAI Large 4 was built, we sat down with the team to learn straight from them!

scaling, specialization, evals, cyber, sandboxing and why open weights matter

a really detailed look at what it takes to build and release an open frontier model, super recommended

https://www.youtube.com/watch?v=FczbSauVvjw
albertvillanova 
posted an update 23 days ago
view post
Post
388
Day-0 Mistral Large 4 support in TRL 🚀

- LoRA SFT of the 1T-A52B model on one 8xB300 node, on top of the FP8 checkpoint
- Experts split 8 ways with expert parallelism, everything else sharded with FSDP, all set at load time
- Assistant-only loss out of the box with its training chat template
- ~32.7k tokens per step in ~8 s

Script + Slurm launcher 👇
https://github.com/huggingface/trl/tree/main/examples/sft_mistral_large_4
https://huggingface.co/posts/sergiopaniego/408182860646834
sergiopaniego 
posted an update 23 days ago
sergiopaniego 
posted an update 24 days ago
sergiopaniego 
posted an update 25 days ago
view post
Post
208
Last week we closed the Training Agents series IRL at the GitHub offices in SF

4 talks, a panel and audience Q&A. almost 2h on post-training agents with open models

> Learnings from enterprise RL (Govind Kamtamneni Microsoft)
> Environments and benchmarks in industry (Victor Barres Mercor)
> How to train your agents with Miles (Tao Lin RadixArk)
> Why we built OpenEnv Arena (Xiangyi Li BenchFlow)

Full recording + Slides

https://www.youtube.com/watch?v=CKFUx7COIwQ
sbrandeis 
posted an update about 1 month ago
sergiopaniego 
posted an update about 2 months ago
sergiopaniego 
posted an update about 2 months ago
view post
Post
360
Repo2RLEnv just shipped TaskSmith + 50 high quality RL envs generated from HF repos 🔨

TaskSmith is a specialized harness that turns a merged PR into a verified RL env

the envs come from HF repos (Transformers, TRL, PEFT, Accelerate, Diffusers), shipped as Harbor tasks you can eval or train on

> code: github.com/huggingface/Repo2RLEnv
> dataset: huggingface.co/datasets/FineEnvs/HF_ML_Tasksmith
sergiopaniego 
posted an update about 2 months ago
view post
Post
3,876
ThinkingBox from @microsoft is now available as an OpenEnv env (cc @tuhink 🤗 )!

> ThinkingBox is a sandbox for testing agents on business workflows. it simulates a customer, gives the agent MCP tools over a real database, and at the end checks what changed in that database instead of trusting the agent's last message

> ThinkingBox-Bench is the benchmark built on it: 507 tasks across retail, insurance, travel, banking and consulting

> the OpenEnv env runs each task as an episode in its own isolated backend and returns a pass/fail reward from those checks

https://huggingface.co/blog/microsoft/thinkingbox