What does this skill do, and when should you use it?
This skill helps agents use Hugging Face TRL to train and fine-tune transformer language models. It covers SFT, DPO, GRPO, RLOO, and reward-model training with executable CLI examples. It also documents LoRA adapters, YAML configuration, Accelerate-based distributed training, and troubleshooting guidance. It fits users who already have models and datasets and want a structured post-training workflow within the Hugging Face ecosystem.
Runs trl sft, trl dpo, trl grpo, trl kto, trl rloo, and trl reward; reads models and datasets from the Hugging Face Hub or local paths; configures learning rates, batch sizes, evaluation, LoRA, and output directories; launches reproducible runs from YAML files; uses Accelerate for multi-GPU, multi-node, FSDP, or DeepSpeed ZeRO training; pushes results to the Hub; and provides troubleshooting guidance for CUDA out-of-memory errors, dataset loading, model loading, slow training, and generation issues.
- A researcher turning a base model into an instruction-following or conversational model can use the SFT workflow and examples.
- An alignment engineer with chosen/rejected preference pairs can use the DPO command for direct preference optimization.
- A team evaluating generations with reward functions or an LLM judge can use GRPO.
- A researcher seeking online generation-based reinforcement learning with custom rewards can use RLOO.
- A team building an RLHF pipeline can use the reward command to train a reward model.
- A resource-constrained user or experimenter needing reproducibility can combine LoRA, YAML configuration, and Accelerate.
How do you install this skill?
- Before using --push_to_hub, confirm upload scope, credential permissions, repository visibility, and sensitive-data risks; rollback or revocation guidance is absent.
- Verify installed TRL, Transformers, Accelerate, PEFT, hardware, and CLI versions before execution because examples may become invalid across releases.
- Remote Hub access, authentication, large-model downloads, and external tracking may be constrained by network, account permissions, or resources; no mainland-China alternative is documented.
- Validate dataset schemas, reward functions, memory use, and outputs in an isolated small run before long or distributed training.
- Shell / CLI
- Network access
- Local filesystem
TRLHugging Face TransformersAccelerate
The repository contains 26 separate skills. Following the README’s Codex method, copy or symlink skills/trl-training/ into a standard Codex .agents/skills location, such as $REPO_ROOT/.agents/skills or $HOME/.agents/skills. A compatible Agent Skills client can then discover its SKILL.md. The supplied source does not provide installation commands for TRL itself.
tmp="$(mktemp -d)"
git clone --depth 1 https://github.com/huggingface/skills.git "$tmp"
mkdir -p ~/.claude/skills
cp -R "$tmp/skills/trl-training" ~/.claude/skills/
rm -rf "$tmp"Generated from the source repository and skill path; it copies only this skill's folder. If the author's install steps above differ, follow those first. To scope it to one project, replace ~/.claude/skills with that project's .claude/skills.
How do you use this skill?
Once installed, send your agent any of these to trigger it:
- Use the trl-training skill to fine-tune Qwen/Qwen2-0.5B on trl-lib/Capybara with SFT and LoRA.
After installation, give the coding agent a specific request such as: "Use the trl-training skill to fine-tune Qwen/Qwen2-0.5B on trl-lib/Capybara with SFT and LoRA." You can also run the documented commands directly, for example trl sft --config sft_config.yaml --learning_rate 1.0e-5. Before training, confirm the model, dataset schema, hardware, and appropriate training method.
What are this skill's strengths and limitations?
- Covers SFT, DPO, GRPO, RLOO, and reward-model training.
- Includes full-training, LoRA, configuration-file, and distributed-training examples.
- Provides troubleshooting guidance for memory, dataset, model-loading, speed, and generation issues.
- Builds on TRL, Transformers, and Accelerate for Hugging Face ecosystem integration.
- Does not document how to install TRL, Transformers, or Accelerate.
- Examples depend on particular models, datasets, hardware, and schemas, so direct reuse is not guaranteed for every project.
- The SKILL.md provides no test suite, benchmark results, or platform validation evidence.
- `kto` appears in the command metadata, but the body provides no corresponding usage example or explanation.
How does this skill compare with similar options?
Side by side with related skills; every score comes from the same FSRS standard.
| Skill | FS score | Stars | Last updated | License |
|---|---|---|---|---|
| TRL Language Model Trainer this page | 48 · Use with care | ★ 11k | 18d ago | Apache-2.0 |
| Cosmos Reason Video QA Fine-Tuning ✓ NVIDIA · Official | 53 · Use with care | ★ 3.6k | 12d ago | Apache-2.0 |
| Nemotron Customization Pipelines ✓ NVIDIA · Official | 57 · Use with care | ★ 3.6k | 12d ago | Apache-2.0 |
| Book Style SFT Pipeline | 36 · Not recommended | ★ 18k | 15d ago | MIT |
| Azure AI Foundry Fine-Tuning ✓ Microsoft · Official | 48 · Use with care | ★ 2.7k | 3mo ago | MIT |
How did FollowSkills review this skill?
The evidence shows CLI guidance without malware or credential-theft behavior; however, examples include --push_to_hub, remote model/dataset loading, and hf auth login. Least privilege, data flows, confirmation, sensitive-data handling, dependency security, and rollback are not documented, so points are deducted.
The document covers SFT, DPO, GRPO, RLOO, reward-model training, configuration, and common troubleshooting, making the happy path plausible. It provides no committed tests for key commands, compatibility matrix, input validation, or diagnostically controlled failure behavior, so the static score remains limited.
The audience and training scenarios are reasonably clear, with several post-training methods and single-node/distributed examples. Boundaries for dataset schemas, hardware, versions, and method selection are incomplete; Chinese-language support and mainland-China network reachability are not addressed, and core use depends on external Hugging Face services.
The SKILL.md has clear frontmatter, progressive sections, examples, configuration, distributed-training guidance, troubleshooting, and best practices. Apache-2.0, version 1.0.0, and documentation links are present, but changelog, maintenance owner, update path, installation/dependency notes, FAQs, and known limitations are missing.
The command examples are directly useful as starting points, especially for LoRA, YAML configuration, and troubleshooting. However, the files contain no execution evidence, quality or evaluation completeness is not established, resource cost is only lightly discussed, and users must fill in important task-specific details, so the score is deducted.
The revision includes auditable source documentation, an Apache license, and CI validating generated artifacts, providing limited evidence. The CI does not exercise TRL training paths, and there are no committed tests, real outputs, or independent corroboration; therefore static verifiability is modest.
Open a dimension to read why it scored that way
Evidence confidence：Low — Mostly static review, author material or a limited demo; useful for discovery, not high-risk decisions.
See the full review method →