Hugging Face Vision Trainer
Train detection, classification, and segmentation models on Hugging Face Jobs GPUs and persist the results to the Hub.
The documentation requires passing HF_TOKEN through Job secrets, validating datasets before GPU use, and preserving results on the Hub, giving partial transparency around permissions and data flow. However, it defaults to paid cloud GPU use and Hub pushes without clearly requiring final submission confirmation, private/public repository choice, rollback, or sensitive-data handling; examples also read and pass tokens directly, so points are deducted.
The workflow includes dataset checks, timeout guidance, configuration rules, and troubleshooting, and it explicitly recommends preflight verification. However, the actual training scripts are not present in the supplied evidence, dependency versions are not consistently pinned, the SAM examples risk inconsistent keys such as original_sizes versus original_image_size, and no statically verifiable key-path tests are included; the score therefore remains below the static ceiling.
The skill covers object detection, classification, and SAM/SAM2, with model lists, dataset schemas, hardware guidance, and semantic trigger terms. Non-fit ranges, abnormal-input behavior, Chinese-language support, and mainland-China network reachability are not described, while core operation depends on Hugging Face cloud services, so points are deducted.
The structure, related-skill guidance, prerequisites, quick starts, references, and troubleshooting are useful, and the supplied license metadata is Apache-2.0. Nevertheless, there is no skill version, changelog, explicit maintenance owner or update path, and compatibility across scripts and referenced library versions is insufficiently specified, so points are deducted.
The material theoretically covers dataset inspection, training, evaluation, monitoring, Hub persistence, and inference, with parameter examples that are close to directly usable. But the scripts themselves are absent from the evidence, there are no verifiable execution results, and configuration/model/hardware assumptions still require manual review; under static calibration the score is capped at 7 and reduced further for these gaps.
The documentation supplies substantial code and configuration details that can be inspected, but it contains no committed test suite, CI coverage, real execution logs, or independent reproduction evidence. Key claims therefore remain primarily documentation assertions, warranting only a low score.
- Submitting a training Job may incur real GPU charges and write models to the Hub; obtain explicit confirmation of budget, repository visibility, and final submission before execution.
- Do not place real tokens in scripts, logs, or ordinary environment variables; verify secret isolation, token scope, and revocation/rotation procedures.
- The supplied evidence does not include the scripts directory or tests, and the SAM reference contains a possible field-name inconsistency; validate dependencies, arguments, and batching in an isolated environment first.
- The core workflow depends on Hugging Face Hub, Jobs, and remote model/data downloads; assess mainland-China reachability and prepare an accessible mirror or local fallback.
What does this skill do, and when should you use it?
This skill guides vision-model training with Hugging Face Transformers for object detection, image classification, and SAM/SAM2 segmentation. It combines dataset validation, production-oriented training templates, hardware and cost guidance, evaluation metrics, Trackio monitoring, and Hub persistence. Jobs can be submitted to managed Hugging Face cloud GPUs, while the helper scripts can also be run locally. Adoption requires a paid Hugging Face plan, a write-capable token, and a compatible Hub dataset.
Inspects Hub datasets for image, label, object, mask, and prompt fields; runs dataset_inspector.py to validate formats; prepares object-detection, image-classification, or SAM/SAM2 training jobs; submits jobs through the hf_jobs MCP tool or HfApi().run_uv_job(); configures hardware, timeouts, secrets, and cost estimates; evaluates mAP, mAR, accuracy, and related metrics; pushes trained models to the Hugging Face Hub; and reports job IDs, monitoring links, and Trackio dashboard URLs.
- A developer needs to fine-tune D-FINE, RT-DETR v2, DETR, or YOLOS on a custom object-detection dataset without configuring a local GPU.
- An ML practitioner needs to train a MobileNetV3, MobileViT, ResNet, ViT/DINOv3, or other Transformers image classifier from a dataset with image and label fields.
- A computer-vision team needs to fine-tune SAM or SAM2 for segmentation or image matting using binary masks and bounding-box or point prompts.
- A practitioner wants to validate dataset structure, choose cloud hardware, estimate training cost, and preserve the resulting model on the Hub before launching a GPU job.
What are this skill's strengths and limitations?
- Covers three substantial vision-training areas: detection, classification, and SAM/SAM2 segmentation.
- Includes a dataset inspector and ready-to-use training templates addressing common box formats, label mappings, and required fields.
- Provides concrete guidance for hardware, timeouts, cost estimation, authentication, Hub persistence, monitoring, and failure recovery.
- Supports managed cloud GPU training and explicitly protects results by pushing them to the Hub.
- Requires a Hugging Face Pro, Team, or Enterprise plan and a token with write permissions.
- Cloud GPU usage is billed hourly, with cost and duration dependent on the dataset, model, hardware, and configuration.
- Depends on Hugging Face Jobs, Hub datasets, and the referenced Python training environment; it is not documented as a fully offline workflow.
- The supplied material does not provide a standalone test suite or evidence of broad platform testing.
How do you install this skill?
Get the repository from https://github.com/huggingface/skills and copy or symlink skills/huggingface-vision-trainer into a standard Codex .agents/skills directory such as $REPO_ROOT/.agents/skills. The README also documents the Hugging Face CLI method: hf skills add huggingface-vision-trainer.
How do you use this skill?
After installation, prompt your coding agent with a request such as: “Use the Hugging Face vision trainer to fine-tune a D-FINE detector on my Hub dataset with Hugging Face Jobs and push the result to the Hub.” The documented workflow verifies the account and token, runs uv run scripts/dataset_inspector.py --dataset username/dataset-name --split train, chooses dataset size and hardware, prepares the relevant training script, submits the job, and waits for the user to request status checks.
How does this skill compare with similar options?
This skill focuses on vision-model training. The README identifies hugging-face-jobs as the companion skill for general Jobs infrastructure such as tokens, hardware, secrets, timeouts, and persistence. It identifies hugging-face-model-trainer as the skill for TRL-based language-model training, rather than an alternative for these vision workflows.