What does this skill do, and when should you use it?
This skill is for developers who want to run Hugging Face models on local hardware. It searches for llama.cpp-compatible GGUF repositories, helps select a quantization, and constructs commands for llama-cli or llama-server. It also verifies exact GGUF filenames, converts Transformers weights when no GGUF exists, and starts an OpenAI-compatible local server. It covers CPU, Mac Metal, CUDA, and ROCm, but the supplied material includes no performance benchmarks.
Searches the Hugging Face Hub for models marked for llama.cpp; opens local-app model pages and uses visible quantization recommendations; confirms exact .gguf filenames through the model tree API; launches models from the Hub with llama-cli or llama-server; uses --hf-repo and --hf-file for custom filenames; downloads, converts, and quantizes Transformers weights only when GGUF files are unavailable; and tests a local server through its OpenAI-compatible chat-completions endpoint with curl.
- A developer who needs to run a Hugging Face model locally on CPU, Mac Metal, CUDA, or ROCm hardware.
- A user choosing among Q4, Q5, Q6, Q3, or repository-specific quantizations under a memory or VRAM budget.
- A developer who needs to identify an exact GGUF file and expose it through llama-server.
- A user with Transformers weights whose repository does not yet provide GGUF files and who needs a conversion path.
How do you install this skill?
- Before running llama-server, confirm it is bound only to a trusted local or network interface; the example's no-op Bearer token is not real access control.
- Verify repository identity, file hashes, licenses, and the security of conversion dependencies before downloading or executing them; the skill provides no such procedure.
- The core Hugging Face URL workflow may be inaccessible from mainland-China networks, so reachability checks or offline/mirror alternatives are needed.
- The model examples, command flags, and quantization-performance figures lack reproducible evidence in the supplied files and should be checked against the actual tool version and hardware.
- Shell / CLI
- Network access
- Local filesystem
llama.cppHugging Face CLI (hf)Python
Copy or symlink the skills/huggingface-local-models folder into a standard .agents/skills location for Codex or another compatible client, such as $REPO_ROOT/.agents/skills or $HOME/.agents/skills. Install llama.cpp with brew install llama.cpp, winget install llama.cpp, or clone https://github.com/ggml-org/llama.cpp and run make. For gated repositories, run hf auth login.
tmp="$(mktemp -d)"
git clone --depth 1 https://github.com/huggingface/skills.git "$tmp"
mkdir -p ~/.claude/skills
cp -R "$tmp/skills/huggingface-local-models" ~/.claude/skills/
rm -rf "$tmp"Generated from the source repository and skill path; it copies only this skill's folder. If the author's install steps above differ, follow those first. To scope it to one project, replace ~/.claude/skills with that project's .claude/skills.
How do you use this skill?
Once installed, send your agent any of these to trigger it:
- Use the Hugging Face Local Models skill to choose a GGUF model for my available memory and launch llama-server.
In an Agent Skills-compatible coding agent, explicitly request the Hugging Face Local Models skill, for example: “Use the Hugging Face Local Models skill to choose a GGUF model for my available memory and launch llama-server.” A typical command is llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M. For an exact file, use --hf-repo and --hf-file.
What are this skill's strengths and limitations?
- Covers Hub discovery, quantization choice, exact file lookup, and local serving in one workflow.
- Preserves repository-native quantization labels and gives guidance for constrained memory and code workloads.
- Supports llama-cli, llama-server, GGUF conversion, and an OpenAI-compatible local endpoint.
- Explicitly avoids conversion when a repository already exposes GGUF files.
- Requires llama.cpp, the Hugging Face CLI, Python, local filesystem access, and network access.
- The supplied material documents no automated test suite, performance benchmarks, or hardware validation results.
- Quantization still depends on the model page and the user's hardware; the skill does not guarantee optimal speed or quality.
- Gated repositories require separate Hugging Face authentication.
How does this skill compare with similar options?
Side by side with related skills; every score comes from the same FSRS standard.
| Skill | FS score | Stars | Last updated | License |
|---|---|---|---|---|
| Hugging Face Local Model Runner this page | 44 · Not recommended | ★ 11k | 18d ago | Apache-2.0 |
| 9Router Gateway Skill | 46 · Use with care | ★ 31k | 19d ago | MIT |
| HF Paper Publisher | 45 · Not recommended | ★ 38k | 14d ago | MIT |
| Hugging Face MCP Assistant | 36 · Not recommended | ★ 11k | 18d ago | Apache-2.0 |
| 9Router Image Generation Skill | 52 · Use with care | ★ 31k | 19d ago | MIT |
How did FollowSkills review this skill?
The material mainly guides local model download, compilation, and execution; no malware, credential theft, or covert exfiltration is evident. However, it lacks model provenance checks, hashes/signatures, sensitive-input handling, confirmation before installation/authentication, service exposure boundaries, and rollback guidance, so it is well below full marks.
The workflow from search and file confirmation to server launch is coherent and includes custom filenames and conversion fallback paths. However, llama.cpp and tool versions are not pinned, no committed tests cover key paths, platform build commands vary, and network, compatibility, abnormal-input, and diagnostic failure handling are thin; the static cap therefore requires a conservative deduction.
The audience, CPU/Metal/CUDA/ROCm scenarios, and main inputs and outputs are reasonably clear, with stated quantization alternatives. Trigger conditions, non-fit model/hardware boundaries, and prerequisites remain incomplete. Core discovery and downloads depend on Hugging Face services, with no mainland-China reachability assessment or fallback.
The SKILL.md and three topic-focused reference documents provide progressive disclosure, examples, troubleshooting, and license context under Apache-2.0. Missing version requirements, changelog, maintenance owner, update path, dependency pinning, freshness checks, and fuller installation/FAQ guidance reduce maintainability.
For users with llama.cpp installed and access to Hugging Face, the skill directly helps search GGUF repositories, choose quants, confirm filenames, and construct llama-cli/llama-server commands. There is no verified representative output or execution evidence, and hardware sizing, compatibility, service security, and conversion results still require user review; static calibration limits this score to at most 7.
The documents provide concrete Hugging Face URLs, a tree API, commands, and quantization tables, while repository context shows generated-artifact CI and a security policy. The selected skill has no committed tests, execution logs, or third-party reproduction evidence, and the numerical quality claims and examples cannot be independently confirmed from the supplied files alone, warranting only limited credit.
Open a dimension to read why it scored that way
Evidence confidence：Low — Mostly static review, author material or a limited demo; useful for discovery, not high-risk decisions.
See the full review method →