Hugging Face Local Model Runner
Choose compatible GGUF quantizations from Hugging Face and run them locally with llama.cpp.
The material mainly guides local model download, compilation, and execution; no malware, credential theft, or covert exfiltration is evident. However, it lacks model provenance checks, hashes/signatures, sensitive-input handling, confirmation before installation/authentication, service exposure boundaries, and rollback guidance, so it is well below full marks.
The workflow from search and file confirmation to server launch is coherent and includes custom filenames and conversion fallback paths. However, llama.cpp and tool versions are not pinned, no committed tests cover key paths, platform build commands vary, and network, compatibility, abnormal-input, and diagnostic failure handling are thin; the static cap therefore requires a conservative deduction.
The audience, CPU/Metal/CUDA/ROCm scenarios, and main inputs and outputs are reasonably clear, with stated quantization alternatives. Trigger conditions, non-fit model/hardware boundaries, and prerequisites remain incomplete. Core discovery and downloads depend on Hugging Face services, with no mainland-China reachability assessment or fallback.
The SKILL.md and three topic-focused reference documents provide progressive disclosure, examples, troubleshooting, and license context under Apache-2.0. Missing version requirements, changelog, maintenance owner, update path, dependency pinning, freshness checks, and fuller installation/FAQ guidance reduce maintainability.
For users with llama.cpp installed and access to Hugging Face, the skill directly helps search GGUF repositories, choose quants, confirm filenames, and construct llama-cli/llama-server commands. There is no verified representative output or execution evidence, and hardware sizing, compatibility, service security, and conversion results still require user review; static calibration limits this score to at most 7.
The documents provide concrete Hugging Face URLs, a tree API, commands, and quantization tables, while repository context shows generated-artifact CI and a security policy. The selected skill has no committed tests, execution logs, or third-party reproduction evidence, and the numerical quality claims and examples cannot be independently confirmed from the supplied files alone, warranting only limited credit.
- Before running llama-server, confirm it is bound only to a trusted local or network interface; the example's no-op Bearer token is not real access control.
- Verify repository identity, file hashes, licenses, and the security of conversion dependencies before downloading or executing them; the skill provides no such procedure.
- The core Hugging Face URL workflow may be inaccessible from mainland-China networks, so reachability checks or offline/mirror alternatives are needed.
- The model examples, command flags, and quantization-performance figures lack reproducible evidence in the supplied files and should be checked against the actual tool version and hardware.
What does this skill do, and when should you use it?
This skill is for developers who want to run Hugging Face models on local hardware. It searches for llama.cpp-compatible GGUF repositories, helps select a quantization, and constructs commands for llama-cli or llama-server. It also verifies exact GGUF filenames, converts Transformers weights when no GGUF exists, and starts an OpenAI-compatible local server. It covers CPU, Mac Metal, CUDA, and ROCm, but the supplied material includes no performance benchmarks.
Searches the Hugging Face Hub for models marked for llama.cpp; opens local-app model pages and uses visible quantization recommendations; confirms exact .gguf filenames through the model tree API; launches models from the Hub with llama-cli or llama-server; uses --hf-repo and --hf-file for custom filenames; downloads, converts, and quantizes Transformers weights only when GGUF files are unavailable; and tests a local server through its OpenAI-compatible chat-completions endpoint with curl.
- A developer who needs to run a Hugging Face model locally on CPU, Mac Metal, CUDA, or ROCm hardware.
- A user choosing among Q4, Q5, Q6, Q3, or repository-specific quantizations under a memory or VRAM budget.
- A developer who needs to identify an exact GGUF file and expose it through llama-server.
- A user with Transformers weights whose repository does not yet provide GGUF files and who needs a conversion path.
What are this skill's strengths and limitations?
- Covers Hub discovery, quantization choice, exact file lookup, and local serving in one workflow.
- Preserves repository-native quantization labels and gives guidance for constrained memory and code workloads.
- Supports llama-cli, llama-server, GGUF conversion, and an OpenAI-compatible local endpoint.
- Explicitly avoids conversion when a repository already exposes GGUF files.
- Requires llama.cpp, the Hugging Face CLI, Python, local filesystem access, and network access.
- The supplied material documents no automated test suite, performance benchmarks, or hardware validation results.
- Quantization still depends on the model page and the user's hardware; the skill does not guarantee optimal speed or quality.
- Gated repositories require separate Hugging Face authentication.
How do you install this skill?
Copy or symlink the skills/huggingface-local-models folder into a standard .agents/skills location for Codex or another compatible client, such as $REPO_ROOT/.agents/skills or $HOME/.agents/skills. Install llama.cpp with brew install llama.cpp, winget install llama.cpp, or clone https://github.com/ggml-org/llama.cpp and run make. For gated repositories, run hf auth login.
How do you use this skill?
In an Agent Skills-compatible coding agent, explicitly request the Hugging Face Local Models skill, for example: “Use the Hugging Face Local Models skill to choose a GGUF model for my available memory and launch llama-server.” A typical command is llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M. For an exact file, use --hf-repo and --hf-file.