Vault Hybrid Retrieval
Paragraph-level Obsidian vault search combining contextual prefixes, BM25 retrieval, and optional vector reranking.
The skill explicitly discloses that the Anthropic API and claude CLI tiers send wiki page bodies off-machine, and gates them behind --allow-egress plus setup-time consent; synthetic prefixes and local Ollama are available by default. The declared tools are limited to Read and Bash, and no broad credential-reading scope is shown. Deductions apply because retention, sensitive-data handling, dependency supply-chain risk, isolation, and recovery are under-specified; the documented rm -rf cleanup command lacks confirmation and rollback guidance.
The documentation specifies setup, feature detection, exit-code-10 fallback, and empty-result behavior, while repository context supplies BM25 unit tests and CI. Deductions apply because the supplied static evidence does not cover the key end-to-end paths of retrieve, contextual-prefix, rerank, or setup; concrete failure behavior for missing Ollama/API/CLI and stale-index handling is limited. Static calibration caps this at 10.
Trigger terms, query input, candidate output, provisioning prerequisites, and the legacy fallback are clearly described; --no-llm provides a local path, and the tests claim Unicode/CJK preservation. Deductions apply because non-fit boundaries, Chinese retrieval quality, mainland-China reachability of Ollama/Anthropic services, and fallback coverage lack evidence.
The skill is well layered, with architecture, feature gating, setup, flags, command examples, cost notes, maintenance instructions, roadmap, and cross-references; repository context supplies an MIT license, SECURITY.md, version information, and CI. Deductions apply because the selected skill lacks an independent changelog, explicit maintenance owner/update path, and compatibility matrix, while several dependency and configuration assumptions must be inferred from other files.
The intended BM25, contextual-prefix, and optional reranking workflow is clear, and the candidate output is directly usable by callers that read and synthesize source pages. Deductions apply because the static material provides no representative verified output or independently checkable quality result for the key path; the README's 50-query, +32-point and +41-percent claims lack corresponding benchmark implementation or results in the supplied evidence.
The revision-pinned source, CI, BM25 tests, and explicit detection/explain modes provide limited auditability. Deductions apply because tests do not cover the complete wiki-retrieve pipeline, the Anthropic research and performance claims cannot be independently reproduced from the supplied files, and there are no multiple corroborating evidence types or real execution artifacts.
- Before using --allow-egress or any non-synthetic tier, confirm that wiki pages contain no sensitive information and review third-party data handling and retention policies.
- The index is not auto-refreshed; pages must be manually reprocessed and the BM25 index rebuilt after substantive changes, or results may be stale.
- The core retrieve/setup/contextual-prefix/rerank paths are not fully covered by the supplied static tests; README performance figures should not be treated as verified.
- Mainland-China users should verify reachability of Anthropic API, the claude CLI, and local Ollama model availability; otherwise use --no-llm or BM25-only.
What does this skill do, and when should you use it?
wiki-retrieve is an optional chunk-level retrieval skill in claude-obsidian's Compound Vault. It splits Markdown pages into chunks, adds contextual prefixes, builds a BM25 index, and can rerank candidates with local ollama embeddings. The command returns ranked candidates with absolute source-page paths for wiki-query or autoresearch to read and cite. It does not replace the base query path unless the feature is fully provisioned; otherwise callers fall back to the legacy hot→index→drill flow.
The setup script checks the required scripts, creates .vault-meta/chunks/ and .vault-meta/bm25/, probes local ollama, and selects an Anthropic API, claude CLI, or synthetic prefix tier. contextual-prefix.py chunks pages on paragraph boundaries, generates prefixes, and writes chunk JSON files; bm25-index.py builds an inverted index; retrieve.py obtains BM25 candidates, optionally reranks them with cosine similarity through rerank.py and ollama, deduplicates by page address, and emits candidates containing absolute_path. Index maintenance is manual after substantive page changes.
- Obsidian users with a growing Markdown vault who need the specific passage containing an answer.
- Compound Vault users who want chunk-level retrieval for long pages or passage-specific questions.
- Knowledge-base builders who want fast BM25 filtering followed by optional local semantic reranking.
- Privacy-conscious users who want synthetic prefixes and local reranking without sending page bodies off-machine.
- Developers debugging retrieval behavior through per-stage diagnostics and explain output.
What are this skill's strengths and limitations?
- Retrieves passages at chunk level instead of relying only on whole-page matching.
- Combines contextual prefixes, BM25 candidate selection, and optional local cosine reranking.
- Keeps egress disabled by default; non-synthetic prefix generation requires explicit consent.
- Supports BM25-only mode, explain diagnostics, local caches, and graceful fallback.
- It is an optional retrieval layer, not a complete knowledge-management application by itself.
- Indexes are not refreshed automatically after vault changes.
- High-quality vector reranking depends on local ollama; API and claude CLI prefix tiers send page bodies off-machine.
- The source does not document a standalone installer or dedicated test suite for this skill; compatibility beyond Claude Code is inferred from its generic Agent Skills structure.
How do you install this skill?
The skill has no separately documented package; it ships with the AgriciDaniel/claude-obsidian collection. After installing the collection, run from the repository root:
bash bin/setup-retrieve.sh
To force synthetic, local prefix generation:
bash bin/setup-retrieve.sh --no-llm
For diagnostics without provisioning, run bash bin/setup-retrieve.sh --check. The script reports ollama status but does not install it.
How do you use this skill?
Standard retrieval:
python3 scripts/retrieve.py "your question" --top 5
BM25-only retrieval:
python3 scripts/retrieve.py "your question" --top 5 --no-rerank
Explain mode:
python3 scripts/retrieve.py "your question" --top 5 --explain
Inspect BM25 directly:
python3 scripts/bm25-index.py query "your question" --top 10
After substantive page changes, manually run contextual-prefix.py --all and then bm25-index.py build. Callers should detect the required retrieval files first and use the legacy query order if provisioning has not completed.
How does this skill compare with similar options?
Compared with the v1.6 hot.md → index.md → read-several-pages flow, this skill adds paragraph chunking, BM25 candidate retrieval, and optional cosine reranking. The --no-rerank mode provides a faster BM25-only alternative with potentially lower quality.