A skill review method that does not confuse popularity with quality
FSRS records observable facts, reproducible tests and editorial judgment separately. The total helps rank; dimensions explain why; confidence tells you how stable the conclusion is.
Four governing principles
Evidence first
Stars, unverified reputation and polished docs are leads. Verified enterprise origin is governance evidence, not a substitute for inspecting behavior, permissions and task results.
Risk is not averaged away
Malice, covert exfiltration and destructive defaults cannot be cancelled out by high scores elsewhere.
Unknown stays unknown
When material or testing is insufficient we say so; missing information is never assumed safe or turned into fake precision.
Scores expire
A score belongs to a reviewed version and date; material changes to the repo, dependencies or platform trigger review.
How official enterprise publication affects a score
Verified organization accounts for publishers such as OpenAI, Anthropic, Microsoft and Google strengthen provenance, ownership and update-path evidence only. There is no fixed brand bonus. Official origin cannot prove reliability, safety or utility and never overrides a red line; independent or unverified publishers are not automatically penalized.
Six dimensions, 100 points
Each dimension uses zero, midpoint and full-score anchors. Reviewers must record why points were withheld. Safety carries the largest weight.
01Utility & fit20
Solves a clear, real problem and states where it does—and does not—fit.
- 0: vague or unverifiable value
- 10: clear use case, limited evidence or boundaries
- 20: clear audience, benefit and limits with support
02Technical quality & reliability20
Instructions, scripts and dependencies agree; key paths repeat; failures are safe and intelligible.
- 0: broken, contradictory or unrunnable
- 10: happy path works; testing or failure handling is thin
- 20: key paths reproduced with controlled failure modes
03Safety, privacy & control25
Uses least privilege, confirms consequential actions, and makes sensitive-data flows explicit.
- 0: red-line risk or deliberate concealment
- 12: risks visible; permissions, consent or rollback incomplete
- 25: least privilege, explicit consent, rollback and transparent data flow
04Evidence & verifiability15
Key claims have traceable sources, reproducible tests or independent usage evidence; facts and inference are separated.
- 0: author claims or marketing only
- 7: primary material or demo with limited coverage
- 15: multiple evidence types corroborate reproducible conclusions
05Usability & transparency10
Installation, dependencies, I/O, cost, compatibility and limits support an informed decision before use.
- 0: critical usage information missing
- 5: usable with significant hidden assumptions
- 10: clear from adoption through removal
06Maintenance & governance10
Verified publisher provenance, licensing, versions, changelogs, responsiveness and dependency pinning reduce long-term risk. Official enterprise origin supports governance only; it does not replace testing or safety evidence.
- 0: unclear origin/license or evidently abandoned
- 5: active but governance is incomplete
- 10: ownership, license, versions and updates are clear
How to read the total
Strongly recommended
Good evidence and no uncontrolled major risk; still check your environment and data sensitivity.
Recommended
Generally reliable with disclosed limitations; trial as directed and keep a rollback path.
Use with care
Useful, but reliability, evidence or controls still have material gaps.
Not recommended
Current benefit does not outweigh risk or uncertainty.
Why the score also has an evidence-confidence label
Two skills scoring 82 should not be trusted equally if one was reproduced end-to-end and the other only read on paper. Confidence never raises the score; it describes how robust the conclusion is.
Skill and key scripts inspected, key paths reproduced, with at least one independent or cross-source form of evidence.
Primary material inspected and partly reproduced; platform, edge-case or third-party coverage remains incomplete.
Mostly static review, author material or a limited demo; useful for discovery, not high-risk decisions.
Evidence ladder
We prefer evidence closer to actual behavior and easier to reproduce independently. Corroboration across evidence types is stronger than one source.
Reproducible behavior
Task runs, safety tests, failure and rollback checks in an isolated environment.
Auditable primary material
SKILL.md, scripts, dependency manifests, commits, releases and licenses.
Independent usage evidence
Verifiable issue reports, third-party reproduction and maintainer responses.
Weak signals
README claims, demos, stars, downloads and social discussion—context only.
Red lines and override rules
Any unresolved item below sets the status to Not recommended / Blocked regardless of the numeric total:
- Malware, credential theft, covert tracking or undisclosed data exfiltration;
- Irreversible or high-impact default actions without clear consent, scoping and recovery;
- Fabricated provenance, test results, compatibility or safety claims;
- Known critical vulnerable dependencies without mitigation, or permissions grossly disproportionate to the task.
How one review is completed
Frame the claim
Identify intended users, core job, claimed platforms, I/O and permission boundaries.
Collect evidence
Pin a repo version; inspect the skill, scripts and dependencies; record links and unknowns.
Screen risk first
Review data flows, external writes, command execution, secrets, permissions and red lines.
Reproduce tasks
Run a representative happy path and one failure case in isolation; verify exit or rollback.
Score independently
Apply anchors dimension by dimension, record deductions, then calculate total and confidence.
Publish and revisit
A second reviewer checks safety, red lines and confidence; then publish version, date, dimensions and risks.
What the score does not mean
FSRS is not a security certification, academic peer review or performance guarantee. It is a structured judgment of evidence for a specific version and test scope. A high score does not fit every user; a personal low-risk choice may differ from an enterprise production decision. Before adopting, consider permissions, data sensitivity, cost and rollback in your own context.
How the standard evolves
The version changes when dimensions, weights or override rules materially change. Editorial clarifications are minor revisions. Historical versions remain available so old scores stay interpretable; scores on an older standard are marked and queued for review.