Playwright CI Test Results
Use SQL to find flaky, failing, and slow Playwright CI tests quickly.
The skill explicitly uses a local DuckDB snapshot and shows GitHub-token and artifact-download commands; however, it does not specify least-privilege permissions, user confirmation, sensitive-log handling, external network effects, or rollback, so points are deducted. No malware, credential theft, or destructive default is evident.
Commands, schema, retry aggregation, and abnormal-result semantics are reasonably clear, and the @duckdb/node-api dependency is supported by the repository. Failure diagnostics for network/auth errors, missing databases, and schema changes are absent, and static review cannot prove execution, so the score is limited.
The audience, question types, and test identity fields are fairly clear, covering failure rates, flakiness, slow tests, and run details. Non-fit cases, input validation, and semantic trigger boundaries are incomplete. Core setup depends on GitHub/npm services, with no mainland-China reachability guidance, so points are deducted.
The document has clear progressive structure with installation, schema, query examples, result semantics, and artifact-retention limits. It lacks skill versioning, changelog, explicit maintenance ownership or update path, and troubleshooting and data-safety guidance. Apache-2.0 licensing and Microsoft provenance are supported by repository files.
The SQL and run-history script directly support flaky-test, failure-rate, slow-test, and per-run investigation tasks and provide meaningful value over manual artifact hunting. Users still must manage downloads, edit identity fields, and handle artifact availability; no static evidence verifies actual outputs, so the score remains below the static ceiling.
The skill names concrete repository scripts, reporter sources, fields, and reproducible SQL, providing useful audit trails. Nothing was executed for this review, and there is no dedicated committed test or independent corroborating execution evidence, so only limited credit is justified.
- Before using GITHUB_TOKEN and gh API, verify token scope, network reachability, and that sensitive test data will not appear in logs.
- The database is a recent, run-count-capped window and artifacts are retained for only seven days; results may be incomplete or unrecoverable.
- The core download flow depends on GitHub, npm, and the gh CLI; mainland-China reachability and authentication availability should be confirmed.
- There are no explicit recovery procedures for missing databases, schema changes, authentication failures, or query errors.
What does this skill do, and when should you use it?
This skill queries Playwright CI results from an aggregated DuckDB database. It supports cross-run flakiness analysis, failure-rate rankings, slow-test investigation, and per-run, SHA, or PR result queries. Results are stored per test attempt, while final verdicts and retry-rescued failures can be derived in SQL. The database contains summaries; full step trees, attachments, and stdio require the original blob report from a recent run.
Uses the repository CLI to download or update the test-results DuckDB snapshot; runs SQL through Node.js and @duckdb/node-api; reads run identity, workflow, event, SHA, PR, CI bot, project, test title, source location, expected and actual status, retry, duration, errors, tags, annotations, and artifact identifiers; groups tests by project_name, file, and test_title; calculates failure and cross-run flakiness metrics; filters tagged tests; renders linked emoji histories for selected test runs; and uses GitHub CLI commands to locate and download a matching blob report when full detail is needed.
- A Playwright maintainer wants to identify tests whose final verdict changes across CI runs.
- A test engineer needs failure-rate rankings by browser project or test identity.
- A CI investigator wants to find slow tests and inspect their recorded error messages.
- A reviewer wants a compact, linked history showing passes, retry-rescued results, and failures.
- A maintainer needs full attachments, step trees, or stdio from a recent GitHub Actions run.
What are this skill's strengths and limitations?
- Replaces manual artifact hunting with SQL over aggregated test summaries.
- Supports cross-run flakiness, retry-rescued failures, failure rates, and slow-test analysis.
- Retains browser project, CI bot, run, SHA, PR, error, and duration fields.
- Can produce compact histories linked to exact GitHub Actions run attempts.
- The database is a recent window; oldest complete runs are evicted by run count.
- A downloaded snapshot may miss the newest runs until update is run.
- The database does not contain full step trees, attachments, or stdio.
- Blob and parquet artifacts are retained for only 7 days, limiting full-detail retrieval.
- The source does not document standalone installation or a separate test suite for this skill.
How do you install this skill?
The source does not document a standalone installation procedure for this skill. It is located at .claude/skills/playwright-test-results/SKILL.md in the repository; the README says that playwright-cli install --skills can install the skill collection. Before querying the database, run npm ci once from the repository root.
How do you use this skill?
Authenticate GitHub CLI, then download the snapshot with GITHUB_TOKEN=$(gh auth token) node utils/test-results-db/cli.ts download. To top it up with the last three days, run GITHUB_TOKEN=$(gh auth token) node utils/test-results-db/cli.ts update --lookback-days 3. Execute SQL against test_results through Node.js and @duckdb/node-api. Example trigger: Find the most cross-run-flaky Playwright tests in the recent database, excluding tests whose expected_status is not passed.
How does this skill compare with similar options?
Compared with manually browsing GitHub Actions artifacts, this skill provides SQL-based aggregation and ranking in DuckDB; manual blob-report retrieval remains necessary for full step trees, attachments, and stdio.