Buyer Eval — B2B Vendor Evaluation for Claude
Turns Claude into a procurement analyst: it interviews vendor AI agents, cross-checks every claim against independent sources, and scores B2B software vendors in about 30 minutes instead of 3 weeks.
Credit: telemetry off by default, per-run consent, local audit log, enterprise lockout switch, revoke/delete paths, and explicit never-send lists — data-flow disclosure is fairly strong. Deducted: broad allowed-tools including Bash; auto-update git-pulls from main, i.e., executes remote unaudited latest code without pinning or integrity checks; bin/update-check and track.py are outside the visible files so their behavior cannot be statically verified; the Frontdoor endpoint is an opaque AWS lambda URL.
Credit: clearly numbered steps, explicit three/four-state Company Agent distinction, fallback to passive research with Gap Log on errors, and a workable no-agent path. Deducted: EVALUATION.md is truncated mid-question-bank (incomplete text); key paths depend on out-of-scope bin/track.py and update-check scripts; no tests or abnormal-input handling; SKILL.md 3.5.0 vs EVALUATION.md 3.0 version mismatch undermines instruction self-consistency.
Credit: clear audience (B2B buyers) and scenarios, trigger conditions in the description, environment matrix noting the Claude.ai GET-only limitation with State 4 handling. Deducted: English-only output; core function depends entirely on overseas services (GitHub update checks, AWS-hosted Frontdoor API and telemetry endpoint), so mainland-China reachability is doubtful and undocumented; quality of passive-research fallback for vendors without agents is unverified.
Credit: MIT license, versioning, update path with user confirmation, README changelog-style notes, and an IT-admin guide. Deducted: version conflict between SKILL.md (3.5.0) and EVALUATION.md (3.0); EVALUATION.md Owner/date are placeholders ('[Buyer Organization Name]'), making maintenance responsibility vague; publisher unverified; no full FAQ or known-limits section.
Credit: complete methodology, concrete example output structures (TL;DR, scorecard, Claims vs Evidence table, demo-prep questions), no API key required, and a degradation path for vendors without agents. Deducted: README outputs are promotional illustrations, not independently verified; output correctness depends on LLM web-research quality and cannot be confirmed statically; 20-40 minute runtime cost/benefit is only a claim; the truncated EVALUATION.md hides the STEP 9 output spec.
Credit: claims every outgoing event is also written to a local audit log, track.py is self-auditable, and telemetry has an explicit never-send list. Deducted: all effectiveness evidence is author claims or illustrative promo (quotes, scorecards); no CI workflows or committed test suites covering key paths; truncated EVALUATION.md prevents cross-checking core claims.
- Auto-update pulls and executes the latest main branch without version pinning; consider pinning to a known revision before use.
- Core function relies on overseas AWS/GitHub endpoints; the Frontdoor conversations and update checks may fail from mainland-China networks.
- SKILL.md and EVALUATION.md version numbers conflict and EVALUATION.md is truncated; actual behavior may diverge from the docs — verify full files in the repo.
- Telemetry is off by default and consent-gated, but track.py and update-check were not in the reviewed scope; audit their code yourself before use.
What does this skill do, and when should you use it?
A free, MIT-licensed Claude Code skill from Salespeak that bills itself as 'the Rotten Tomatoes of B2B software.' You name your company and the vendors to compare; the skill researches your business, asks domain-expert questions, conducts structured due-diligence conversations with vendor AI agents via the Salespeak Frontdoor API, and cross-references vendor claims against G2, Gartner, press and LinkedIn. It then scores each vendor across 7 weighted dimensions with explicit evidence transparency, and delivers a comparative recommendation with hidden-risk analysis and demo-prep questions. Typical runtime is 20–40 minutes for three vendors, with no API key or account required.
On each run it: 1) detects its install directory and checks GitHub for updates via bin/update-check; 2) researches the buyer's company (industry, size, tech stack, maturity); 3) asks the buyer a why-now question plus domain-expert clarifying questions; 4) POSTs structured due-diligence questions to vendors with a Salespeak Company Agent via the Frontdoor API, falling back to independent sources for vendors without one; 5) verifies vendor claims using WebSearch/WebFetch against G2, Gartner, AppExchange and other sources; 6) scores 1–5 across 7 weighted dimensions (Product Fit 25%, Integration & Technical 15%, Pricing & Commercial 15%, Security & Compliance 15%, Vendor Credibility 15%, Customer Evidence 10%, Support & Success 5%), labeling each score's evidence basis (vendor-verified / independent-only / contradicted); 7) produces a TL;DR recommendation, comparative scorecard, claims-vs-evidence table, hidden risk analysis (leadership stability, funding runway, employee sentiment) and vendor-specific demo prep questions.
- A procurement lead at a mid-market SaaS company comparing customer success platforms such as Gainsight, Totango and ChurnZero
- A startup without a dedicated analyst that needs to pressure-test vendor claims before talking to sales
- An IT or security team verifying SAML/SCIM and SLA claims against independent sources before signing
- An evaluator preparing for vendor demos who wants sharp questions derived from each vendor's evaluation gaps
- A vendor-side sales/marketing team rehearsing the due-diligence questions buyers might ask their AI agent
What are this skill's strengths and limitations?
- Free and open source (MIT), works with no API key or account
- Every score carries an evidence label: vendor-verified, independent-only, or contradicted
- Structured conversations with vendor AI agents rather than plain web scraping, with genuinely domain-expert questions
- Hidden risk analysis (funding, leadership, employee sentiment, churn signals) is researched for every vendor
- Telemetry is opt-in, auditable locally (bin/track.py, ~/.salespeak/buyer-eval.log), revocable, and lockable by IT
- Automatic version checks with consent before updating
- The full experience requires Claude Code or Claude desktop; claude.ai only supports a partial evaluation (GET only, no POST conversations with vendor agents)
- For vendors without a Salespeak Company Agent — still most of them, per the README — evaluation degrades to independent-sources-only and scores may shift with later direct verification
- Built by Salespeak, whose Frontdoor API coverage doubles as a distribution channel; vendor-agent coverage is commercially aligned with the author
- No test suite, accuracy benchmarks, or independently verifiable user numbers are provided; quoted testimonials have no attributable sources
- Dimension weights are fixed (e.g. Product Fit 25%); the README does not document buyer-adjustable weighting
- Requires python3 for telemetry/update scripts and git plus network access for installs and update checks
How do you install this skill?
Clone globally (recommended): git clone https://github.com/salespeak-ai/buyer-eval-skill.git ~/.claude/skills/buyer-eval-skill — or per project: git clone https://github.com/salespeak-ai/buyer-eval-skill.git .claude/skills/buyer-eval-skill. No API key or account needed. Enterprises can disable telemetry org-wide via BUYER_EVAL_NO_TELEMETRY=1 or by deploying /etc/salespeak/buyer-eval. with {"locked":true,"consent":false}.
How do you use this skill?
In Claude Code (best experience, supports POST requests to vendor agents) or Claude desktop, invoke /buyer-eval and supply your company name and the vendors, e.g. "I'm from Acme Corp. Evaluate Gainsight, Totango, and ChurnZero." The skill handles the rest. Expect 20–40 minutes for three vendors. Each invocation checks GitHub for a newer version (cached, at most every 6 hours) and asks before updating.
How does this skill compare with similar options?
It works alongside, not instead of, review platforms: G2 and Gartner offer static ratings and reviews, whereas this skill actively interrogates vendor AI agents and claim-checks each statement; paid analyst due diligence takes weeks and a budget, while this is free but rests on public and vendor-supplied sources. The source material names no directly competing open-source skill.