Job Scraper — Local-First Job Search Skill
Automatically searches multiple job portals for new postings matching your profile, deduplicates across runs, and presents them sorted by fit — no more manual site-hopping.
Scoped allowlist (bun-run installed CLIs, job_key.py, WebFetch/WebSearch), health check may edit only the enabled toggle and only with confirmation, Step 4.5 explicitly forbids programmatic LinkedIn people scraping; SECURITY.md and CI security-guards back least-privilege and data-boundary claims. Deducted: instruction-level defenses are not a sandbox (self-acknowledged), agent issues network requests to multiple overseas portals, no explicit rollback for state-file writes, publisher unverified. Not full marks.
Highly self-consistent instructions: WebSearch fallback when bun is missing, log-and-continue on CLI failures, client-side date filtering, URL-resolvability rule, closed-at-source detection; the underlying CLIs carry fixture tests. Deducted: the orchestration itself was not executed in this static review, search-queries.md is full of placeholders requiring correct user setup, and multi-agent parallel state consistency cannot be verified statically.
Triggers and optional arguments (focus/broad/health) are explicit, scenario clear, portal skills are swappable for localization. Deducted: core search depends on LinkedIn/Google/WebSearch — poor reachability from mainland-China networks; no Chinese-language support declared; non-fit boundaries (non-job-search input) not explicitly stated.
Well-layered docs (SKILL.md + search-queries.md), MIT license, framework-version CI guard, upstream update tooling, and a security reporting channel; known limitations are disclosed. Deducted: the skill has no independent version/changelog, cross-file dependencies (CLAUDE.md, /rank) must be inferred, and maintenance rests on a single author.
Dedup, state persistence, health checks and quality guardrails give clear marginal value over manual searching; output format is specified. Deducted: static review cannot verify real scrape success; substantial upfront configuration (profile, queries, CLI installs) limits out-of-box value; README author results are self-reported.
CI workflows, committed CLI fixture/mock tests, SECURITY.md, and cited concrete incidents (e.g. the stale freehire posting) provide auditable material. Deducted: no committed tests target the orchestration key path (search→dedup→present), no independent third-party reproduction, and the static-review ceiling is 5, so 4.
- This is a static source review; no code was executed. Real scrape success rates and cross-portal behavior are unverified.
- Core search depends on overseas services (LinkedIn, Google/WebSearch); reachability from mainland-China networks may require a proxy.
- Placeholders in search-queries.md and the personal profile must be configured before use, or queries will run with literal template tokens.
- Defenses are instruction-level, not a sandbox: on untrusted job boards, review what the agent fetched and wrote before sending anything out.
- Publisher is not verified by the FollowSkills registry; the repo contains 9 SKILL.md files and only job-scraper was scored here.
What does this skill do, and when should you use it?
Job Scraper is the job-searching skill of the ai-job-search framework (.claude/skills/job-scraper). It searches installed job-portal CLIs for new postings matching your profile, falling back to web search where no CLI exists; it deduplicates across runs against a seen-jobs file and your application tracker, and gives each new job a quick high/medium/low fit assessment. Results arrive as a fit-sorted table, with LinkedIn referral-contact search links generated for high and medium matches. The surrounding framework was built and dogfooded by a geophysicist during his own job search (he reports it landed him an AI engineer role) and targets individual job seekers. Caveat: of the six bundled portal CLIs, four cover the Danish market only — users elsewhere must generate their own portal skills via the repo's /add-portal command.
Reads the search strategy in search-queries.md plus historical state in seen_jobs. and job_search_tracker.csv; primarily calls each installed portal CLI under .agents/skills/ (run with Bun, scoped to the last 14 days, capped at ~20 results per call, portals run in parallel), falling back to WebSearch/WebFetch when CLIs are missing or fail; fetches full detail via each portal's detail command for promising hits (key requirements, application deadline, description snippet) and detects closed-at-source postings, marking them expired; consolidates identical cross-city mass postings into a single flagged row; assigns a quick fit rating with a language-gate override rule; derives canonical dedup keys via tools/job_key.py and writes every fetched job to seen_jobs. (with portal, source, posted_date, and more); builds two LinkedIn people-search URLs per high/medium-fit job (recruiter path and team-peer path) — links only, never scraped; runs a bounded portal health check (at most one probe and one retry per portal) producing degraded/broken/inconclusive verdicts; finally presents new jobs in a table, noting disabled and fallback portals.
- A job seeker who wants to proactively surface new postings weekly instead of manually browsing Jobindex, Jobnet, LinkedIn, and other boards
- Existing ai-job-search users who want scraped jobs deduplicated and handed straight into the /rank and /apply pipeline
- Users targeting a specific direction, e.g. '/scrape data science' to prioritize a query category, or '/scrape broad' to run everything
- Users unsure whether their portals still work, running '/scrape health' to probe each portal CLI (e.g. '/scrape health jobnet' for a single portal) without searching
- Applicants hunting referrals, using the generated LinkedIn recruiter/peer search links to make warm outreach themselves
What are this skill's strengths and limitations?
- Runs entirely on your machine; the framework is free and open source (MIT), with no subscriptions or API costs
- Durable cross-run deduplication (seen_jobs. + application tracker + canonical keys via a helper script) means you never see the same posting twice
- Robust engineering: per-portal enable/disable toggles, automatic WebSearch fallback on CLI failure, and a bounded health check that catches silently rotting parsers
- Honest about data: it forbids fabricated postings, records posted_date and source for later ghost-job audits, and flags mass-posting patterns without accusing employers
- The referral-contact step is deliberately zero-dependency — it generates LinkedIn search links only, no scraping, no credentials
- The four bundled Danish portal CLIs (Jobbank, Jobdanmark, Jobindex, Jobnet) cover only Denmark; other markets require generating and maintaining your own portal skills via /add-portal
- Portal CLIs depend on target-site structure — a site redesign makes parsers silently return junk until the health check flags it and you fix the skill
- linkedin-search rides unofficial public endpoints; automated access violates LinkedIn's Terms of Service, so it must stay low-volume personal use and carries block risk
- Deeply tied to Claude Code: allowed-tools, the Agent tool, AskUserQuestion, and /add-portal are Claude Code mechanisms; porting to other agent tools requires adaptation
- The quick fit rating is only a coarse signal — full evaluation lives elsewhere in the framework (/rank and 04-job-evaluation.md), so the skill alone has limited judgment
How do you install this skill?
The skill ships inside the MadsLorentzen/ai-job-search repository (MIT licensed) and installs with the framework: 1) fork and clone with gh repo fork MadsLorentzen/ai-job-search --clone; 2) run bun install inside each portal CLI directory under .agents/skills/ (jobbank-search, jobdanmark-search, jobindex-search, jobnet-search, linkedin-search, freehire-search; linkedin-search and freehire-search have zero runtime dependencies and can skip this); 3) prerequisites: Claude Code CLI, Python 3.10+, and Bun; 4) run /setup to build your profile. The skill itself lives at .claude/skills/job-scraper/ and needs no separate installation.
How do you use this skill?
Inside Claude Code, run /scrape, or say 'Find new jobs' or 'Any new positions?'. Optional arguments: '/scrape data science' (prioritize a query category), '/scrape broad' (run all categories), or '/scrape health' (portal health check only, e.g. '/scrape health jobnet' to probe one portal even if disabled). After the run, pick a table row number to hand that job to the job-application-assistant skill for full evaluation (/apply); with roughly 8+ new jobs, prefer /rank, which batch-scores everything into a ranked shortlist.
How does this skill compare with similar options?
The README contrasts the framework with 'most LaTeX-resume templates', which lack PDF compile verification — but that argument targets /apply, not scraping. For job searching itself, the de facto alternatives are manually browsing job boards or using LinkedIn's built-in alerts; this skill's differentiators are parallel multi-portal search, local deduplication, and handoff into the apply pipeline — though its coverage depends entirely on whether your market has a working portal CLI.