find-paper-PDF

Find and download academic papers by author, institution, and date range. Searches PubMed with fuzzy author/institution matching, flags field outliers, downloads PDFs via a bundled paper-fetch engine, and reports failed papers with one-click DOI links.

Workflow

Type /find-paper-PDF to start.

Execute every step in order. Stop and report if a hard failure occurs (network down, Python missing).

Path definitions for this run:

• SKILL_DIR = directory containing this SKILL.md
• SCRIPTS = SKILL_DIR/scripts/
• CWD = the user's current working directory (where they invoked the skill)
• All output (temp files, PDFs) goes under <CWD>/output/

Step 1 — Gather user input

Ask the user directly in your response (do NOT use AskUserQuestion). Ask the questions one at a time — wait for the user's reply before asking the next:

1. Ask: "Author name? (First Name + Last Name, e.g. Jane Doe, Wei Zhang — both name orders will be tried automatically)" Wait for reply.
2. Ask: "Institution? (Be specific — department, hospital, university, city)" Wait for reply.
3. Ask: "Date from? (YYYY-MM-DD)" Wait for reply.
4. Ask: "Date to? (YYYY-MM-DD)" Wait for reply.

Step 2 — Run PubMed search

Run the bundled search script:

python3 "<SCRIPTS>/pubmed_search.py" \
  --author "<user_author>" \
  --institution "<user_institution>" \
  --date-from "<date_from or empty>" \
  --date-to "<date_to or empty>" \
  --out "<CWD>/output"

What it does:

1. Generates multiple name-order variants (Chinese Last-First and Western First-Last) and tries all of them
2. Searches PubMed with full-name queries only (no initials fallback). Returns up to 300 papers per variant, sorted newest first
3. If zero full-name matches across all variants, stops early and reports "No papers found"
4. Fetches metadata (title, authors, DOI, journal, date) and full affiliation text for every candidate PMID
5. Deduplicates by DOI
6. Outputs NDJSON — one paper per line, then a summary line with "summary": true

The script does NOT filter by institution or field — it returns everything. The final summary includes a common_affiliations list showing which affiliation strings appear most frequently across all papers. This is the data you need for the next step.

Step 3 — Claude judges institution match and field outliers

Read through the paper output lines (everything before the summary line). For each paper, look at its affiliations array.

Institution matching (Claude's judgment):

Read the paper's affiliations and decide if this author-at-this-institution plausibly appears on this paper. Use your language understanding:

• Handle typos: "Nineth" → "Ninth", "Opthalmology" → "Ophthalmology"
• Handle partial names: "City Hospital" should match "City Hospital, University School of Medicine"
• Handle dual affiliations: an author may have papers listing either of two labs. If a clear secondary institution appears frequently in the results, accept it.
• Reject clearly different people: same name + same hospital name but different department AND different city means a different person. Same person can co-author across departments within the same hospital.

The common_affiliations list in the summary is a good reference for spotting the author's true affiliations.

Field outlier detection (Claude's judgment):

Read each paper's title and journal. Determine the author's dominant research area (e.g., ophthalmology, myopia). Flag papers that are clearly in a completely different field. Examples of true outliers:

• A myopia researcher appearing on an antibiotic-resistance paper → likely different person with same name → EXCLUDE
• A myopia researcher appearing as one of 200+ authors on a national aging consortium paper → same person, peripheral contribution → KEEP but mark as OUTLIER

Do NOT over-flag: a myopia researcher publishing on retinal dopamine or scleral biomechanics is still in the same broad field.

After judging, produce two lists:

• KEPT: papers that pass institution + field checks (including those marked OUTLIER)
• EXCLUDED: papers that fail (with brief reason — "different person, same name in another city", "wrong department", etc.)

Step 4 — Save DOI list and start download

Extract DOIs from the KEPT papers only. Write them to a temp batch file:

echo "<doi1>" > "<CWD>/output/dois.txt"
echo "<doi2>" >> "<CWD>/output/dois.txt"
...

Then run paper-fetch in batch mode:

python3 "<SCRIPTS>/paper_fetch.py" \
  --batch "<CWD>/output/dois.txt" \
  --out "<CWD>/output/papers" \
  --format text

Important rules when calling paper-fetch (from learned errors):

• Never pass multiple DOIs as positional arguments — always use --batch with a file
• Always quote file paths that contain spaces or parentheses
• Run each batch once — if retrying is needed later, add --overwrite and delete broken files first

Step 5 — Verify downloads

Check which PDFs are valid:

python3 -c "
import os, glob
out = r'<CWD>/output/papers'
for f in sorted(glob.glob(os.path.join(out, '*.pdf'))):
    with open(f, 'rb') as fh:
        is_pdf = fh.read(4) == b'%PDF'
    size_kb = os.path.getsize(f) / 1024
    print(f'[{\"OK\" if is_pdf else \"BROKEN\"}] {os.path.basename(f)} ({size_kb:.0f} KB)')
"

Important: Use fh.read(4) == b'%PDF' (4 bytes), NOT fh.read(5) == b'%PDF' (5 bytes — will always fail).

Delete any files that are NOT valid PDFs (broken downloads).

Step 6 — Report results

Present a structured summary to the user.

Downloaded papers table:

Status	PMID	Date	Journal	Title (truncated)	File
DOWNLOADED	...	...	...	...	filename.pdf
OUTLIER	...	...	...	...	filename.pdf

Failed papers table (for any DOI where no PDF was downloaded):

PMID	Date	Journal	Title	DOI Link
...	...	...	...	https://doi.org/...

Format DOI links as clickable markdown: [https://doi.org/10.xxx/yyy](https://doi.org/10.xxx/yyy)

If any failed paper is marked as "is_open_access": true by the search script, add a note: "This paper is open access but the publisher's anti-bot system blocked the download. Opening the DOI link in a browser should work."

Summarize by failure reason categories (paywall, too-new, publisher-anti-bot, etc.).

Step 7 — Clean up

Remove all temporary files created during the run:

rm -f "<CWD>/output/dois.txt" "<CWD>/output/metadata.json" "<CWD>/output/search_results.jsonl"

Keep the downloaded PDFs in <CWD>/output/papers/.

---

Script files

File	Purpose
scripts/pubmed_search.py	PubMed search, institution filtering, field outlier detection, dedup
scripts/paper_fetch.py	Bundled paper-fetch engine (Unpaywall → Semantic Scholar → arXiv → Europe PMC → PMC → bioRxiv → publisher-direct → Sci-Hub)

---

Error recovery

• Python not found: If python3 fails, tell the user "Python 3 is required. Install it from https://python.org, then try again." and stop.
• Network errors during PubMed search: The script retries E-utilities calls up to 3 times with 2-second backoff.
• paper-fetch exit code 4 (transport error): Re-run the failed DOIs once with --overwrite.
• paper-fetch exit code 1 (unresolved): Move those DOIs to the failed list — they legitimately have no OA copy.
• No papers found: Check if the author name order is reversed (Chinese vs Western order) and suggest the user try the other order.

Notes

• Paper-fetch is bundled internally — no separate install needed.
• For better download coverage, set UNPAYWALL_EMAIL and PAPER_FETCH_INSTITUTIONAL=1 as environment variables yourself.
• All temp files go under <CWD>/output/ and are cleaned up at the end.