name	documentation-testing
description	Test documentation completeness, accuracy, findability - copy-paste-run validation before release

Documentation Testing

Overview

Test documentation like you test code. Core principle: If you haven't tried it, it's broken.

Key insight: Untested documentation always has issues. Copy-paste-run test finds them before users do.

When to Use

Load this skill when:

Finalizing documentation before release
Reviewing documentation quality
Creating documentation quality gates
After major doc updates

Symptoms you need this:

"Is this documentation good enough to ship?"
Preparing installation guides, quick starts, tutorials
Documentation quality review
Pre-release documentation checklist

Don't use for:

Writing documentation (use muna/technical-writer/clarity-and-style)
Structuring documentation (use muna/technical-writer/documentation-structure)

Five Testing Dimensions

Test documentation across 5 dimensions:

1. Completeness Testing

Question: Can reader accomplish the task with ONLY this documentation?

2. Accuracy Testing

Question: Do all examples, commands, and instructions actually work?

3. Findability Testing

Question: Can users find this documentation when they need it?

4. Example Verification

Question: Can you copy-paste every example and have it run without modification?

5. Walkthrough Testing

Question: Can a new user follow this successfully on a clean system?

Dimension 1: Completeness Testing

Goal: Verify reader can complete task without external resources.

Checklist

All prerequisites listed - OS, language versions, required accounts, tools
All configuration options documented - Every setting, not just defaults
Error cases covered - What can go wrong and how to fix
Troubleshooting section - Common issues with solutions
Success criteria - "How do I know it worked?"
Next steps - What to do after completing this doc

Example: Installation Guide

❌ Incomplete:

## Installation

Run:
\`\`\`bash
npm install our-app
\`\`\`

You're done!

Missing: Prerequisites, error handling, verification

✅ Complete:

## Installation

### Prerequisites
- Node.js 18+ (`node --version` to check)
- npm 9+ (`npm --version` to check)
- Active internet connection

### Install

\`\`\`bash
npm install our-app
\`\`\`

### Verify Installation

Check installed version:
\`\`\`bash
npx our-app --version
# Expected output: our-app v2.1.0
\`\`\`

### Troubleshooting

**Error: "EACCES: permission denied"**
- Solution: Run with sudo: `sudo npm install -g our-app`

**Error: "Unsupported engine"**
- Solution: Upgrade Node.js to 18+

### Next Steps
- [Quick Start Guide](./quick-start.md)
- [Configuration Reference](./config.md)

Testing Method

The "Clean Slate Test":

Can someone with ZERO context complete this?
Read doc, follow instructions
Note every moment you had to Google or guess

Dimension 2: Accuracy Testing

Goal: Verify all examples, commands, and instructions work.

Checklist

Code examples run - Copy-paste into environment, executes without errors
Commands correct - No typos, correct options, work on stated OS
Version numbers current - Not referencing outdated versions
Screenshots up-to-date - Match current UI
Links work - No 404s, links go to correct pages
Output matches examples - Documented output = actual output

Example: API Documentation

❌ Inaccurate:

Make a request:
\`\`\`bash
curl https://api.example.com/users
\`\`\`

Returns user list.

Issues: No authentication, vague output

✅ Accurate:

Make a request:
\`\`\`bash
curl -H "Authorization: Bearer YOUR_API_KEY" \\
     https://api.example.com/v1/users
\`\`\`

Response:
\`\`\`json
{
  "users": [
    {"id": 1, "name": "Alice"},
    {"id": 2, "name": "Bob"}
  ],
  "total": 2
}
\`\`\`

Status: 200 OK

Testing Method

Copy-Paste-Run Test:

Copy EVERY code example
Paste into clean environment
Run without modifications
Verify output matches documentation
If ANY example fails, documentation is inaccurate

Dimension 3: Findability Testing

Goal: Verify users can find documentation when needed.

Checklist

Keywords present - Terms users would search for
Linked from related pages - Cross-references in both directions
In navigation/TOC - Appears in sidebar, index, sitemap
Search engine optimized - Title, headers, meta description
Clear title - Describes content accurately

Example: Deployment Guide

❌ Not Findable:

# Guide

Deploy the app...

Issues: Generic title, no keywords

✅ Findable:

# Deploying to AWS ECS with Docker (Production)

**Keywords**: AWS, ECS, Fargate, Docker, deployment, production, continuous deployment, CI/CD

Deploy our application to AWS Elastic Container Service (ECS) using Docker containers...

**Related**:
- [Docker Configuration](./docker.md)
- [CI/CD Pipeline](./cicd.md)
- [Environment Variables](./env-vars.md)

Testing Method

Search Simulation:

What would user search for? ("deploy to AWS", "ECS deployment", "Docker production")
Search your docs with those terms
Does this page appear in top 3 results?

Dimension 4: Example Verification

Goal: All examples work without modification.

Checklist

Examples are complete - Include all necessary imports, setup
No placeholders without explanation - If using YOUR_API_KEY, explain how to get it
Environment specified - Language version, OS, dependencies
Async/await correct - Don't forget await on promises
Error handling shown - Not just happy path

Example: Code Example

❌ Unverified:

const users = client.get('/users');
console.log(users);

Issues: Missing await, won't work

✅ Verified:

// Prerequisites: npm install api-client
// Environment: Node.js 18+

import APIClient from 'api-client';

// Get API key from https://dashboard.example.com/settings
const client = new APIClient(process.env.API_KEY);

async function getUsers() {
  try {
    const users = await client.get('/users');
    console.log('Users:', users);
    // Output: Users: [{id: 1, name: 'Alice'}, {id: 2, name: 'Bob'}]
  } catch (error) {
    console.error('Failed to fetch users:', error.message);
  }
}

getUsers();

Testing Method

Literal Copy-Paste Test:

Copy example
Create new file
Paste (no modifications)
Run
Does it work? If not, example is broken.

Dimension 5: Walkthrough Testing

Goal: New user can follow successfully on clean system.

Checklist

Test on clean system - Fresh VM/container, not your dev machine
Follow every step literally - No shortcuts, no cached knowledge
Note confusion points - Every time you have to guess or Google
Verify timing claims - "5 minute setup" actually takes 5 minutes?
Test with beginner - Colleague unfamiliar with project

Example: Quick Start

❌ Untested:

# Quick Start (5 minutes)

1. Install the CLI
2. Configure your credentials
3. Deploy your first app

Done!

Issues: Vague steps, unverified timing

✅ Walkthrough-Tested:

# Quick Start (15 minutes)

## Prerequisites (2 min)
- [ ] Ubuntu 22.04 or macOS 12+
- [ ] 2GB free disk space
- [ ] Internet connection

## Step 1: Install CLI (5 min)

\`\`\`bash
curl -L https://releases.example.com/cli.sh | bash
\`\`\`

Verify:
\`\`\`bash
our-cli --version
# Should output: our-cli v2.1.0
\`\`\`

**Troubleshooting**: If command not found, close and reopen terminal.

## Step 2: Configure Credentials (3 min)

Get API key: https://dashboard.example.com/settings

\`\`\`bash
our-cli auth login YOUR_API_KEY_HERE
\`\`\`

Success message: "✓ Authenticated as user@example.com"

## Step 3: Deploy First App (5 min)

\`\`\`bash
mkdir my-app && cd my-app
our-cli init
our-cli deploy
\`\`\`

**Success criteria**: URL shown: "✓ Deployed to https://my-app-abc123.example.com"

Visit URL in browser - you should see "Hello World"

## Next Steps
- [Add custom domain](./custom-domains.md)
- [Configure environment variables](./env-vars.md)

Testing Method

New User Walkthrough:

Spin up clean VM/container
Follow guide step-by-step as written
Don't use any cached knowledge
Note EVERY point of confusion
Time how long it actually takes

Or: Give to colleague who's never used the product. Watch them follow it. Note every question they ask.

Testing Workflow

Use this workflow before releasing documentation:

Phase 1: Quick Checks (10 min)

Read through once - obvious errors?
Check all links - do they work?
Scan for placeholders - any unexplained YOUR_X_HERE?
Verify versions - are version numbers current?

Phase 2: Example Verification (30 min)

Copy EVERY code example
Paste into clean environment
Run without modifications
Verify output matches docs

Phase 3: Completeness Check (15 min)

Prerequisites listed?
Error cases covered?
Troubleshooting section?
Success criteria ("how do I know it worked")?

Phase 4: Walkthrough Test (60 min)

Fresh VM/container
Follow as new user
Note confusion points
Verify timing claims

Phase 5: Findability Check (10 min)

Search docs with user keywords
Check cross-references
Verify in navigation/TOC

Total time: ~2 hours for thorough documentation testing

Common Issues Found By Testing

Issue: Async/Await Missing

Example:

const data = api.get('/endpoint'); // Missing await
console.log(data); // Prints Promise object, not data

Found by: Copy-paste-run test

Issue: Prerequisites Not Listed

Example:

Run: `docker-compose up`

Missing: Docker installed, docker-compose.yml file exists

Found by: Clean system walkthrough

Issue: Environment Variables Not Explained

Example:

const key = process.env.API_KEY; // How do I set this?

Found by: New user walkthrough (where do I get API key?)

Issue: Timing Claims Unverified

Example: "Setup in 5 minutes" actually takes 20 minutes (npm install, account creation, key generation)

Found by: Walkthrough testing with timer

Issue: Success Criteria Missing

Example:

Deploy your app:
\`\`\`bash
deploy.sh
\`\`\`

Missing: How do I know it worked? What URL? What should I see?

Found by: Completeness testing