Project Testing Guidelines

Starting the Test Environment

1. Check if Backend is Running with Test Helpers

curl -s http://localhost:8000/test/health | grep -q '"testing": true' && echo "✅ Test backend ready" || echo "❌ Test backend not running"

2. If Not Running, Start Backend

cd backend
source venv/bin/activate
TESTING=true MOCK_LLM=true python3 -m uvicorn app:app --reload

3. Check if Frontend is Running

curl -s http://localhost:5173 > /dev/null && echo "✅ Frontend ready" || echo "❌ Frontend not running"

4. If Not Running, Start Frontend

cd frontend/frontend
npm run dev

Environment Variables:

• TESTING=true - Required - Enables test helper endpoints (/test/*)
• MOCK_LLM=true - Optional - Uses mock LLM responses for fast testing (no API calls)

URLs:

• Frontend: http://localhost:5173
• Backend: http://localhost:8000

Test Directory Structure

Playwright E2E Tests (Frontend)

frontend/frontend/tests/
├── analytics/           # GA4 tracking & analytics tests
│   ├── helpers.js       # Analytics capture utilities
│   ├── validate-analytics.spec.js
│   ├── upload-analytics.spec.js
│   └── results-revealed-analytics.spec.js
├── components/          # React component tests
│   ├── pricing-table-credits.spec.js
│   └── pricing-table-passes.spec.js
├── e2e/                 # End-to-end flow tests
│   ├── checkout/        # Payment & upgrade flows
│   │   ├── checkout-flow.spec.js
│   │   ├── pricing-integration.spec.js  # DB tests (serial)
│   │   ├── pricing_variants.spec.cjs    # DB tests (serial)
│   │   ├── upgrade-event.spec.cjs
│   │   ├── upgrade-funnel.spec.cjs
│   │   └── upgrade-tracking.spec.js
│   ├── core/            # Core validation features
│   │   ├── async-polling-validation.spec.js
│   │   ├── auto-scroll.spec.js
│   │   ├── e2e-full-flow.spec.cjs       # Production-only
│   │   ├── error-recovery.spec.js
│   │   └── markdown-formatting.spec.js
│   ├── gated-results/   # Freemium gating tests
│   ├── pseo/            # Programmatic SEO tests
│   │   └── pseo-smoke.spec.js
│   ├── ui/              # UI component integration
│   │   └── validation-table-header.spec.js
│   ├── upload/          # File upload tests
│   │   ├── file-processing.spec.js
│   │   └── upload.spec.js
│   └── user/            # User flow tests
│       ├── user-access.spec.js
│       └── user-tracking-flow.spec.js
├── helpers/             # Shared test utilities
│   └── test-utils.js
├── internal/            # VPN-only dashboard tests
│   └── e2e-internal-dashboard.spec.cjs
├── global-setup.js
└── global-teardown.js

Pytest (Backend)

• Location: tests/ and backend/tests/ (Python)
• Config: pytest.ini
• Run: source venv/bin/activate && pytest

Playwright Project Configuration

The test suite uses smart parallelization via Playwright projects:

Always ignored:

• **/internal/** - VPN-only dashboard tests
• **/e2e-full-flow.spec.cjs - Production-only (run via deploy script)

Critical Patterns

1. Database Tests Must Run Serially

Database tests (pricing-integration.spec.js, pricing_variants.spec.cjs) must run with workers: 1 due to SQLite WAL visibility issues. The Playwright config handles this automatically.

2. Use Robust Waiting Patterns (Avoid waitForTimeout)

Replace waitForTimeout with proper waiting strategies:

// ❌ DON'T: Fixed timeout
await page.waitForTimeout(2000);

// ✅ DO: Wait for condition
await expect(submitButton).toBeEnabled();
await expect(page.locator('[data-testid="results"]')).toBeVisible();

// ✅ DO: Wait for network
await page.waitForLoadState('networkidle');

// ✅ DO: Poll for state changes
await expect.poll(async () => {
  const text = await page.locator('.status').textContent();
  return text;
}, { timeout: 30000 }).toBe('Complete');

3. Database Polling for WAL Visibility

After writing to the database via test helpers, poll until the data is visible:

await page.evaluate(async (userId) => {
  for (let i = 0; i < 60; i++) {
    const response = await fetch(`http://localhost:8000/test/get-user?user_id=${userId}`);
    const user = await response.json();
    if (user.has_pass) return;
    await new Promise(resolve => setTimeout(resolve, 200));
  }
  throw new Error('Data not visible within timeout');
}, userId);

4. Handle Gated Results Overlay

Always check for and dismiss gated results before asserting on validation results:

if (await page.locator('[data-testid="gated-results"]').first().isVisible()) {
  await page.locator('button:has-text("View Results")').first().click();
}

5. Analytics Testing Pattern

Use the analytics helpers for GA4 event testing:

import { setupAnalyticsCapture, waitForEvent, getEventsByName } from '../analytics/helpers.js';

const capturedRequests = setupAnalyticsCapture(page);
// ... trigger action ...
const eventCaptured = await waitForEvent(capturedRequests, 'page_view', 5000);
expect(eventCaptured).toBe(true);

Test Helper Endpoints

Base URL: http://localhost:8000/test/ (requires TESTING=true)

Test Job Indicator

Use "testtesttest" in citations to flag test jobs. These are filtered from dashboard metrics.

Anti-Patterns to Avoid

1. DON'T use waitForTimeout for waiting on state - use expect.poll() or waitFor()
2. DON'T assume data is immediately visible after database writes - poll for state
3. DON'T forget to handle gated results overlay in E2E tests
4. DON'T modify database tests to run in parallel - they share SQLite
5. DON'T hardcode variant values - use localStorage.setItem('experiment_v1', 'X')
6. DON'T start backend without TESTING=true when running E2E tests
7. DON'T run internal tests locally - they require VPN access

Running Tests

Playwright E2E - Common Commands

cd frontend/frontend

# Run all tests on Chromium (default)
npx playwright test --project=chromium

# Run database tests only (serial execution)
npx playwright test --project=database-tests

# Run a specific test file
npx playwright test tests/e2e/core/auto-scroll.spec.js

# Run tests matching a pattern
npx playwright test -g "auto-scroll"

# Run with headed browser (for debugging)
npx playwright test --headed

# Run cross-browser suite
npx playwright test --project=firefox --project=webkit

Pytest

source venv/bin/activate
pytest                                 # All tests
pytest tests/test_app.py              # Single file
pytest -k "test_name"                 # By name pattern

Related Skills

• superpowers:test-driven-development - TDD workflow
• superpowers:testing-anti-patterns - Common testing mistakes