GDUnit4 Test Runner - Core Validation Skill

Instructions for QAEngineer Agent

This is the core validation skill for the "Automated Remediation Loop". You MUST use the scripts/run_gdunit4_tests.py skill (exposed as GodotProject.run_tests()).

This skill performs a complex, multi-step task:

1. Executes the Godot headless instance with the correct, non-obvious command
2. Waits for the process to complete
3. Locates the generated junit_report.xml file
4. Parses this complex XML report to find failure messages and stack traces
5. Returns a structured JSON object summarizing the pass/fail status and all failures

When to Use

Use this skill whenever you need to:

• Validate GDScript implementation quality
• Run comprehensive test suites for automated QA
• Generate structured failure reports for remediation
• Enforce quality gates in the migration pipeline

What This Skill Does

This skill is the lynchpin of the "Agentic Immune System" - it "Reduces" the entire, massively complex task of running a headless test suite and parsing its XML output into a single, deterministic function call that returns structured JSON.

Core Functionality

• Headless Execution: Runs Godot without graphics for CI/CD compatibility
• XML Parsing: Extracts detailed failure information from JUnit XML
• Structured Output: Returns machine-readable JSON for automation
• Error Classification: Categorizes failures by type and severity

Output Structure

The skill returns comprehensive JSON including:

• Test Execution Summary: Pass/fail counts and timing
• Detailed Failures: Stack traces and error messages
• Performance Metrics: Execution times and bottlenecks
• Remediation Data: Structured information for feedback loop

Expected Input/Output

Input Requirements

• Godot Project Path: Root directory of the Godot project
• Test Directory: Location of gdUnit4 test files (optional, defaults to tests/)

Output Deliverables

• Test Results: Complete pass/fail status with details
• Failure Reports: Structured error information for each failure
• JUnit XML: Parsed test report data
• Performance Data: Execution timing and resource usage

Usage Instructions

Basic Usage

python scripts/run_gdunit4_tests.py --project-path /path/to/godot/project

Advanced Usage

# With custom output directory
python scripts/run_gdunit4_tests.py --project-path /project --output-dir /reports

# With timeout and performance analysis
python scripts/run_gdunit4_tests.py --project-path /project --timeout 600 --profile

# With detailed failure analysis
python scripts/run_gdunit4_tests.py --project-path /project --analyze-failures

Headless Godot Execution

The skill uses the precise command for headless testing:

godot --headless --path [project_root] --script res://addons/gdUnit4/runtest.gd --report-junit=junit_report.xml

Command Components

• --headless: Disables graphics rendering for CI/CD
• --path: Sets project working directory
• --script: Executes gdUnit4 test runner
• --report-junit: Generates JUnit XML output for parsing

XML Parsing and Analysis

JUnit XML Structure

<testsuites name="gdUnit4" tests="15" failures="2" errors="0" time="1.234">
    <testsuite name="TestShipStats" tests="5" failures="1" errors="0" time="0.456">
        <testcase name="test_ship_stats_creation" classname="TestShipStats" time="0.023"/>
        <testcase name="test_ship_stats_validation" classname="TestShipStats" time="0.034">
            <failure message="ShipStats should extend Resource">
                TestShipStats.gd:15 - Expected Resource but got Node
                Stack trace...
            </failure>
        </testcase>
    </testsuite>
</testsuites>

Structured JSON Output

{
  "test_execution": {
    "status": "completed",
    "timestamp": "2025-01-15T10:30:00Z",
    "command": "godot --headless --path /project --script res://addons/gdUnit4/runtest.gd",
    "exit_code": 1,
    "execution_time_ms": 1234
  },
  "test_summary": {
    "total_tests": 15,
    "tests_passed": 13,
    "tests_failed": 2,
    "tests_errors": 0,
    "success_rate": 86.7
  },
  "failures": [
    {
      "test_name": "test_ship_stats_validation",
      "class_name": "TestShipStats",
      "failure_type": "assertion_failure",
      "message": "ShipStats should extend Resource",
      "details": "TestShipStats.gd:15 - Expected Resource but got Node",
      "stack_trace": "..."
    }
  ]
}

Error Handling

The skill provides structured JSON error reporting:

{
  "test_execution": {
    "status": "error",
    "error": {
      "type": "godot_execution_error",
      "message": "Failed to start Godot headless instance",
      "command": "godot --headless ...",
      "exit_code": 127,
      "stdout": "...",
      "stderr": "godot: command not found"
    },
    "recovery": {
      "automated_fix_possible": false,
      "suggestion": "Install Godot and ensure it's in PATH",
      "manual_steps": [
        "Install Godot 4.x",
        "Add Godot to system PATH",
        "Verify gdUnit4 addon is installed"
      ]
    }
  }
}

Error Types

• Godot Execution Errors: Missing Godot, invalid path
• Test Discovery Errors: No test files found
• XML Parsing Errors: Invalid JUnit report format
• Timeout Errors: Tests exceeded time limit
• Resource Errors: Missing test resources

Integration with Automated Remediation

Feedback Loop Integration

If this skill returns failures, you MUST append the structured JSON failure data to the ## Feedback section of the Markdown Contract. This enables:

1. Structured Communication: Machine-readable failure data
2. Automated Analysis: Pattern recognition across failures
3. Targeted Fixes: Specific file and line number references
4. Quality Tracking: Metrics on fix success rates

Example Feedback Addition

## Feedback

### Test Failures Detected
The QAEngineer identified 2 test failures requiring attention:

#### 1. ShipStats Class Inheritance Error
- **Test**: TestShipStats.test_ship_stats_validation
- **File**: scripts/ship_stats.gd:1
- **Issue**: Class extends Node instead of Resource
- **Suggested Fix**: Change `extends Node` to `extends Resource`
- **Automated Fix Possible**: Yes (confidence: 0.95)

#### 2. Weapon Data Type Mismatch
- **Test**: TestWeaponData.test_weapon_properties
- **File**: scripts/weapon_data.gd:23
- **Issue**: Property type mismatch
- **Suggested Fix**: Update property type declaration
- **Automated Fix Possible**: Yes (confidence: 0.80)

Quality Assurance

The skill ensures comprehensive validation:

• Test Discovery: Finds all gdUnit4 test files automatically
• Complete Execution: Runs full test suite with proper environment
• Accurate Parsing: Extracts all failure information from XML
• Structured Output: Returns machine-readable JSON for automation
• Error Recovery: Handles execution and parsing errors gracefully

This skill transforms the complex task of test validation into a reliable, automated process that provides the structured data necessary for the "Agentic Immune System" to self-correct and maintain code quality throughout the migration process.