Claude Code Plugins

Community-maintained marketplace

Feedback

extract-test-failures

@mvillmow/ProjectOdyssey
8
0

Extract and summarize test failures from logs. Use to quickly understand what tests failed and why.

Install Skill

1Download skill
2Enable skills in Claude

Open claude.ai/settings/capabilities and find the "Skills" section

3Upload to Claude

Click "Upload skill" and select the downloaded ZIP file

Note: Please verify skill by going through its instructions before using it.

SKILL.md

name extract-test-failures
description Extract and summarize test failures from logs. Use to quickly understand what tests failed and why.
category testing
mcp_fallback none

Extract and Summarize Test Failures

Parse test logs to extract failure information and create summary.

When to Use

  • CI test run failed, need to understand failures
  • Large test suite with many failures, need summary
  • Categorizing failures by type (assertion, runtime, timeout)
  • Creating PR review comments about failures
  • Tracking which tests are failing over time

Quick Reference

# Extract failed test names
grep "FAILED" test_output.log

# Get failure details with context
grep -A 10 "FAILED\|Error\|AssertionError" test_output.log

# Count failures by type
grep "FAILED\|AssertionError\|ValueError\|TypeError" test_output.log | sort | uniq -c

# Extract test summary line
tail -20 test_output.log | grep -E "passed|failed|error"

# Get specific failure info
grep -B 5 "AssertionError" test_output.log | head -50

Failure Extraction Workflow

  1. Collect log: Get test output from CI or local run
  2. Find failures: Extract FAILED markers and error types
  3. Group by type: Categorize assertion vs runtime vs timeout
  4. Extract details: Get error messages and stack traces
  5. Count totals: Summary of how many failures
  6. Identify patterns: Common failure causes
  7. Summarize: Create concise failure report

Failure Categories

Assertion Failures:

  • AssertionError - Expected value check failed
  • assert_equal() - Values don't match
  • assert_true() - Condition not true
  • Fix: Check test logic and expected values

Type/Attribute Errors:

  • AttributeError - Missing method/attribute
  • TypeError - Wrong argument type
  • ValueError - Invalid value for operation
  • Fix: Check code syntax and types

Runtime Errors:

  • IndexError - Out of bounds access
  • KeyError - Dictionary key not found
  • ZeroDivisionError - Division by zero
  • Fix: Check boundary conditions

Timeout/Hanging:

  • Test takes too long
  • Infinite loop or deadlock
  • Resource exhaustion
  • Fix: Optimize or add timeout

Output Format

Report failures with:

  1. Total Failures - Count and percentage of total tests
  2. Failure Summary - By type (assertion, runtime, timeout, other)
  3. Failed Tests - List of test names that failed
  4. Top Issues - Most common failure patterns
  5. Error Messages - Representative error snippets
  6. Recommendations - Which tests to focus on first

Error Handling

Problem Solution
No FAILED markers Check log format, may use different pattern
Truncated output Get full log from artifacts instead of view
Mixed output types Filter by log level or timestamp
Encoding issues Convert to UTF-8 first
Very large logs Split and process in chunks

References

  • See test-diff-analyzer for identifying flaky tests
  • See analyze-ci-failure-logs for CI-specific failures
  • See generate-fix-suggestions for automated fixes