Claude Code Plugins

Community-maintained marketplace

Feedback

learn-from-mistake

@cowwoc/styler
0
0

Investigate agent mistakes, perform root cause analysis, and update configurations to prevent recurrence

Install Skill

1Download skill
2Enable skills in Claude

Open claude.ai/settings/capabilities and find the "Skills" section

3Upload to Claude

Click "Upload skill" and select the downloaded ZIP file

Note: Please verify skill by going through its instructions before using it.

SKILL.md

name learn-from-mistake
description Investigate agent mistakes, perform root cause analysis, and update configurations to prevent recurrence
allowed-tools Read, Write, Edit, Bash, Grep, Glob

Learn From Mistake Skill

Purpose: Investigate agent mistakes, perform root cause analysis, update agent configurations, and TEST fixes by reproducing the original mistake to verify prevention.

When to Use:

  • After an agent makes a mistake that causes delays, rework, or violations
  • When a pattern of similar mistakes is observed across tasks
  • To proactively improve agent behavior based on lessons learned
  • When protocol violations occur that should have been caught

Skill Workflow

8-phase process:

  1. Mistake Identification - Gather context
  2. Conversation Analysis - Review logs
  3. Root Cause Analysis - Categorize and investigate cause
  4. Configuration Updates - Design prevention measures
  5. Implement Updates - Apply changes
  6. Validation - Review completeness
  7. Test by Reproduction - ⚠️ Attempt to reproduce mistake to verify prevention
  8. Documentation - Add inline comments and commit messages

Critical: Phase 7 requires actually reproducing the mistake (not mental simulation) to verify fixes work.

Phase 1: Mistake Identification

Input Requirements:

  • Agent name that made the mistake (e.g., architect, style, main)
  • Task name if applicable (e.g., implement-formatter-api)
  • Description of what went wrong
  • Conversation ID or timestamp range (optional - will search recent if not provided)

Questions to Ask User:

  1. Which agent made the mistake?
  2. What task were they working on?
  3. What specifically went wrong?
  4. Do you have the conversation ID? (Can skip - will search recent)

Phase 2: Conversation Analysis

Access Conversation Logs:

First, get the session ID from the SessionStart system reminder (provided by get-session-id skill):

✅ Session ID: 88194cb6-734b-498c-ab5d-ac7c773d8b34

Then access conversation logs:

# REQUIRED: Use session ID from system reminder
SESSION_ID="88194cb6-734b-498c-ab5d-ac7c773d8b34"

# Verify session ID is provided
if [ -z "$SESSION_ID" ]; then
  echo "ERROR: Session ID not available in context." >&2
  echo "Expected system reminder: '✅ Session ID: {uuid}'" >&2
  echo "Provided by get-session-id skill at SessionStart." >&2
  exit 1
fi

# Main session conversation
MAIN_CONVERSATION="/home/node/.config/projects/-workspace/${SESSION_ID}.jsonl"

# Verify conversation file exists
if [ ! -f "$MAIN_CONVERSATION" ]; then
  echo "ERROR: Conversation file not found: $MAIN_CONVERSATION" >&2
  exit 1
fi

# Agent sidechain conversations (if investigating agent-specific mistakes)
AGENT_CONVERSATIONS=$(ls -t /home/node/.config/projects/-workspace/agent-*.jsonl 2>/dev/null | head -10)

# Read and parse conversation for agent mentions
jq -r "select(.message.content | tostring | contains(\"$AGENT_NAME\"))" "$MAIN_CONVERSATION"

# For agent sidechain logs
for agent_log in $AGENT_CONVERSATIONS; do
  grep -l "$AGENT_NAME" "$agent_log" 2>/dev/null && echo "Found in: $agent_log"
done

What to Extract:

  1. Agent Invocation Context:

    • What prompt was given to the agent
    • What files/context were available
    • What state the task was in
    • Working directory specified
  2. Agent Actions:

    • Sequence of tool calls made
    • Files read, written, edited
    • Commands executed
    • Directories changed to
  3. Mistake Manifestation:

    • Error messages
    • Incorrect output
    • Protocol violations
    • Build failures
    • Rework needed
  4. Recovery Actions:

    • What was done to fix the mistake
    • How much rework was required
    • What additional context was needed

Phase 3: Root Cause Analysis

⚠️ MANDATORY: Identify Triggering Thought Before Creating Fixes

Before Phase 4, MUST:

  1. Access /home/node/.config/projects/-workspace/{session-id}.jsonl
  2. Find assistant message immediately before mistake
  3. Extract exact text/thought before wrong decision
  4. Document triggering quote

Example:

# Search for when file was created
grep -i "safety-analysis" conversation.jsonl | jq '.timestamp'
# Returns: 2025-11-02T04:31:37.673Z

# Get messages before that timestamp
jq 'select(.timestamp < "2025-11-02T04:31:37.673Z" and .timestamp > "2025-11-02T04:30:00.000Z")' conversation.jsonl

# Find triggering thought
jq 'select(.timestamp == "2025-11-02T04:31:10.998Z") | .message.content[].text' conversation.jsonl

# Result: "Given the complexity and length of fixing both skills completely,
#          let me create a summary document showing the critical fixes needed"

Root cause: Agent used "complexity and length" to justify creating summary instead of doing work (Token Usage Policy violation).

Without triggering thought, prevention will be ineffective.

Automated Detection:

#!/bin/bash
# Helper: Find triggering thought automatically

SESSION_ID="${1:-$(grep 'Session ID:' /tmp/session_context.txt | cut -d: -f2 | tr -d ' ')}"
MISTAKE_INDICATOR="${2:-}"  # e.g., "SAFETY-ANALYSIS.md", "BUILD FAILURE", "PROTOCOL VIOLATION"

if [[ -z "$SESSION_ID" || -z "$MISTAKE_INDICATOR" ]]; then
  echo "Usage: $0 <session-id> <mistake-indicator>" >&2
  echo "Example: $0 abc123 'SAFETY-ANALYSIS.md'" >&2
  exit 1
fi

CONVERSATION="/home/node/.config/projects/-workspace/${SESSION_ID}.jsonl"

if [[ ! -f "$CONVERSATION" ]]; then
  echo "ERROR: Conversation file not found: $CONVERSATION" >&2
  exit 1
fi

# Find timestamp when mistake occurred
MISTAKE_TIMESTAMP=$(jq -r --arg indicator "$MISTAKE_INDICATOR" \
  'select(.message.content | tostring | contains($indicator)) | .timestamp' \
  "$CONVERSATION" | head -1)

if [[ -z "$MISTAKE_TIMESTAMP" ]]; then
  echo "ERROR: Could not find mistake indicator '$MISTAKE_INDICATOR' in conversation" >&2
  exit 1
fi

echo "Mistake occurred at: $MISTAKE_TIMESTAMP"

# Find last assistant message before mistake (within 5 minutes)
FIVE_MINUTES_BEFORE=$(date -d "$MISTAKE_TIMESTAMP - 5 minutes" -Iseconds 2>/dev/null || \
                      date -v-5M -jf "%Y-%m-%dT%H:%M:%S" "${MISTAKE_TIMESTAMP%+*}" "+%Y-%m-%dT%H:%M:%S" 2>/dev/null)

TRIGGERING_THOUGHT=$(jq -r --arg before "$MISTAKE_TIMESTAMP" --arg after "$FIVE_MINUTES_BEFORE" \
  'select(.timestamp < $before and .timestamp > $after and .message.role == "assistant") |
   {timestamp, content: .message.content}' \
  "$CONVERSATION" | jq -s 'last')

if [[ -z "$TRIGGERING_THOUGHT" || "$TRIGGERING_THOUGHT" == "null" ]]; then
  echo "WARNING: Could not find assistant message before mistake" >&2
  exit 1
fi

echo ""
echo "TRIGGERING THOUGHT:"
echo "$TRIGGERING_THOUGHT" | jq -r '.content[] | select(.type == "text") | .text' | tail -20

# Also check for thinking blocks (often contain the decision reasoning)
echo ""
echo "ASSOCIATED THINKING (if any):"
echo "$TRIGGERING_THOUGHT" | jq -r '.content[] | select(.type == "thinking") | .thinking' | tail -20

Usage:

# Automatic detection
./find-triggering-thought.sh "$SESSION_ID" "PROTOCOL VIOLATION"

# Or inline
SESSION_ID="abc-123"
MISTAKE="wrong worktree"
jq -r --arg indicator "$MISTAKE" \
  'select(.message.content | tostring | contains($indicator))' \
  /home/node/.config/projects/-workspace/${SESSION_ID}.jsonl | \
  jq -r '.timestamp' | head -1

Categorize the Mistake:

A. Missing Information

  • Agent prompt lacked necessary context
  • Required files not mentioned
  • Assumptions not stated explicitly
  • Working directory not clear

B. Misunderstood Requirements

  • Ambiguous protocol documentation
  • Conflicting instructions
  • Unclear success criteria
  • No examples for this scenario

C. Tool Usage Errors

  • Wrong tool for the job
  • Incorrect parameters
  • Missing validation steps
  • Tool limitations not understood

D. Logic Errors

  • Incorrect algorithm
  • Missing edge cases
  • Faulty assumptions
  • Incomplete understanding of domain

E. Protocol Violations

  • Skipped required steps
  • Wrong state for action
  • Missing validations
  • Worktree confusion

F. Configuration Gaps

  • Agent config missing examples
  • No guidance for this scenario
  • Anti-patterns not documented
  • Missing checklists

Investigate:

  1. Available Information: Agent prompt, files accessed, missing context
  2. Correct Action: Expected sequence, needed information, missing checks
  3. Wrong Choice Reason: Documentation gaps, ambiguous instructions, misleading examples, insufficient validation

Phase 4: Configuration Updates

⚠️ CRITICAL: Active Enforcement vs Passive Policies

When policies exist but are violated, implement ACTIVE ENFORCEMENT (hooks), not just passive documentation.

Decision Tree:

  1. Policy exists AND was violated?

    • ✅ YES → Create enforcement hook (PreToolUse/PostToolUse)
    • ❌ NO (new issue) → Document policy first, then add hook if violations likely
  2. Policy enforceability:

    • ✅ Can be checked mechanically → MUST create hook
    • ❌ Requires judgment → Document with examples, consider detection hook

Why Active Enforcement Matters:

  • Passive policies (text in config) rely on agents reading and remembering
  • Active hooks (validation scripts) prevent violations automatically
  • If it violated once, it WILL violate again without enforcement

Examples:

CORRECT - Active Enforcement:

Issue: Agent created files in wrong worktree (policy exists in docs)
Fix: Create pre-write.sh hook that validates worktree before Write tool
Result: Future violations automatically blocked

WRONG - Passive Policy Only:

Issue: Agent created files in wrong worktree (policy exists in docs)
Fix: Add more warnings to documentation, bold the text
Result: Violation will recur because agent may not read/remember

Hook Creation Priority:

  1. HIGH: Protocol violations (state machine, worktree, approval gates)
  2. MEDIUM: Tool misuse (wrong parameters, missing validation)
  3. LOW: Style preferences (can be post-hoc validated)

Update Strategy by Category:

A. Missing Information → Update Agent Prompt Template

File: .claude/agents/{agent-name}.md

Add to "Required Context" or "Before You Begin":

  • Checklist to verify information
  • Required file reads
  • Questions for unclear context
  • Working directory verification
## Before You Begin - MANDATORY CHECKS

Before ANY implementation work:

- [ ] Verify working directory: `pwd`
- [ ] Confirm you're in: `/workspace/tasks/{task}/agents/{agent-name}/code/`
- [ ] Read task requirements: `/workspace/tasks/{task}/task.md`
- [ ] Check task state: `cat /workspace/tasks/{task}/task.json`
- [ ] Verify dependencies available

If ANY check fails: STOP and report the issue.

B. Misunderstood Requirements → Clarify Documentation

File: docs/project/{relevant-protocol}.md

Add:

  • Explicit requirements
  • ✅/❌ examples
  • Edge cases
  • Anti-pattern warnings
  • Decision trees
## Working Directory Requirements

⚠️ **CRITICAL**: Sub-agents MUST work in their agent worktree

**Your worktree**: `/workspace/tasks/{task}/agents/{agent-name}/code/`
**NOT the task worktree**: `/workspace/tasks/{task}/code/` ← WRONG

✅ **CORRECT**:
\```bash
cd /workspace/tasks/implement-api/agents/architect/code
Write: src/main/java/MyClass.java
\```

❌ **WRONG** (Protocol Violation):
\```bash
cd /workspace/tasks/implement-api/code
Write: src/main/java/MyClass.java  # Creates in task worktree!
\```

C. Tool Usage Errors → Add Tool Guidance

File: .claude/agents/{agent-name}.md

Add "Tool Usage Patterns":

  • When to use each tool
  • Common pitfalls
  • Validation steps
  • Example sequences
  • Error recovery
## Tool Usage Patterns

### Write Tool
**When**: Creating new files
**Before**: Verify directory with `pwd`
**After**: Verify file created with `ls -la {file}`
**Validation**: File must be in agent worktree

### Edit Tool
**When**: Modifying existing files
**Before**: Read file first to see exact content
**After**: Read file again to verify changes
**Pitfall**: Don't include line numbers in old_string

D. Logic Errors → Add Examples and Tests

File: .claude/agents/{agent-name}.md

Add:

  • Worked examples
  • Edge cases
  • Validation checklist
  • Self-test questions
  • Common pitfalls
## Common Scenarios

### Scenario: Creating New Maven Module

**Context**: Task requires new Maven module

**Steps**:
1. Create directory: `mkdir -p {module}/src/main/java`
2. Create POM: Use template from existing module
3. Add to parent POM: `<module>{module}</module>`
4. Create module-info.java: Define exports
5. Verify: `mvn compile -pl {module}`

**Edge Cases**:
- Module dependencies on other modules
- JPMS transitive requirements
- Test module descriptor

**Self-Test**:
- [ ] Did I update parent POM?
- [ ] Does module compile?
- [ ] Are exports correct?

E. Protocol Violations → Add Validation Hooks

File: .claude/hooks/pre-{action}.sh

Create:

  • Pre-action validation
  • Prerequisite checks
  • State verification
  • Helpful error messages
#!/bin/bash
# .claude/hooks/pre-write.sh
# Validates worktree before Write tool use

set -euo pipefail

# Extract agent name from context
AGENT_NAME=${CLAUDE_AGENT_NAME:-main}

# For sub-agents, verify working in agent worktree
if [[ "$AGENT_NAME" != "main" ]]; then
  EXPECTED_PATH="/workspace/tasks/.*/agents/$AGENT_NAME/code"
  if [[ ! "$PWD" =~ $EXPECTED_PATH ]]; then
    echo "❌ ERROR: Agent $AGENT_NAME in wrong directory" >&2
    echo "Current: $PWD" >&2
    echo "Expected: /workspace/tasks/{task}/agents/$AGENT_NAME/code/" >&2
    echo "" >&2
    echo "SOLUTION: cd to your agent worktree first" >&2
    exit 1
  fi
fi

F. Configuration Gaps → Enhance Agent Config

File: .claude/agents/{agent-name}.md

Add Sections:

  • "Common Scenarios" with examples
  • "Anti-Patterns to Avoid"
  • "Decision Trees" for complex choices
  • "Verification Steps" before completion
  • "Self-Validation Checklist"

Example:

## Anti-Patterns to Avoid

### ❌ Creating files without directory verification
**Problem**: Files created in wrong location
**Solution**: Always `pwd` before Write/Edit

### ❌ Skipping build verification
**Problem**: Broken code merged to task branch
**Solution**: Run `mvn compile` before merge

### ❌ Assuming file locations
**Problem**: Files not found or created in wrong place
**Solution**: Read files to confirm paths

## Self-Validation Checklist

Before reporting completion:
- [ ] All files in correct worktree
- [ ] Code compiles: `mvn compile`
- [ ] Tests pass: `mvn test`
- [ ] No protocol violations
- [ ] Status.json updated

Phase 5: Implement Updates

⚠️ CRITICAL: Create Rollback Point Before Changes

# Create backup branch with timestamp
BACKUP_BRANCH="learn-from-mistakes-backup-$(date +%s)"
git branch "$BACKUP_BRANCH" HEAD
echo "Created rollback point: $BACKUP_BRANCH"

# Store backup branch name for potential rollback
echo "$BACKUP_BRANCH" > /tmp/learn-from-mistakes-rollback.txt

Rollback Procedure (if Phase 7 testing fails):

# Read backup branch name
BACKUP_BRANCH=$(cat /tmp/learn-from-mistakes-rollback.txt)

# Reset to backup
git reset --hard "$BACKUP_BRANCH"

# Clean up
git branch -D "$BACKUP_BRANCH"
rm /tmp/learn-from-mistakes-rollback.txt

For Each Update:

  1. Read current config: .claude/agents/{agent-name}.md or docs/project/{protocol-file}.md

  2. Locate relevant section, check conflicts

  3. Draft clear guidance with ✅/❌ examples, validation steps, consistent terminology

  4. Apply with Edit tool

  5. Hook Registration (if creating hook):

    # After creating hook file: .claude/hooks/{hook-name}.sh
    
    # Step 1: Make executable
    chmod +x .claude/hooks/{hook-name}.sh
    
    # Step 2: Determine hook trigger event
    # - SessionStart: Runs at session start
    # - UserPromptSubmit: Runs when user submits prompt
    # - PreToolUse: Runs before tool execution
    # - PostToolUse: Runs after tool execution
    # - PreCompact: Runs before context compaction
    
    # Step 3: Auto-register in settings.json
    HOOK_PATH="/workspace/.claude/hooks/{hook-name}.sh"
    TRIGGER_EVENT="PreToolUse"  # Or appropriate trigger
    
    # Read current settings
    Read: .claude/settings.json
    
    # Add hook registration using jq
    jq --arg hook "$HOOK_PATH" --arg trigger "$TRIGGER_EVENT" \
      '.[$trigger] += [{"hooks": [{"type": "command", "command": $hook}]}]' \
      .claude/settings.json > .claude/settings.json.tmp
    
    # Apply update
    mv .claude/settings.json.tmp .claude/settings.json
    
    # Verify registration
    grep -A3 "{hook-name}" .claude/settings.json
    
    # CRITICAL: Notify user to restart Claude Code
    echo "⚠️ IMPORTANT: settings.json was modified. Please restart Claude Code for hook to take effect." >&2
    

    ⚠️ CRITICAL: When settings.json is modified, you MUST notify the user:

    ⚠️ IMPORTANT: I've updated .claude/settings.json to register the new hook.
    Please restart Claude Code for the hook to take effect.
    

    Hook Registration by Pattern:

    • Validation hooks (pre-write.sh, pre-edit.sh): PreToolUse with tool matcher
    • Detection hooks (detect-*.sh): PostToolUse or UserPromptSubmit
    • Enforcement hooks (enforce-*.sh): PreToolUse with matcher
    • Setup hooks: SessionStart
  6. Verify: Read updated file, confirm clarity, check conflicts, verify hook registration

Phase 6: Validation

Checklist:

  • Addresses root cause
  • Concrete examples (✅/❌)
  • Clear validation steps
  • Explicit anti-patterns
  • Self-test questions
  • No conflicts with other configs
  • Consistent terminology
  • Matches existing format

Phase 7: Test by Reproduction

⚠️ CRITICAL: Verify prevention by reproducing mistake

Workflow:

  1. Verify Updates:

    Read: .claude/agents/{agent-name}.md  # Confirm changes
    Read: .claude/hooks/{hook-name}.sh    # Confirm hook
    
  2. Prepare Hook (if created):

    chmod +x .claude/hooks/{hook-name}.sh
    grep -A5 "{hook-name}" /workspace/.claude/settings.json  # Verify registration
    
  3. Reproduce Mistake: Execute same action sequence that caused original mistake

    • Wrong worktree → Try creating file in wrong location
    • Missing validation → Try skipping validation step
    • Tool misuse → Try using tool incorrectly
    • Protocol violation → Try violating protocol

    Expected: Hook blocks or guidance prevents

  4. Verify Prevention:

    • Hooks: Test blocks incorrect action with clear error
    • Documentation: Read config as agent, verify guidance prevents mistake
  5. Interpret Results:

    • ✅ SUCCESS: Hook blocks OR guidance directs correctly, error message clear
    • ❌ FAILURE: Mistake reproducible, hook doesn't trigger, guidance unclear

    On Failure: Return to Phase 4, refine

6a. Automated Test Suite (Optional but Recommended):

Create reusable automated tests for hooks to enable regression testing:

#!/bin/bash
# .claude/hooks/tests/test-{hook-name}.sh
# Automated test suite for {hook-name}.sh

set -euo pipefail

HOOK_PATH="/workspace/.claude/hooks/{hook-name}.sh"
TEST_RESULTS="/tmp/hook-test-results.txt"

echo "Testing: ${HOOK_PATH##*/}" > "$TEST_RESULTS"

# Test 1: Hook blocks invalid scenario
test_blocks_invalid() {
  echo "Test 1: Hook blocks invalid scenario..." >> "$TEST_RESULTS"

  # Set up invalid scenario (e.g., wrong worktree)
  cd /workspace/invalid/location 2>/dev/null || mkdir -p /workspace/invalid/location && cd /workspace/invalid/location

  # Simulate hook execution with invalid context
  MOCK_CONTEXT='{"tool": {"name": "Write"}, "actor": "sub-agent"}'
  echo "$MOCK_CONTEXT" | bash "$HOOK_PATH" 2>&1 | grep -q "ERROR"

  if [[ $? -eq 0 ]]; then
    echo "  ✅ PASS: Hook correctly blocked invalid scenario" >> "$TEST_RESULTS"
    return 0
  else
    echo "  ❌ FAIL: Hook did not block invalid scenario" >> "$TEST_RESULTS"
    return 1
  fi
}

# Test 2: Hook allows valid scenario
test_allows_valid() {
  echo "Test 2: Hook allows valid scenario..." >> "$TEST_RESULTS"

  # Set up valid scenario
  cd /workspace/tasks/test/agents/architect/code 2>/dev/null || {
    mkdir -p /workspace/tasks/test/agents/architect/code
    cd /workspace/tasks/test/agents/architect/code
  }

  # Simulate hook execution with valid context
  MOCK_CONTEXT='{"tool": {"name": "Write"}, "actor": "architect"}'
  echo "$MOCK_CONTEXT" | bash "$HOOK_PATH" 2>&1 | grep -q "ERROR"

  if [[ $? -ne 0 ]]; then
    echo "  ✅ PASS: Hook correctly allowed valid scenario" >> "$TEST_RESULTS"
    return 0
  else
    echo "  ❌ FAIL: Hook incorrectly blocked valid scenario (false positive)" >> "$TEST_RESULTS"
    return 1
  fi
}

# Test 3: Error message quality
test_error_message_quality() {
  echo "Test 3: Error message clarity..." >> "$TEST_RESULTS"

  cd /workspace/invalid/location
  MOCK_CONTEXT='{"tool": {"name": "Write"}, "actor": "sub-agent"}'
  ERROR_MSG=$(echo "$MOCK_CONTEXT" | bash "$HOOK_PATH" 2>&1)

  # Check error message contains key elements
  if echo "$ERROR_MSG" | grep -q "ERROR" && \
     echo "$ERROR_MSG" | grep -q "Expected:" && \
     echo "$ERROR_MSG" | grep -q "Current:"; then
    echo "  ✅ PASS: Error message is clear and actionable" >> "$TEST_RESULTS"
    return 0
  else
    echo "  ❌ FAIL: Error message lacks clarity" >> "$TEST_RESULTS"
    echo "  Message: $ERROR_MSG" >> "$TEST_RESULTS"
    return 1
  fi
}

# Run all tests
FAILED=0
test_blocks_invalid || FAILED=$((FAILED + 1))
test_allows_valid || FAILED=$((FAILED + 1))
test_error_message_quality || FAILED=$((FAILED + 1))

# Summary
echo "" >> "$TEST_RESULTS"
if [[ $FAILED -eq 0 ]]; then
  echo "✅ All tests passed" >> "$TEST_RESULTS"
  cat "$TEST_RESULTS"
  exit 0
else
  echo "❌ $FAILED test(s) failed" >> "$TEST_RESULTS"
  cat "$TEST_RESULTS"
  exit 1
fi

Test Suite Benefits:

  • Regression testing: Verify fixes still work after code changes
  • Edge case coverage: Systematically test boundary conditions
  • False positive detection: Ensure legitimate use cases aren't blocked
  • Documentation: Tests serve as executable examples of hook behavior

Running Tests:

# Run specific hook test
bash .claude/hooks/tests/test-pre-write.sh

# Run all hook tests
for test in .claude/hooks/tests/test-*.sh; do
  echo "Running $test..."
  bash "$test" || echo "FAILED: $test"
done
  1. Iteration (if test fails):

    • Diagnose: Hook too narrow? Guidance unclear? Edge case?
    • Refine: Broaden patterns, add ⚠️ CRITICAL markers, more examples
    • Re-test until success
  2. Edge Cases: Test variations, different agents, legitimate use cases (no false positives)

Testing Checklist:

  • All configuration updates applied and verified
  • New hooks are executable and registered
  • Attempted to reproduce original mistake
  • Prevention mechanism activated correctly
  • Error messages are clear and actionable
  • Legitimate use cases not blocked (no false positives)
  • Edge cases tested
  • If test failed: Iterated and refined until successful

Test Results in Commit Message:

Add worktree validation to pre-write hook

**Testing**: Attempted file in wrong worktree - hook blocked with clear error

**Verified**: Fix prevents recurrence

Phase 8: Documentation

⚠️ NO RETROSPECTIVE DOCUMENTS (per CLAUDE.md policy)

Document via:

  1. Inline Comments: Pattern evolution, context, history
  2. Git Commits: What fixed, why prevents recurrence, original mistake context
  3. Code Comments: Rationale, alternatives, edge cases

Prohibited:

  • ❌ lessons-learned.md
  • ❌ Standalone retrospectives
  • ❌ Development process chronicles

Implementation Example

Mistake: Agent created files in task worktree instead of agent worktree (protocol violation)

Root Cause: Prompt mentioned "working directory" without emphasizing agent worktree distinction

Updates:

  1. Agent Config (.claude/agents/architect.md):

    ## ⚠️ CRITICAL: Working Directory
    
    **YOU MUST WORK IN YOUR AGENT WORKTREE**
    
    **Your worktree**: `/workspace/tasks/{task-name}/agents/architect/code/`
    **NOT the task worktree**: `/workspace/tasks/{task-name}/code/` ← PROTOCOL VIOLATION
    
    **Before ANY Write/Edit tool use**:
    1. Verify current directory: `pwd`
    2. Confirm you're in `/workspace/tasks/{task}/agents/architect/code/`
    3. If not, this is a CRITICAL ERROR - stop and report
    
    **Example**:
    ```bash
    # ✅ CORRECT
    cd /workspace/tasks/implement-api/agents/architect/code
    Write: src/main/java/MyClass.java
    
    # ❌ WRONG - PROTOCOL VIOLATION
    cd /workspace/tasks/implement-api/code
    Write: src/main/java/MyClass.java  # Creates in task worktree!
    
    
    
  2. Validation Hook (.claude/hooks/pre-write.sh):

    #!/bin/bash
    # Pre-Write hook: Verify agent is in correct worktree
    
    set -euo pipefail
    trap 'echo "ERROR in pre-write.sh at line $LINENO" >&2; exit 1' ERR
    
    AGENT_NAME=${CLAUDE_AGENT_NAME:-main}
    
    if [[ "$AGENT_NAME" != "main" ]]; then
      EXPECTED_PATH="/workspace/tasks/.*/agents/$AGENT_NAME/code"
      if [[ ! "$PWD" =~ $EXPECTED_PATH ]]; then
        echo "❌ ERROR: Agent $AGENT_NAME in wrong directory" >&2
        echo "Current: $PWD" >&2
        echo "Expected: /workspace/tasks/{task}/agents/$AGENT_NAME/code/" >&2
        exit 1
      fi
    fi
    
  3. Documentation (docs/project/task-protocol-agents.md):

    ## ⚠️ CRITICAL: Agent Worktree Isolation
    
    **Each agent MUST work in their own worktree**:
    - Main agent: `/workspace/tasks/{task}/code/`
    - Sub-agent: `/workspace/tasks/{task}/agents/{agent-name}/code/`
    
    **Common Mistake**: Creating files in task worktree instead of agent worktree
    
    **Prevention**:
    1. Always `cd` to your agent worktree first
    2. Verify with `pwd` before Write/Edit
    3. Check path includes `agents/{agent-name}/code/`
    
    **Verification**: Pre-write hook blocks if in wrong directory
    
  4. Inline Comments in Hook (.claude/hooks/pre-write.sh):

    #!/bin/bash
    # Pre-Write hook: Verify agent is in correct worktree
    #
    # ADDED: 2025-10-30 after architect created files in task worktree
    # instead of agent worktree during implement-formatter-api task.
    # PREVENTS: Protocol violations from wrong working directory
    
    set -euo pipefail
    
    AGENT_NAME=${CLAUDE_AGENT_NAME:-main}
    
    # For sub-agents, verify working in agent worktree
    if [[ "$AGENT_NAME" != "main" ]]; then
      EXPECTED_PATH="/workspace/tasks/.*/agents/$AGENT_NAME/code"
      if [[ ! "$PWD" =~ $EXPECTED_PATH ]]; then
        echo "❌ ERROR: Agent $AGENT_NAME in wrong directory" >&2
        echo "Current: $PWD" >&2
        echo "Expected: /workspace/tasks/{task}/agents/$AGENT_NAME/code/" >&2
        exit 1
      fi
    fi
    
  5. Git Commit Message:

    Add worktree validation to pre-write hook
    
    **Fix Applied**:
    - Updated: `.claude/agents/architect.md` (added critical warning)
    - Created: `.claude/hooks/pre-write.sh` (worktree validation)
    - Enhanced: `docs/project/task-protocol-agents.md` (common mistakes section)
    
    **Prevention**:
    - Pre-write hook blocks creation in wrong worktree
    - Agent config emphasizes worktree requirement
    - Examples show correct vs wrong patterns
    
    **Verification**:
    Hook tested by attempting write in wrong directory - correctly blocked
    

Success Criteria

Skill execution is successful when:

  1. Root cause identified: Clear understanding of why mistake occurred
  2. Updates applied: Agent config/documentation enhanced with specific guidance
  3. Fixes tested by reproduction: Attempted to reproduce original mistake after applying fixes
  4. Prevention verified: Reproduction test confirms mistake is now blocked/prevented
  5. Documentation added: Inline comments in hooks/configs + git commit message with test results
  6. No side effects: Updates don't break existing functionality or create false positives
  7. Examples included: Concrete ✅/❌ examples added
  8. Validation added: Hooks or checklists created where appropriate
  9. No retrospective docs: Did NOT create standalone lessons-learned.md or similar

Usage

Interactive Mode

/learn-from-mistake

Then answer the prompts:
- Agent: architect
- Task: implement-formatter-api
- Issue: Created files in wrong worktree

Direct Invocation

Learn from the mistake where architect created source files in the task worktree instead of their agent worktree during implement-formatter-api task. This caused a protocol violation and required rework.

Via Skill Tool

Skill: "learn-from-mistake"

Output Files

After execution, expect these files to be created/modified:

  1. Agent Configuration: .claude/agents/{agent-name}.md (updated with examples)
  2. Protocol Documentation: docs/project/{relevant-file}.md (enhanced if protocol-related)
  3. Validation Hooks: .claude/hooks/pre-{action}.sh (created/updated with inline comments)
  4. Git Commit: Detailed commit message documenting the fix and rationale

NO retrospective documents created - all documentation is inline or in git commits.

Related Skills

get-session-id: Provides session ID automatically at SessionStart via hook

  • Use session ID from system reminder: ✅ Session ID: {uuid}
  • Required for accessing conversation logs in Phase 2

read-conversation-history: Access raw conversation logs for analysis

  • Uses session ID to locate: /home/node/.config/projects/-workspace/{session-id}.jsonl
  • Provides agent sidechain logs: /home/node/.config/projects/-workspace/agent-*.jsonl

Integration with Audit

This skill complements the audit system:

  • Audit identifies → Skill fixes
  • Audit measures → Skill prevents
  • Audit finds patterns → Skill updates configs

Continuous Improvement Loop:

Execute Task → Audit → Learn → Update → Improved Execution

Metrics Tracking for Prevention Effectiveness

Purpose: Track mistake recurrence to measure prevention effectiveness and identify gaps.

Metrics Database: /tmp/mistake-prevention-metrics.json

Schema

{
  "mistake_types": {
    "build_failure": {
      "last_occurrence": "2025-11-03T10:00:00Z",
      "prevention_applied": "2025-11-02T15:30:00Z",
      "prevention_method": "hook",
      "hook_path": ".claude/hooks/validate-build.sh",
      "recurrence_count_after_fix": 0,
      "total_occurrences_before_fix": 3,
      "effectiveness": "100%",
      "time_to_fix_avg_before": "45min",
      "time_to_fix_avg_after": "0min"
    },
    "wrong_worktree": {
      "last_occurrence": "2025-11-01T14:22:00Z",
      "prevention_applied": "2025-10-30T09:15:00Z",
      "prevention_method": "hook",
      "hook_path": ".claude/hooks/pre-write.sh",
      "recurrence_count_after_fix": 0,
      "total_occurrences_before_fix": 5,
      "effectiveness": "100%",
      "time_to_fix_avg_before": "30min",
      "time_to_fix_avg_after": "0min"
    }
  },
  "summary": {
    "total_mistake_types": 2,
    "total_preventions_applied": 2,
    "average_effectiveness": "100%",
    "total_time_saved": "3.75hr"
  }
}

Recording Metrics

During Phase 8 (after successful prevention verification):

#!/bin/bash
# Record prevention metrics

METRICS_FILE="/tmp/mistake-prevention-metrics.json"
MISTAKE_TYPE="wrong_worktree"  # From Phase 1
PREVENTION_METHOD="hook"       # hook, documentation, or example
PREVENTION_PATH=".claude/hooks/pre-write.sh"
OCCURRENCES_BEFORE=5           # From conversation analysis
AVG_TIME_BEFORE="30min"

# Initialize metrics file if doesn't exist
if [[ ! -f "$METRICS_FILE" ]]; then
  echo '{"mistake_types": {}, "summary": {}}' > "$METRICS_FILE"
fi

# Record prevention
jq --arg type "$MISTAKE_TYPE" \
   --arg applied "$(date -Iseconds)" \
   --arg method "$PREVENTION_METHOD" \
   --arg path "$PREVENTION_PATH" \
   --arg count "$OCCURRENCES_BEFORE" \
   --arg time "$AVG_TIME_BEFORE" \
   '.mistake_types[$type] = {
     last_occurrence: $applied,
     prevention_applied: $applied,
     prevention_method: $method,
     hook_path: $path,
     recurrence_count_after_fix: 0,
     total_occurrences_before_fix: ($count | tonumber),
     effectiveness: "100%",
     time_to_fix_avg_before: $time,
     time_to_fix_avg_after: "0min"
   }' "$METRICS_FILE" > "${METRICS_FILE}.tmp" && mv "${METRICS_FILE}.tmp" "$METRICS_FILE"

Updating Metrics on Recurrence

If mistake recurs (detected by auto-learn-from-mistakes hook):

#!/bin/bash
# Update metrics when mistake recurs

METRICS_FILE="/tmp/mistake-prevention-metrics.json"
MISTAKE_TYPE="wrong_worktree"
CURRENT_TIME=$(date -Iseconds)

# Increment recurrence count
jq --arg type "$MISTAKE_TYPE" \
   --arg time "$CURRENT_TIME" \
   '.mistake_types[$type].recurrence_count_after_fix += 1 |
    .mistake_types[$type].last_occurrence = $time |
    .mistake_types[$type].effectiveness =
      (100 - (.mistake_types[$type].recurrence_count_after_fix /
              .mistake_types[$type].total_occurrences_before_fix * 100) |
       tostring + "%")' \
   "$METRICS_FILE" > "${METRICS_FILE}.tmp" && mv "${METRICS_FILE}.tmp" "$METRICS_FILE"

Viewing Metrics

Show all metrics:

jq '.' /tmp/mistake-prevention-metrics.json

Show effectiveness summary:

jq '.mistake_types | to_entries | map({
  type: .key,
  effectiveness: .value.effectiveness,
  recurrences: .value.recurrence_count_after_fix,
  prevention: .value.prevention_method
}) | sort_by(.effectiveness)' /tmp/mistake-prevention-metrics.json

Find ineffective preventions (recurrence rate > 20%):

jq '.mistake_types | to_entries |
    map(select(.value.recurrence_count_after_fix > 0)) |
    map({
      type: .key,
      effectiveness: .value.effectiveness,
      recurrences: .value.recurrence_count_after_fix,
      hook: .value.hook_path
    })' /tmp/mistake-prevention-metrics.json

Integration with auto-learn-from-mistakes Hook

Automatic Detection: The .claude/hooks/auto-learn-from-mistakes.sh hook runs on PostToolUse for ALL tools and automatically detects common mistake patterns, recommending invocation of this skill.

Currently Detected Patterns (as of 2025-11-05):

  1. Pattern 1: Build failures (CRITICAL)

    • Triggers: "BUILD FAILURE", "COMPILATION ERROR", "compilation failure"
    • Context: Maven/Gradle build output with errors
  2. Pattern 2: Test failures (CRITICAL)

    • Triggers: "Tests run:.*Failures: [1-9]", "test.*failed"
    • Context: Test execution results with failures
  3. Pattern 3: Protocol violations (CRITICAL)

    • Triggers: "PROTOCOL VIOLATION", "🚨.*VIOLATION"
    • Context: Hook-detected protocol breaches
  4. Pattern 4: Merge conflicts (HIGH)

    • Triggers: "CONFLICT", "merge conflict"
    • Context: Git merge operations
  5. Pattern 5: Edit tool failures (MEDIUM)

    • Triggers: "String to replace not found", "old_string not found"
    • Context: Edit tool string matching failures
  6. Pattern 6: Skill step failures (HIGH) - Added 2025-11-05

    • Triggers: Tool is "Skill" AND (\bERROR\b, \bFAILED\b, "failed to", "step.(failed|failure)", "operation.(failed|failure)", "could not", "unable to")
    • Context: Skill execution encountering errors during steps
    • Example: Git-squash skill step failing, git-rebase operation error
    • False Positive Prevention: Uses word boundaries to avoid matching "finished"
  7. Pattern 7: Git operation failures (HIGH) - Added 2025-11-05

    • Triggers: "fatal:", "error:", "git.*failed", "rebase.*failed", "merge.*failed"
    • Context: Git command failures
    • Example: Rebase conflicts, merge failures, invalid git operations

Rate Limiting: Hook enforces 5-minute cooldown between reminders to prevent spam.

Hook Output: When mistake detected, outputs recommendation to stderr:

📚 MISTAKE DETECTED: skill_step_failure

A significant mistake was detected in the Skill tool result.

**Recommendation**: Invoke the learn-from-mistakes skill:

Skill: learn-from-mistakes

Context: Detected skill_step_failure during Skill execution.

Metrics Integration: Hook can record metrics to track prevention effectiveness (see Metrics section above).

Update .claude/hooks/auto-learn-from-mistakes.sh to record metrics:

# After detecting mistake, record to metrics
METRICS_FILE="/tmp/mistake-prevention-metrics.json"

# Check if prevention exists for this mistake type
if jq -e --arg type "$MISTAKE_TYPE" '.mistake_types[$type]' "$METRICS_FILE" >/dev/null 2>&1; then
  # Prevention exists but mistake recurred - update metrics
  echo "⚠️ RECURRENCE: Prevention exists for $MISTAKE_TYPE but mistake occurred again" >&2

  # Increment recurrence counter (shown above)
  # ...
else
  # First occurrence - will be handled by learn-from-mistakes skill
  echo "📊 NEW MISTAKE TYPE: $MISTAKE_TYPE (no prevention yet)" >&2
fi

Metrics-Driven Improvement

Review metrics quarterly to identify:

  1. High-impact preventions: Mistakes eliminated completely
  2. Ineffective preventions: High recurrence rate (>20%)
  3. Common mistake patterns: Types with many occurrences
  4. Time savings: Accumulated time saved by preventing rework

Action items based on metrics:

  • Recurrence >20%: Re-invoke learn-from-mistakes to strengthen prevention
  • Zero recurrence: Document success pattern for reuse
  • High occurrence count: Prioritize systemic improvements

When to Use

Use for:

  • Protocol violations requiring rework
  • Repeated mistakes across tasks
  • Documentation gaps
  • Systematic tool usage errors
  • Configuration ambiguities

Skip for:

  • One-time errors unlikely to recur
  • Well-documented issues
  • No clear systemic fix
  • Minor delays (<10 min) without pattern
  • User errors

Best Practices

  • Focus on systemic improvements, not one-off fixes
  • Prioritize high-impact mistakes (violations, significant delays)
  • Use specific, actionable examples from actual mistakes
  • Verify no conflicts with existing guidance
  • Test hooks to ensure correctness
  • Track metrics for effectiveness