Archive Logs by Type (Type-Aware Retention)

When archiving logs based on retention policy:

Step 1: Discover Archival Candidates

Invoke log-lister skill:

• Filter by log_type (if specified)
• Get all logs with metadata

Step 2: Load Retention Policies from Config

Read user's config file: .fractary/plugins/logs/config.json

• Load retention.default - fallback policy for unmatched paths
• Load retention.paths array - path-specific retention rules
• For each log, match against path patterns to find applicable policy

Example config structure:

{
  "retention": {
    "default": {
      "local_days": 30,
      "cloud_days": "forever",
      "priority": "medium",
      "auto_archive": true,
      "cleanup_after_archive": true
    },
    "paths": [
      {
        "pattern": "sessions/*",
        "log_type": "session",
        "local_days": 7,
        "cloud_days": "forever",
        "priority": "high",
        "auto_archive": true,
        "cleanup_after_archive": false,
        "retention_exceptions": {
          "keep_if_linked_to_open_issue": true,
          "keep_recent_n": 10
        },
        "archive_triggers": {
          "age_days": 7,
          "size_mb": null,
          "status": ["stopped", "error"]
        },
        "compression": {
          "enabled": true,
          "format": "gzip",
          "threshold_mb": 1
        }
      },
      {
        "pattern": "test/*",
        "log_type": "test",
        "local_days": 3,
        "cloud_days": 7,
        "priority": "low",
        "auto_archive": true,
        "cleanup_after_archive": true
      },
      {
        "pattern": "audit/*",
        "log_type": "audit",
        "local_days": 90,
        "cloud_days": "forever",
        "priority": "critical",
        "retention_exceptions": {
          "never_delete_security_incidents": true,
          "never_delete_compliance_audits": true
        }
      }
    ]
  }
}

Path matching algorithm:

1. For each log file, extract relative path from /logs/ directory
2. Test against each pattern in retention.paths array (in order)
3. First match wins - use that path's retention policy
4. If no match, use retention.default policy

Step 3: Calculate Retention Status

Execute scripts/check-retention-status.sh: For each log:

• Parse log date from frontmatter
• Calculate age (now - log.date)
• Check retention policy for log's type
• Determine status:
  • active: Within retention period
  • expiring_soon: < 3 days until expiry
  • expired: Past local_retention_days
  • protected: Retention exception applies

Step 4: Filter by Retention Exceptions

Check exceptions from retention-config.json:

// Session example
if (retention_exceptions.keep_if_linked_to_open_issue) {
  // Check if issue still open via GitHub API
  if (issue_is_open) {
    status = "protected"
  }
}

if (retention_exceptions.keep_recent_n) {
  // Keep N most recent logs regardless of age
  if (log_rank <= retention_exceptions.keep_recent_n) {
    status = "protected"
  }
}

// Deployment example
if (retention_exceptions.never_delete_production && log.environment === "production") {
  status = "protected"
}

// Audit example
if (retention_exceptions.never_delete_security_incidents && log.audit_type === "security") {
  status = "protected"
}

Step 5: Group Logs for Archival

Group expired logs by type:

• Count per type
• Calculate total size
• Estimate compression savings

Step 6: Compress Large Logs

Execute scripts/compress-logs.sh:

• For each log > 1MB:
  • Compress with gzip
  • Verify compressed size < original
  • Calculate compression ratio

Step 7: Upload to Cloud

Execute scripts/upload-to-cloud.sh:

• For each log (or compressed version):
  • Upload via fractary-file skill
  • Path: archive/logs/{year}/{month}/{log_type}/{filename}
  • Receive cloud URL
  • Verify upload successful

Step 8: Update Type-Aware Index

Execute scripts/update-archive-index.sh:

{
  "version": "2.0",
  "type_aware": true,
  "archives": [
    {
      "log_id": "session-550e8400",
      "log_type": "session",
      "issue_number": 123,
      "archived_at": "2025-11-23T10:00:00Z",
      "local_path": ".fractary/logs/session/session-550e8400.md",
      "cloud_url": "r2://logs/2025/11/session/session-550e8400.md.gz",
      "original_size_bytes": 125000,
      "compressed_size_bytes": 42000,
      "retention_policy": {
        "local_days": 7,
        "cloud_policy": "forever"
      },
      "delete_local_after": "2025-11-30T10:00:00Z"
    }
  ],
  "by_type": {
    "session": {"count": 12, "total_size_mb": 15.2},
    "test": {"count": 45, "total_size_mb": 8.7},
    "audit": {"count": 3, "total_size_mb": 2.1}
  }
}

Step 9: Clean Local Storage (Per Retention)

Execute scripts/cleanup-local.sh:

• For each archived log:
  • Check if past local retention period
  • Verify cloud backup exists
  • Delete local copy
  • Update index with deletion timestamp

Step 10: Copy Session Summaries to Docs (Optional)

If docs_integration.copy_summary_to_docs is enabled in config:

Execute scripts/copy-to-docs.sh:

./scripts/copy-to-docs.sh \
  --summary-path "$SUMMARY_PATH" \
  --docs-path "$DOCS_PATH" \
  --issue-number "$ISSUE_NUMBER" \
  --update-index "$UPDATE_INDEX"

This step:

• Copies session summary to docs/conversations/ directory
• Names file using pattern: {date}-{issue_number}-{slug}.md
• Creates directory if it doesn't exist
• Updates README.md index with new entry (if configured)
• Limits index to max_index_entries most recent

Step 11: Comment on Issues (Optional)

If archiving issue-related logs:

• Comment with archive summary and cloud URLs

Step 12: Output Summary

Report archival results grouped by type

Archive Issue Logs (Legacy - Type-Aware)

When archiving logs for completed issue:

Step 1: Collect Issue Logs

Execute scripts/collect-issue-logs.sh:

• Find all logs with matching issue_number
• Group by log_type (session, build, deployment, test, etc.)

Step 2: Archive Each Type

For each log type found:

• Load type's retention policy
• Archive according to type rules
• Use type-specific cloud path

Verify Archive

When verifying archived logs:

Step 1: Load Archive Index

Read .fractary/logs/.archive-index.json

Step 2: Verify Cloud Files

For each archived entry:

• Check cloud file exists via fractary-file
• Verify file integrity (checksum if available)
• Check retention policy compliance

Step 3: Report Status

Archive Verification Report
───────────────────────────────────────
Total archived: 60 logs across 5 types

By type:
  ✓ session: 12 logs (all verified)
  ✓ test: 45 logs (all verified)
  ⚠ build: 2 logs (1 missing in cloud)
  ✓ audit: 1 log (verified)

Issues:
  - build-2025-11-10-001.md.gz: Cloud file not found

Recommendation: Re-upload missing build log