Testing Patterns and Infrastructure

Test Pyramid

• E2E: Critical user workflows against real services (Testcontainers)
• Integration: Complete workflows with mocked externals (Git, HTTP, filesystem)
• Unit: Edge cases integration cannot reach

Directory Structure

tests/
  Recyclarr.EndToEndTests/
  Recyclarr.Core.Tests/IntegrationTests/
  Recyclarr.Cli.Tests/IntegrationTests/
  Recyclarr.TestLibrary/
  Recyclarr.Core.TestLibrary/

Naming

• Classes: {Component}Test or {Component}IntegrationTest
• Methods: Underscore-separated behavior (Load_many_iterations_of_config)
• Pattern: internal sealed class

Integration Test Setup

internal sealed class MyFeatureIntegrationTest : CliIntegrationFixture
{
    protected override void RegisterStubsAndMocks(ContainerBuilder builder)
    {
        // Register custom mocks here
    }
}

Mock externals only: Git (LibGit2Sharp), HTTP APIs, filesystem (MockFileSystem).

AutoFixture Attributes

• [AutoMockData]: Basic DI with mocks
• [InlineAutoMockData(params)]: Parameterized tests
• [Frozen] or [Frozen(Matching.ImplementedInterfaces)]: Shared mock instances
• [CustomizeWith(typeof(T))]: Custom configuration
• [AutoMockData(typeof(TestClass), nameof(Method))]: DI container integration

NSubstitute Patterns

dependency.Method().Returns(value);
dependency.Property.ReturnsNull();
dependency.Method(default!).ReturnsForAnyArgs(value);
dependency.Method().Returns([item1, item2]);
mock.Received().Method(arguments);
Verify.That<T>(x => x.Property.Should().Be(expected));

AwesomeAssertions

Preferred:

result.Should().BeEquivalentTo(expected);
result.Select(x => x.Property).Should().BeEquivalentTo(expected);
act.Should().Throw<ExceptionType>().WithMessage("pattern");
collection.Should().HaveCount(n).And.Contain(item);
dict.Should().ContainKey(key).WhoseValue.Should().Be(expected);

Anti-patterns:

• dict!["key"]! - use ContainKey().WhoseValue instead
• HaveCount() + BeEquivalentTo() - redundant; equivalence checks count
• Multiple assertions instead of .And chaining

Utilities

• IntegrationTestFixture: Core library integration tests
• CliIntegrationFixture: CLI integration with composition root
• Verify.That<T>(): NSubstitute matcher with assertions
• TestableLogger: Capture log messages
• NUnitAnsiConsole: Console output verification
• MockFileSystem: Filesystem testing (avoid absolute paths)
• Factory classes: NewCf, NewConfig, NewQualitySize

Filesystem Paths

Avoid absolute paths in MockFileSystem (platform-incompatible):

// Good
Fs.CurrentDirectory().SubDirectory("a", "b").File("c.json")

// Bad
"/absolute/path/file.json"

Debugging Test Failures

Gather evidence before changing code. Avoid guess-and-check cycles.

1. Read assertion output carefully - Diff output often reveals the issue immediately
2. Add adhoc logs - Trace execution in tests or production code; remove when done
3. Compare with passing tests - Diff similar working tests to spot differences
4. Add intermediate assertions - Verify state at each step to pinpoint divergence
5. Simplify to minimal reproduction - Strip test down, add back until failure
6. Write adhoc granular tests - Isolate suspected areas; remove when done
7. Check test isolation - Run alone (--filter) vs. suite to detect state leakage

Test Framing

Tests serve as documentation. Choose framing based on what the test documents:

• Positive tests (expected behavior): Lead with what SHOULD happen, then verify absence of unintended side effects
• Negative tests (error conditions): Assert the error/rejection IS raised; essential for validating error paths

Both are equally important. The distinction is about clarity, not preference.

Anti-Patterns

• Over-mocking or mocking business logic
• Tests coupled to implementation details
• Duplicate coverage for same logical paths
• Production code added solely for testing
• Unexplained magic constants

End-to-End Tests

E2E tests run the full Recyclarr CLI against containerized Sonarr/Radarr instances. Tests verify that sync operations produce expected state in the services.

Running E2E Tests

MANDATORY: Use ./scripts/Run-E2ETests.ps1 - never run dotnet test directly for E2E tests. The script outputs a log file path; use rg to search logs without rerunning tests.

Resource Provider Strategy

The test uses multiple resource providers to verify different loading mechanisms:

Official Trash Guides (Pinned SHA)

- name: trash-guides-pinned
  type: trash-guides
  clone_url: https://github.com/TRaSH-Guides/Guides.git
  reference: <pinned-sha>
  replace_default: true

Purpose: Baseline data that tests real-world compatibility.

Use for: Stable CFs that exist in official guides (e.g., Bad Dual Groups, Obfuscated).

Why pinned: Prevents upstream changes from breaking tests unexpectedly.

Local Custom Format Providers

- name: sonarr-cfs-local
  type: custom-formats
  service: sonarr
  path: <local-path>

Purpose: Tests type: custom-formats provider behavior specifically.

Use for: CFs that need controlled structure or don't exist in official guides.

Trash Guides Override

- name: radarr-override
  type: trash-guides
  path: <local-path>

Purpose: Tests override/layering behavior (higher precedence than official guides).

Use for:

• Quality profiles with known structure for testing inheritance
• CF groups with controlled members for testing group behavior
• CFs that override official guide CFs (e.g., HybridOverride)

Fixture Directory Structure

Fixtures/
  recyclarr.yml              # Test configuration
  settings.yml               # Resource provider definitions
  custom-formats-sonarr/     # type: custom-formats provider (Sonarr)
  custom-formats-radarr/     # type: custom-formats provider (Radarr)
  trash-guides-override/     # type: trash-guides provider (override layer)
    metadata.json            # Defines paths for each resource type
    docs/
      Radarr/
        cf/                  # Custom formats
        cf-groups/           # CF groups
        quality-profiles/    # Quality profiles
      Sonarr/
        cf/
        cf-groups/
        quality-profiles/

When to Use Each Provider Type

Use Official Guides When

• Testing sync of real-world CFs that are stable
• Testing compatibility with actual guide data structures
• The specific CF content doesn't matter, just that syncing works

Use Local Fixtures When

• Testing specific inheritance/override behavior
• Testing resources that don't exist in official guides
• Testing provider-specific loading behavior
• You need controlled, predictable resource structure

Trash ID Conventions

• e2e00000000000000000000000000001 - E2E test Radarr quality profile
• e2e00000000000000000000000000002 - E2E test Sonarr quality profile
• e2e00000000000000000000000000003 - E2E test Sonarr guide-only profile
• e2e00000000000000000000000000010 - E2E test Sonarr CF group
• e2e00000000000000000000000000011 - E2E test Radarr CF group
• 00000000000000000000000000000001 through 00000000000000000000000000000007 - Local test CFs

Adding New Test Cases

1. For new CFs: Add JSON to appropriate custom-formats-* or trash-guides-override/docs/*/cf/
2. For new QPs: Add JSON to trash-guides-override/docs/*/quality-profiles/
3. For new CF groups: Add JSON to trash-guides-override/docs/*/cf-groups/
4. Update metadata.json if adding new resource type paths
5. Update recyclarr.yml to reference the new trash_ids
6. Update test assertions in RecyclarrSyncTests.cs

metadata.json Structure

The metadata.json file tells Recyclarr where to find each resource type:

{
  "json_paths": {
    "radarr": {
      "custom_formats": ["docs/Radarr/cf"],
      "qualities": [],
      "naming": [],
      "custom_format_groups": ["docs/Radarr/cf-groups"],
      "quality_profiles": ["docs/Radarr/quality-profiles"]
    },
    "sonarr": { "..." }
  }
}

Important: Paths must not contain spaces. Use cf instead of Custom Formats.