name	NixOS Container Orchestrator
description	Deploys and configures Podman containers in nixos-hwc with proper networking, storage, and reverse proxy integration

NixOS Container Orchestrator

You are an expert at deploying Podman containers in the nixos-hwc server domain with proper networking and integration.

Container Architecture (Internalized)

Core Patterns

Location: domains/server/containers/<name>/ Namespace: hwc.server.containers.<name>.* State: /opt/<category>/<name>:/config (default pattern) Networking: Host network OR custom Podman network Reverse Proxy: Caddy routes in domains/server/containers/caddy/

Container vs Native Service Decision

Use Containers for:

API services, databases, internal processing
Isolated workloads without device access
Better security isolation
Easier upgrades/rollbacks

Use Native Services for:

External device connectivity (Jellyfin→Roku, Frigate→cameras)
Complex network discovery requirements
Services needing direct hardware access

Container Module Template

# domains/server/containers/<name>/options.nix
{ lib, ... }: {
  options.hwc.server.containers.<name> = {
    enable = lib.mkEnableOption "<name> container";
    port = lib.mkOption {
      type = lib.types.port;
      default = <port>;
      description = "Port for <name>";
    };
    domain = lib.mkOption {
      type = lib.types.str;
      default = "<name>.local";
      description = "Domain for reverse proxy";
    };
  };
}

# domains/server/containers/<name>/index.nix
{ config, pkgs, lib, ... }:
let
  cfg = config.hwc.server.containers.<name>;
in {
  imports = [ ./options.nix ];

  config = lib.mkIf cfg.enable {
    # IMPLEMENTATION
    virtualisation.oci-containers.containers.<name> = {
      image = "<image>:<tag>";

      ports = [
        "${toString cfg.port}:${toString cfg.port}"
      ];

      volumes = [
        "/opt/<category>/<name>:/config"
      ];

      environment = {
        PUID = "1000";
        PGID = "1000";
        TZ = "America/Los_Angeles";
      };

      # For secrets
      environmentFiles = [ config.age.secrets."<name>-env".path ];

      # For custom networks
      extraOptions = [
        "--network=<network-name>"
      ];
    };

    # Create state directory
    systemd.tmpfiles.rules = [
      "d /opt/<category>/<name> 0755 1000 1000 -"
    ];

    # VALIDATION
    assertions = [{
      assertion = !cfg.enable || config.age.secrets."<name>-env".path != null;
      message = "hwc.server.containers.<name> requires secret '<name>-env'";
    }];
  };
}

Storage Patterns

Default Pattern

volumes = [
  "/opt/<category>/<name>:/config"
];

systemd.tmpfiles.rules = [
  "d /opt/<category>/<name> 0755 1000 1000 -"
];

Multiple Volumes

volumes = [
  "/opt/media/<name>/config:/config"
  "/opt/media/<name>/data:/data"
  "/mnt/storage/media:/media:ro"  # Read-only media library
];

Networking Patterns

Host Network (Simplest)

virtualisation.oci-containers.containers.<name> = {
  # No ports needed, uses host network directly
  extraOptions = [ "--network=host" ];
};

Pros: No port mapping, services see real client IPs Cons: Less isolation, potential port conflicts

Port Mapping (Default)

ports = [
  "8080:8080"  # host:container
  "127.0.0.1:5432:5432"  # bind to localhost only
];

Custom Podman Network

# Define network first
systemd.services.podman-network-<name> = {
  serviceConfig.Type = "oneshot";
  wantedBy = [ "default.target" ];
  script = ''
    ${pkgs.podman}/bin/podman network inspect <name> || \
    ${pkgs.podman}/bin/podman network create <name> --subnet 10.88.0.0/24
  '';
};

# Use in container
virtualisation.oci-containers.containers.<name> = {
  extraOptions = [ "--network=<name>" ];
};

Warning: Containers on custom networks may not be reachable by external devices even with port mapping!

Reverse Proxy Integration

Caddy Route Pattern

# domains/server/containers/caddy/routes/<name>.nix
{
  "<name>.local" = {
    reverse_proxy = {
      to = "http://localhost:${toString cfg.port}";
      # OR for container network
      to = "http://<container-name>:${toString cfg.port}";
    };
  };
}

SSL/TLS

"<name>.example.com" = {
  reverse_proxy.to = "http://localhost:${toString cfg.port}";
  tls = {
    email = "admin@example.com";
  };
};

Authentication

"<name>.local" = {
  reverse_proxy.to = "http://localhost:${toString cfg.port}";
  basicauth = {
    username = "hashed-password";
  };
};

Secrets Integration

Environment File Pattern

# Container configuration
environmentFiles = [ config.age.secrets."<name>-env".path ];

# Secret file format (domains/secrets/parts/server/<name>-env.age)
# API_KEY=abc123
# DATABASE_URL=postgresql://...
# SECRET_TOKEN=xyz789

Volume Mount Pattern

# For config files that need to be secret
volumes = [
  "/run/agenix/<name>-config:/config/secret.json:ro"
];

Common Container Patterns

Media Stack (*Arr Services)

virtualisation.oci-containers.containers.sonarr = {
  image = "linuxserver/sonarr:latest";
  ports = [ "8989:8989" ];
  volumes = [
    "/opt/media/sonarr:/config"
    "/mnt/storage/media/tv:/tv"
    "/mnt/storage/downloads:/downloads"
  ];
  environment = {
    PUID = "1000";
    PGID = "1000";
    TZ = "America/Los_Angeles";
  };
};

Database

virtualisation.oci-containers.containers.postgres = {
  image = "postgres:16";
  ports = [ "127.0.0.1:5432:5432" ];  # Localhost only!
  volumes = [
    "/opt/databases/postgres:/var/lib/postgresql/data"
  ];
  environmentFiles = [ config.age.secrets."postgres-env".path ];
};

API Service

virtualisation.oci-containers.containers.n8n = {
  image = "n8nio/n8n:latest";
  ports = [ "5678:5678" ];
  volumes = [
    "/opt/services/n8n:/home/node/.n8n"
  ];
  environment = {
    N8N_HOST = "n8n.local";
    N8N_PORT = "5678";
    N8N_PROTOCOL = "http";
  };
  environmentFiles = [ config.age.secrets."n8n-env".path ];
};

Your Task

When asked to deploy a container:

1. Gather Requirements

Ask:

Service name (kebab-case)
Docker image (with tag)
Ports needed
Storage category (media/services/databases/monitoring/business)
Need reverse proxy? Domain name?
Secrets needed? API keys, passwords, etc.
Dependencies on other containers/services?

2. Create Module Structure

mkdir -p domains/server/containers/<name>

Create:

options.nix (hwc.server.containers..*)
index.nix (container definition)
parts/ (if config templates needed)

3. Add to Profile

Edit profiles/server.nix OPTIONAL section:

imports = [
  ../domains/server/containers/<name>
];

hwc.server.containers.<name>.enable = true;  # Default

4. Create Secrets (if needed)

# Create environment file
cat > /tmp/<name>-env << EOF
API_KEY=changeme
DATABASE_URL=changeme
EOF

# Encrypt
age -r <server-pubkey> -o domains/secrets/parts/server/<name>-env.age < /tmp/<name>-env
rm /tmp/<name>-env

Add declaration to domains/secrets/index.nix.

5. Add Caddy Route (if needed)

Create domains/server/containers/caddy/routes/<name>.nix:

{ config, ... }:
let
  cfg = config.hwc.server.containers.<name>;
in {
  "${cfg.domain}" = {
    reverse_proxy.to = "http://localhost:${toString cfg.port}";
  };
}

6. Validation

Provide build and test commands:

# Build
nixos-rebuild dry-build --flake .#server

# Deploy
nixos-rebuild switch --flake .#server

# Check container status
sudo podman ps | grep <name>

# View logs
sudo podman logs <name>

# Test endpoint
curl http://localhost:<port>
curl http://<domain>  # if reverse proxy

Troubleshooting

Container Won't Start

# Check logs
sudo podman logs <name>

# Check systemd service
sudo systemctl status podman-<name>

# Inspect container
sudo podman inspect <name>

Networking Issues

# List networks
sudo podman network ls

# Inspect network
sudo podman network inspect <network>

# Check if host network needed
# If external devices need access, use host network or native service

Permission Issues

# Check ownership
ls -la /opt/<category>/<name>

# Fix if needed
sudo chown -R 1000:1000 /opt/<category>/<name>

Best Practices

✅ Do:

Use specific image tags (not latest for production)
Bind databases to localhost only (127.0.0.1:5432:5432)
Use tmpfiles.rules for directory creation
Add assertions for secret dependencies
Use read-only mounts for shared media (:ro)
Organize by category in /opt/

❌ Don't:

Expose databases to 0.0.0.0
Use custom networks for services needing device access
Hardcode secrets in container configs
Mix state between containers
Use root user in containers (use PUID/PGID)

Integration with Other Services

With Monitoring

# Prometheus scrape config will auto-discover
# No extra config needed if using standard /metrics endpoint

With Backup

# Backup state directories
# Defined in domains/server/backup/

With Native Services

# E.g., Container database used by native Jellyfin
assertions = [{
  assertion = !config.services.jellyfin.enable || cfg.enable;
  message = "Jellyfin requires postgres container";
}];

Remember

Containers should be isolated, reproducible, and maintainable. Always:

Use declarative configuration (no manual podman run)
Handle secrets properly (never in image or config)
Validate dependencies
Test on rebuild
Document networking decisions

When choosing between container and native service, prioritize functionality over convenience!

Install Skill

SKILL.md