Navigator Submodule Development

Why Navigator is a Submodule

Navigator is included as a Git submodule because showcase routinely needs Navigator fixes to implement deployment plans. Changes are tested in showcase context before being pushed upstream.

Location: navigator/ (Git submodule) Language: Go Purpose: Multi-tenant web server with framework independence

Project Structure

navigator/
├── cmd/navigator/            # Production Navigator implementation
├── internal/                  # Modular packages
│   ├── config/               # Configuration loading and parsing
│   ├── server/               # HTTP handling, routing, static files
│   ├── auth/                 # Authentication (htpasswd)
│   ├── process/              # Web app lifecycle management
│   └── proxy/                # Reverse proxy and Fly-Replay
└── docs/                     # MkDocs documentation

Critical Architecture: Configuration Flow

Understanding this flow is essential for fixing config-related bugs:

1. YAML file (user-facing config)
2. YAMLConfig struct in types.go (mirrors YAML structure with yaml tags)
3. ConfigParser in parser.go (converts YAML to internal format)
4. Config struct in types.go (optimized internal representation)

Common Bug Pattern: When adding config fields, developers often:

• ✅ Add field to Config struct (internal)
• ❌ FORGET to add field to YAMLConfig struct
• ❌ FORGET to add parsing logic in parser.go
• Result: YAML config is valid but silently ignored!

Request Flow in Handler (Order Matters!)

// From internal/server/handler.go ServeHTTP()
1. Health checks      // BEFORE everything (bypasses auth)
2. Authentication     // EARLY check (before routing)
3. Rewrites/redirects
4. CGI scripts
5. Reverse proxies
6. Static files
7. Maintenance mode
8. Web app proxy      // Tenant routing

Security Critical: Health checks MUST come before auth. Auth MUST come before tenant routing.

Common Development Tasks

Adding a New Config Option

Example: Health Check Config (Recent Fix)

Step 1: Add to YAMLConfig in internal/config/types.go:

type YAMLConfig struct {
    Server struct {
        // ... other fields
        HealthCheck HealthCheckConfig `yaml:"health_check"` // ← Must have yaml tag
    } `yaml:"server"`
}

Step 2: Add to internal Config (if different structure needed):

type Config struct {
    Server struct {
        // ... other fields
        HealthCheck HealthCheckConfig
    }
}

Step 3: Add parsing logic in internal/config/parser.go:

func (p *ConfigParser) parseServerConfig() {
    // ... other parsing
    p.config.Server.HealthCheck = p.yamlConfig.Server.HealthCheck // ← CRITICAL
}

Step 4: Add tests in internal/config/parser_test.go:

func TestConfigParser_ParseHealthCheck(t *testing.T) {
    yamlConfig := YAMLConfig{}
    yamlConfig.Server.HealthCheck = HealthCheckConfig{
        Path: "/up",
        Response: &HealthCheckResponse{Status: 200, Body: "OK"},
    }

    parser := NewConfigParser(&yamlConfig)
    config, err := parser.Parse()

    if config.Server.HealthCheck.Path != "/up" {
        t.Errorf("HealthCheck not parsed correctly")
    }
}

Step 5: Add integration tests in internal/server/handler_test.go (or appropriate file).

Adding Handler Behavior

Example: Health Check Handler (Recent Fix)

// Add early in ServeHTTP - order matters!
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    // Health checks BEFORE authentication
    if h.config.Server.HealthCheck.Path != "" && r.URL.Path == h.config.Server.HealthCheck.Path {
        h.handleHealthCheck(recorder, r)
        return  // Stop processing
    }

    // Authentication check comes next
    // ... rest of handler
}

Integration Test (Security Critical):

func TestHandler_ServeHTTP_HealthCheckBeforeAuth(t *testing.T) {
    cfg := &config.Config{}
    cfg.Server.HealthCheck = config.HealthCheckConfig{
        Path: "/up",
        Response: &config.HealthCheckResponse{Status: 200, Body: "OK"},
    }
    cfg.Auth.Enabled = true  // Enable auth

    basicAuth := &auth.BasicAuth{}
    handler := CreateTestHandler(cfg, nil, basicAuth, nil)

    req := httptest.NewRequest("GET", "/up", nil)
    // NO auth credentials provided
    recorder := httptest.NewRecorder()
    handler.ServeHTTP(recorder, req)

    // MUST succeed without auth - critical security requirement
    if recorder.Code != 200 {
        t.Errorf("Health check should bypass auth")
    }
}

Testing Requirements

Test Checklist

• Config parser tests (internal/config/parser_test.go)

  • YAML unmarshaling works
  • Parsing logic copies fields correctly
  • Default values applied
  • Edge cases (empty, nil)

• Integration tests (appropriate *_test.go)

  • Full request/response through ServeHTTP
  • Security implications (especially auth bypass scenarios)
  • Edge cases and error conditions

Running Tests

# All tests
go test ./...

# Specific package
go test -v ./internal/config/
go test -v ./internal/server/

# With coverage
go test -cover ./...

# Pre-commit validation (CI requirements)
gofmt -s -l . && \
golangci-lint run && \
go vet ./... && \
go test -race -cover -timeout=3m ./... && \
go build ./cmd/navigator-refactored && \
echo "✓ All CI checks passed!"

Recent Fixes (Nov 2024)

Fix 1: CGI reload_config Not Parsed

Problem: reload_config field in CGI scripts ignored Cause: Field missing from YAMLConfig.Server.CGIScripts Fix: Added field to CGIScriptConfig struct Lesson: Always check YAMLConfig has all fields with yaml tags

Fix 2: Health Checks Not Working

Problem: /up returned 401 (auth) or started index tenant unnecessarily Cause:

• YAMLConfig.Server missing HealthCheck field
• parseServerConfig() not copying health check config

Fix:

// types.go - Added to YAMLConfig.Server
HealthCheck HealthCheckConfig `yaml:"health_check"`

// parser.go - Added to parseServerConfig()
p.config.Server.HealthCheck = p.yamlConfig.Server.HealthCheck

Test Coverage: 270 lines added

• Parser tests: YAML → Config transformation
• Handler tests: before auth, different paths, custom headers
• Security test: health checks bypass authentication

Common Bug Patterns

Pattern 1: Config Parser Forgot to Copy Field

// ❌ BUG: Field exists in YAMLConfig but not copied
type YAMLConfig struct {
    Server struct {
        HealthCheck HealthCheckConfig `yaml:"health_check"`
    }
}

func (p *ConfigParser) parseServerConfig() {
    // ... other fields copied
    // ❌ FORGOT: p.config.Server.HealthCheck = p.yamlConfig.Server.HealthCheck
}

// ✅ FIX: Add the copy statement
p.config.Server.HealthCheck = p.yamlConfig.Server.HealthCheck

How to catch: Write parser tests that verify field is copied.

Pattern 2: Missing yaml Tag

// ❌ BUG: No yaml tag, field won't unmarshal
type ServerConfig struct {
    Listen string  // Won't unmarshal from YAML
}

// ✅ FIX: Add yaml tag
type ServerConfig struct {
    Listen string `yaml:"listen"`
}

Pattern 3: Wrong Handler Order

// ❌ BUG: Auth check after tenant routing (can be bypassed!)
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    h.handleWebAppProxy(w, r)  // Routes first
    if !h.auth.CheckAuth(r) {  // Auth never runs!
        return
    }
}

// ✅ FIX: Correct order
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    // 1. Health checks (before everything)
    if h.config.Server.HealthCheck.Path != "" { /* ... */ }

    // 2. Authentication (EARLY)
    isPublic := auth.ShouldExcludeFromAuth(r.URL.Path, h.config)
    needsAuth := h.auth.IsEnabled() && !isPublic
    if needsAuth && !h.auth.CheckAuth(r) {
        h.auth.RequireAuth(recorder)
        return
    }

    // 3. Then routing
    h.handleWebAppProxy(w, r)
}

How to catch: Write integration tests that verify security (auth bypass scenarios).

Deployment Workflow

1. Rebase submodule on main before making changes:

cd navigator

# Fetch latest from remote
git fetch origin

# Rebase on main (creates a clean linear history)
git rebase origin/main

# If conflicts, resolve them and continue
# git rebase --continue

2. Work in Navigator submodule:

# Make changes, run tests
go test ./...
golangci-lint run
git add -A
git commit -m "Fix: health check parsing"

3. Test in showcase context:

cd ..  # Back to showcase root
# Test Navigator fix with showcase deployment

4. Push to both repos:

# Push Navigator changes
cd navigator
git push origin HEAD:refs/heads/main

# Update submodule reference in showcase
cd ..
git add navigator
git commit -m "Update navigator: fix health check parsing"
git push

Note: If submodule is in detached HEAD state (common after git submodule update), the rebase step ensures you're building on the latest main branch before making changes.

Quick Reference Commands

# Build Navigator
cd navigator
go build -o bin/navigator cmd/navigator

# Or use make
make build

# Run with config
./bin/navigator config/navigator.yml

# Reload config (SIGHUP)
kill -HUP $(cat /tmp/navigator.pid)

# Debug config loading
LOG_LEVEL=debug ./bin/navigator config/navigator.yml

When to Fix Navigator

You need Navigator changes when:

• ✅ Deployment plan requires new config options
• ✅ New routing/proxy behavior needed
• ✅ Authentication/security enhancements required
• ✅ Config not being parsed correctly (check YAMLConfig + parser)
• ✅ Request flow order needs adjustment

Pattern: Fix Navigator first, test in showcase, then continue deployment plan.

Essential Files

• navigator/CLAUDE.md - Comprehensive development guide (read first!)
• internal/config/types.go - All config structures (YAMLConfig + Config)
• internal/config/parser.go - YAML → Config conversion
• internal/server/handler.go - Main HTTP request routing

Getting Help

1. Read navigator/CLAUDE.md for comprehensive guide
2. Check navigator/docs/ for user documentation
3. Search tests for similar patterns
4. Review Git history for "Fix:" commits
5. Run with LOG_LEVEL=debug to see what's happening