Analyze Compile Times with DuckDB

Use DuckDB to query and analyze rustc self-profiling data (chrome_profiler.json format) to identify compilation bottlenecks.

Prerequisites

• chrome_profiler.json file generated by crox (see rustc-self-profile skill)
• duckdb installed

Chrome Profiler JSON Schema

The JSON contains events with these fields:

• name: Activity or query name (e.g., "typeck", "mir_borrowck", "items_of_instance")
• cat: Category ("Query", "GenericActivity", etc.)
• ph: Phase ("X" = complete event with duration)
• ts: Timestamp in microseconds
• dur: Duration in microseconds
• pid, tid: Process and thread IDs
• args: Additional arguments (often NULL)

Common Queries

1. Overall Time Distribution

duckdb -c "
SELECT
    name,
    ROUND(SUM(dur) / 1000000.0, 2) as total_seconds,
    COUNT(*) as count,
    ROUND(AVG(dur) / 1000000.0, 3) as avg_seconds,
    ROUND(100.0 * SUM(dur) / (SELECT SUM(dur) FROM read_json('/tmp/project-profile/chrome_profiler.json')), 2) as pct
FROM read_json('/tmp/project-profile/chrome_profiler.json')
WHERE dur IS NOT NULL
GROUP BY name
ORDER BY SUM(dur) DESC
LIMIT 30
"

2. Query Events Only (Trait Resolution, Type Checking)

duckdb -c "
SELECT
    name,
    ROUND(SUM(dur) / 1000000.0, 3) as total_seconds,
    COUNT(*) as invocations,
    ROUND(AVG(dur) / 1000000.0, 3) as avg_seconds,
    ROUND(MAX(dur) / 1000000.0, 3) as max_seconds
FROM read_json('/tmp/project-profile/chrome_profiler.json')
WHERE dur IS NOT NULL
  AND cat = 'Query'
GROUP BY name
ORDER BY SUM(dur) DESC
LIMIT 50
"

Key query events:

• items_of_instance: Monomorphization
• codegen_select_candidate: Trait selection
• type_op_prove_predicate: Proving trait bounds
• typeck: Type checking
• mir_borrowck: Borrow checking
• layout_of: Computing type layouts

3. Most Expensive Individual Operations

duckdb -c "
SELECT
    name,
    cat,
    ROUND(dur / 1000000.0, 3) as seconds,
    tid
FROM read_json('/tmp/project-profile/chrome_profiler.json')
WHERE dur IS NOT NULL
ORDER BY dur DESC
LIMIT 100
"

4. Monomorphization Analysis

duckdb -c "
SELECT
    ROUND(SUM(dur) / 1000000.0, 3) as total_seconds,
    COUNT(*) as instances,
    ROUND(AVG(dur) / 1000000.0, 6) as avg_ms,
    ROUND(MAX(dur) / 1000000.0, 3) as max_seconds
FROM read_json('/tmp/project-profile/chrome_profiler.json')
WHERE name = 'items_of_instance'
  AND dur IS NOT NULL
"

If instances is > 30,000, you likely have trait/generic explosion.

5. Expensive Queries (>10ms)

duckdb -c "
SELECT
    name,
    cat,
    ROUND(dur / 1000000.0, 3) as seconds
FROM read_json('/tmp/project-profile/chrome_profiler.json')
WHERE dur IS NOT NULL
  AND cat = 'Query'
  AND dur > 10000  -- 10ms
ORDER BY dur DESC
LIMIT 100
"

6. LLVM Backend Time Breakdown

duckdb -c "
SELECT
    name,
    ROUND(SUM(dur) / 1000000.0, 2) as total_seconds,
    COUNT(*) as count
FROM read_json('/tmp/project-profile/chrome_profiler.json')
WHERE dur IS NOT NULL
  AND name LIKE 'LLVM%'
GROUP BY name
ORDER BY SUM(dur) DESC
"

7. Time by Category

duckdb -c "
SELECT
    cat,
    ROUND(SUM(dur) / 1000000.0, 2) as total_seconds,
    COUNT(*) as events
FROM read_json('/tmp/project-profile/chrome_profiler.json')
WHERE dur IS NOT NULL
GROUP BY cat
ORDER BY SUM(dur) DESC
"

8. Export to CSV for Further Analysis

duckdb -c "
COPY (
  SELECT
      name,
      cat,
      dur,
      ts,
      tid
  FROM read_json('/tmp/project-profile/chrome_profiler.json')
  WHERE dur IS NOT NULL
  ORDER BY dur DESC
  LIMIT 10000
) TO '/tmp/compile-profile-top10k.csv' (HEADER, DELIMITER ',')
"

Interpreting Results

LLVM Backend Dominates (>60% of time)

• LLVM_module_codegen_emit_obj, LLVM_module_optimize, LLVM_lto_optimize
• Root cause: Large amount of code, complex optimizations
• Solutions:
  • Reduce opt-level in Cargo.toml
  • Use lto = "thin" instead of "fat"
  • Increase codegen-units (trades runtime performance for compile time)

High Monomorphization (items_of_instance)

• Root cause: Many generic instantiations, trait explosion
• Look at: Total invocations (>30k is excessive)
• Solutions:
  • Use dyn Trait instead of generic parameters where possible
  • Consolidate similar types into fewer generics
  • Check for unintended trait bound combinatorics

Trait Resolution Issues

• codegen_select_candidate, type_op_prove_predicate taking >2s combined
• Root cause: Complex trait bounds, overlapping impls
• Solutions:
  • Simplify where clauses
  • Avoid deeply nested generic bounds
  • Check for trait coherence issues

Type Checking Slow

• typeck, mir_borrowck taking >3s combined
• Root cause: Large functions, complex types
• Solutions:
  • Split large functions
  • Simplify type signatures
  • Reduce nested generics

Advanced: Finding Specific Culprits

Unfortunately, chrome_profiler.json doesn't include which specific types/functions cause monomorphization. For that, you need to:

1. Use cargo build -Z timings for crate-level timing
2. Check cargo +nightly rustc -- -Z print-type-sizes for large types
3. Use cargo +nightly rustc -- -Z print-mono-items to see all monomorphized items (warning: very verbose)

Tips

• DuckDB can query JSON directly without loading into memory
• Use LIMIT to avoid overwhelming output
• The dur field is in microseconds, divide by 1,000,000 for seconds
• Focus on the top 20-30 items by total time
• Compare counts to see if a single expensive operation vs many small operations