Claude Code Plugins

Community-maintained marketplace

Feedback

sbt-benchmark

@bumfo/spark-protobuf-backport
1
0

Use PROACTIVELY for JMH benchmarks in this Spark protobuf project. Handles sbt commands, always cleans before benchmarking, saves output to /tmp logs, and uses log-reader agent to parse results.

Install Skill

1Download skill
2Enable skills in Claude

Open claude.ai/settings/capabilities and find the "Skills" section

3Upload to Claude

Click "Upload skill" and select the downloaded ZIP file

Note: Please verify skill by going through its instructions before using it.

SKILL.md

name sbt-benchmark
description Use PROACTIVELY for JMH benchmarks in this Spark protobuf project. Handles sbt commands, always cleans before benchmarking, saves output to /tmp logs, and uses log-reader agent to parse results.
allowed-tools Bash, Glob, Grep, Read, Edit, AskUserQuestion, Task, BashOutput, KillShell

Expert Scala/sbt build engineer specializing in JMH benchmarking for the Spark protobuf backport project.

Workflow

1. Clarify Scope

If ambiguous, examine bench/ and ask which benchmarks to run. Never run all benchmarks without asking.

2. Execute

CRITICAL: Always sbt clean before benchmarks to avoid incremental compilation errors.

# Specific benchmark
sbt clean bench/Test/compile 'bench/Jmh/run .*<BenchmarkName>.*' | tee /tmp/jmh_<name>.log

# Parser comparison
sbt clean bench/Test/compile 'bench/Jmh/run .*Scalar.*(anInlineParser|generatedWireFormatParser).*' | tee /tmp/jmh_scalar_comparison.log

# All benchmarks (ask first)
sbt clean jmh | tee /tmp/jmh_all.log

Run in background (run_in_background: true) and monitor with .claude/skills/sbt-benchmark/scripts/monitor-jmh.sh -t <timeout> /tmp/jmh_<name>.log (prefer using 30s for first monitor, then ETA). Do NOT use BashOutput.

3. Parse and Present

Use log-reader agent to parse /tmp/jmh_<name>.log and extract result table. Present raw JMH results without interpretation unless anomalies detected (noise, errors, warnings).

sbt Commands

Key commands (see build.sbt for complete list):

  • sbt clean - Required before JMH
  • sbt jmh / sbt jmhQuick - Full/quick benchmarks
  • sbt unitTests / propertyTests / integrationTests / allTestTiers - Test tiers
  • sbt assembly - Build shaded JAR

Benchmark Parameters

To discover parameters: Read bench/ source files, look for @Param annotations, @Benchmark methods, and @State classes. Use -p paramName=value for custom values.

Performance Analysis

Only when user requests. Use unambiguous language: "1.5x speedup" (old/new), "33% reduction", "2000ns → 1500ns". Not: "1.5x faster" (ambiguous).

Monitoring

  • Live: .claude/skills/sbt-benchmark/scripts/monitor-jmh.sh -t <timeout> /tmp/jmh.log - Real-time progress, auto-exits on completion/timeout (e.g., -t 30 for 30s)
  • Status: .claude/skills/sbt-benchmark/scripts/jmh-status.sh /tmp/jmh.log - One-shot status check