Browser Automation with browser-pilot

Control web browsers via the bp CLI. Execute actions, extract content, and automate workflows.

For complete action reference, patterns, and troubleshooting, see REFERENCE.md.

The Workflow: Snapshot → Ref

Always use refs from snapshots for reliable element targeting. Refs work even inside Shadow DOM.

# Step 1: Navigate and see what's on the page
bp exec '[{"action":"goto","url":"https://example.com"},{"action":"snapshot"}]'
# Output shows refs:
#   button "Submit" [ref=e4]
#   textbox "Email" [ref=e5]

# Step 2: Use refs (cached for this session+URL)
bp exec '[
  {"action":"fill","selector":"ref:e5","value":"user@example.com"},
  {"action":"click","selector":"ref:e4"}
]'

# Step 3: After navigation, snapshot again
bp exec '{"action":"snapshot"}'

Why refs? Work in Shadow DOM, no CSS guessing, stable within page load, cached across CLI calls.

Quick Reference

# Connect
bp connect --provider generic              # Local Chrome (auto-discovers)
bp connect --provider browserbase --name s # Cloud browser

# Execute actions
bp exec '[{"action":"goto","url":"..."},{"action":"snapshot"}]'
bp exec '[{"action":"click","selector":"ref:e4"}]'

# Handle dialogs (CRITICAL - blocks without this)
bp exec --dialog accept '{"action":"click","selector":"#delete-btn"}'

# Session management
bp list                    # List sessions
bp close -s session-name   # Close session
bp actions                 # Complete action reference

Basic Workflow

1. Connect

bp connect --provider generic --name dev              # Local Chrome
bp connect --provider browserbase --name prod --api-key $KEY  # Cloud

2. Execute Actions

bp exec -s dev '[
  {"action":"goto","url":"https://example.com/login"},
  {"action":"fill","selector":"#email","value":"user@example.com"},
  {"action":"submit","selector":"form"},
  {"action":"wait","waitFor":"navigation"},
  {"action":"snapshot"}
]'

3. Read Page State

bp snapshot -s dev --format text        # Accessibility tree
bp snapshot -s dev --format interactive # Interactive elements only

4. Close When Done

bp close -s dev

Dialog Handling (CRITICAL)

Native browser dialogs (alert(), confirm(), prompt()) block ALL automation until handled.

# ALWAYS use --dialog when actions might trigger native dialogs
bp exec --dialog accept '[{"action":"click","selector":"#delete-btn"}]'
bp exec --dialog dismiss '[{"action":"click","selector":"#cancel-action"}]'

Custom modals (role="dialog") work fine without this flag.

Multi-Selector & Optional Actions

Every selector accepts an array - tries each until one succeeds:

{"action": "click", "selector": ["#submit", "button[type=submit]", ".submit-btn"]}

Use optional: true to skip gracefully if element not found:

{"action": "click", "selector": "#cookie-banner", "optional": true, "timeout": 3000}

Tips

1. Take a snapshot before using refs - Populates the ref cache
2. Refs solve Shadow DOM - If CSS selector fails, use ref from snapshot
3. Always use --dialog when actions might trigger native dialogs
4. Trigger blur for validation - {"action":"press","key":"Tab"} after fill
5. Run bp actions for complete action reference

More: Action DSL Reference | Patterns | Troubleshooting