Multi-AI Consultant

Consult external AIs for second opinions when Claude Code is stuck or making critical decisions.

---

What This Skill Does

This skill enables future Claude Code sessions to consult other AIs when:

• Stuck on a bug after one failed attempt
• Making architectural decisions
• Security concerns need validation
• Fresh perspective needed

Key innovation: Uses existing CLI tools (gemini, codex) instead of building MCP servers - much simpler and more maintainable.

---

When to Use This Skill

Automatic Triggers (No User Action Needed)

Claude Code should automatically suggest using this skill when:

1. After 1 failed debugging attempt

   • Tried one approach to fix a bug
   • Still not working or different error
   • → Suggest: "Should I consult [Gemini|Fresh Claude] for a second opinion?"

2. Before architectural decisions

   • Significant design choices (state management, routing, data flow)
   • Framework selection
   • Database schema design
   • → Auto-consult (mention to user): "Consulting Gemini for architectural validation..."

3. Security changes

   • Authentication logic
   • Authorization rules
   • Cryptography
   • Input validation
   • → Auto-consult: "Consulting Gemini to verify security approach..."

4. When uncertain

   • Multiple valid approaches
   • Trade-offs not clear
   • Conflicting advice in documentation
   • → Suggest: "Would you like me to consult another AI for additional perspective?"

Manual Invocation (User Commands)

User can explicitly request consultation with:

• /consult-gemini [question] - Gemini 2.5 Pro with thinking, search, grounding
• /consult-codex [question] - OpenAI GPT-4 via Codex CLI (repo-aware)
• /consult-claude [question] - Fresh Claude subagent (free, fast)
• /consult-ai [question] - Router that asks which AI to use

---

The Three AIs

AI	Tool	When to Use	Special Features	Cost
Gemini 2.5 Pro	gemini CLI	Web research, latest docs, thinking	Google Search, extended reasoning, grounding	~$0.10-0.50
OpenAI GPT-4	codex CLI	Repo-aware analysis, code review	Auto-scans directory, OpenAI reasoning	~$0.05-0.30
Fresh Claude	Task tool	Quick second opinion, budget-friendly	Same capabilities, fresh perspective	Free

---

How It Works

Architecture

Claude Code encounters bug/decision
        ↓
Suggests consultation (or user requests)
        ↓
User approves
        ↓
Execute appropriate slash command
        ↓
CLI command calls external AI
        ↓
Parse response
        ↓
Synthesize: Claude's analysis + External AI's analysis
        ↓
Present 5-part comparison
        ↓
Ask permission to implement

The 5-Part Synthesis Format

Every consultation must follow this format (prevents parroting):

1. 🤖 My Analysis - Claude's original reasoning and attempts
2. 💎/🔷/🔄 Other AI's Analysis - External AI's complete response
3. 🔍 Key Differences - Agreement, divergence, what each AI caught/missed
4. ⚡ Synthesis - Combined perspective, root cause, trade-offs
5. ✅ Recommended Action - Specific next steps with file paths/line numbers

End with: "Should I proceed with this approach?"

---

Setup Instructions

1. Install CLIs

Gemini CLI (required):

npm install -g @google/generative-ai-cli
export GEMINI_API_KEY="your-key"

Get API key: https://aistudio.google.com/apikey

Codex CLI (optional):

npm install -g codex
export OPENAI_API_KEY="sk-..."

Get API key: https://platform.openai.com/api-keys

Fresh Claude: Built-in (uses Task tool, no setup)

2. Install Skill

cd ~/.claude/skills
git clone [this-repo] multi-ai-consultant
# OR symlink from development location

3. Copy Templates to Project

# Copy to project root
cp ~/.claude/skills/multi-ai-consultant/templates/GEMINI.md ./
cp ~/.claude/skills/multi-ai-consultant/templates/codex.md ./
cp ~/.claude/skills/multi-ai-consultant/templates/.geminiignore ./

# Copy log parser (optional)
cp ~/.claude/skills/multi-ai-consultant/templates/consultation-log-parser.sh ~/bin/
chmod +x ~/bin/consultation-log-parser.sh

4. Verify Installation

# Test Gemini CLI
gemini -p "test" && echo "✅ Gemini working"

# Test Codex CLI (if installed)
codex exec "test" --yolo && echo "✅ Codex working"

# Test skill discovery
# Ask Claude Code: "I'm stuck on this bug, can you help?"
# Claude should suggest consultation

---

Usage Examples

Example 1: Stuck on Bug

Scenario: JWT authentication failing after one attempt

Claude's process:

1. Try initial fix (token expiry check)
2. Still failing (now "invalid signature" error)
3. → Automatically suggest consultation
4. User approves
5. Execute /consult-gemini with:
   - Problem: 401 error details
   - What was tried: token expiry fix
   - Current status: invalid signature error
   - Context: @src/auth/session.ts @src/middleware/jwt.ts
6. Gemini finds: Using process.env instead of env binding (Cloudflare)
7. Synthesize both perspectives
8. Ask permission to implement Gemini's fix

Result: Bug fixed with second opinion, avoided 2-3 more trial-and-error attempts

Example 2: Architecture Decision

Scenario: Choosing state management for new feature

Claude's process:

1. User asks: "How should we handle state for this feature?"
2. → Auto-consult Gemini (architectural decision)
3. Execute /consult-gemini with:
   - Problem: State management choice
   - Options considered: Redux, Zustand, Context
   - Context: @src/ (existing patterns)
4. Gemini searches for latest React state management best practices
5. Synthesize Claude's + Gemini's analysis
6. Present recommendation with trade-offs

Result: Informed decision with latest best practices

Example 3: Manual Consultation

User: "I want a second opinion on this refactoring"

Claude's process:

1. User explicitly requests consultation
2. Ask which AI: Gemini, Codex, or Fresh Claude?
3. User chooses Codex (wants repo-aware analysis)
4. Execute /consult-codex with:
   - Problem: Refactoring proposal
   - Context: (Codex scans repo automatically)
5. Codex checks consistency with existing code
6. Synthesize both perspectives
7. Present recommendation

Result: Validation of refactoring approach + consistency check

---

Slash Commands

/consult-gemini [question]

Use when: Need web research, latest docs, extended thinking

What it does:

1. Pre-flight check (Gemini CLI working?)
2. Smart context selection based on problem type
3. Execute with locked config: gemini-2.5-pro --thinking --google-search --grounding
4. Parse JSON response
5. Synthesize with 5-part format
6. Log cost to ~/.claude/ai-consultations/consultations.log

Example:

/consult-gemini Is this JWT validation secure by 2025 standards?

Context: Uses @<path> syntax for selective context

System instructions: Auto-loads GEMINI.md from project root

Privacy: Respects .gitignore + .geminiignore

---

/consult-codex [question]

Use when: Need repo-aware analysis, code review, OpenAI reasoning

What it does:

1. Pre-flight check (OpenAI API key valid?)
2. cd to project directory (Codex scans automatically)
3. Execute with --yolo flag: codex exec - -m gpt-4-turbo --yolo
4. Parse output from temp file
5. Synthesize with 5-part format
6. Log cost (estimated tokens)

Example:

/consult-codex Review this codebase for performance issues

Context: Repo-aware (automatically reads all non-gitignored files)

System instructions: Auto-loads codex.md from project root or ~/.codex/instructions.md

Privacy: Respects .gitignore, warns if not in Git repo

---

/consult-claude [question]

Use when: Quick second opinion, free, fresh perspective

What it does:

1. Gather context with Read tool
2. Build detailed prompt for subagent
3. Launch Task tool (general-purpose subagent)
4. Receive fresh perspective
5. Synthesize with 5-part format
6. Log consultation (free, but tracked)

Example:

/consult-claude Am I missing something obvious in this state management bug?

Context: You manually read relevant files and pass inline

Advantages: Free, same capabilities, fast, fresh perspective

Limitations: No web search, no extended thinking, same knowledge cutoff

---

/consult-ai [question]

Use when: Unsure which AI to use

What it does:

1. Analyze problem type (bug, architecture, security, etc.)
2. Recommend which AI based on needs
3. Ask user to choose
4. Route to appropriate command

Example:

/consult-ai How should we structure this microservices architecture?

→ Recommends Gemini (needs web research for latest patterns)
→ User approves
→ Executes /consult-gemini internally

Decision factors:

• Need web research? → Gemini
• Need repo-aware? → Codex
• Need budget-friendly? → Fresh Claude
• Need extended thinking? → Gemini
• Need fresh perspective? → Fresh Claude

---

Templates

GEMINI.md (Project Root)

System instructions for Gemini consultations. Enforces:

• 5-part response format
• Comparison with Claude's analysis
• Web search for latest docs
• Specific code references
• No parroting

Auto-loaded: Place in project root, Gemini CLI reads automatically

codex.md (Project Root or ~/.codex/instructions.md)

System instructions for Codex consultations. Enforces:

• 5-part response format
• Repo-aware analysis
• Consistency checks
• Impact assessment
• Specific file references

Auto-loaded: Codex CLI reads from project root or global config

.geminiignore (Project Root)

Extra privacy exclusions beyond .gitignore. Excludes:

• .env* files
• *secret*, *credentials*
• Build artifacts
• Large media files
• API keys and tokens

Auto-loaded: Gemini CLI respects this file

consultation-log-parser.sh

View consultation history:

consultation-log-parser.sh           # Last 10
consultation-log-parser.sh --all     # All consultations
consultation-log-parser.sh --summary # Stats only
consultation-log-parser.sh --ai gemini # Filter by AI

Log format: timestamp,ai,model,input_tokens,output_tokens,cost,project_path

---

Cost Tracking

Every consultation is logged to ~/.claude/ai-consultations/consultations.log

Format:

2025-11-07T14:23:45-05:00,gemini,gemini-2.5-pro,15420,850,0.1834,/home/user/project
2025-11-07T15:10:22-05:00,codex,gpt-4-turbo,8230,430,0.0952,/home/user/project
2025-11-07T16:05:11-05:00,claude-subagent,claude-sonnet-4-5,0,0,0.00,/home/user/project

View logs:

consultation-log-parser.sh --summary
# Total consultations: 47
# Gemini: 23, Codex: 12, Fresh Claude: 12
# Total cost: $8.45

Pricing (verify current rates):

• Gemini 2.5 Pro: ~$0.000015/input token, ~$0.00006/output token
• GPT-4 Turbo: ~$0.00001/input token, ~$0.00003/output token
• Fresh Claude: Free (same API call)

---

Privacy & Security

Automatic Protection

Both CLIs respect .gitignore automatically. Files in .gitignore are never sent.

Additional Protection (.geminiignore)

Create .geminiignore in project root for extra exclusions:

*.env*
*secret*
*credentials*
.dev.vars
wrangler.toml

Pre-Consultation Check

Claude Code should warn if:

• Sensitive file patterns detected in context
• Not in Git repo (Codex)
• Large context (>100k tokens estimated)

Ask permission: "About to send context to [AI]. Files include: [list]. Proceed?"

Manual Verification

Before consultation, user can check:

# What will be sent to Gemini
ls -la @src/

# Check gitignore is working
git status --ignored

---

Known Issues & Solutions

1. CLI Not Installed

Error: gemini: command not found or codex: command not found

Fix:

npm install -g @google/generative-ai-cli
npm install -g codex

Prevention: Pre-flight checks in slash commands

---

2. API Keys Invalid

Error: "API key invalid" or authentication failures

Fix:

export GEMINI_API_KEY="your-key"
export OPENAI_API_KEY="sk-..."

# Or add to ~/.bashrc
echo 'export GEMINI_API_KEY="your-key"' >> ~/.bashrc
source ~/.bashrc

Prevention: Pre-flight checks test API before consultation

---

3. Context Too Large

Error: Token limit exceeded or very expensive consultation

Fix: Use smart context selection

• Bug: Just buggy file + imports (not entire codebase)
• Architecture: Relevant directories only
• Use @path/to/specific/file.ts not @.

Prevention: Slash commands guide smart context selection

---

4. Privacy Leaks

Risk: Accidentally sending .env or secrets

Fix:

• Both CLIs respect .gitignore (automatic)
• Create .geminiignore for extra protection
• Claude warns if sensitive patterns detected

Prevention: Always check .gitignore is configured

---

5. Cost Overruns

Issue: Expensive consultations accumulating

Fix:

• Check logs: consultation-log-parser.sh --summary
• Use Fresh Claude for quick questions (free)
• Use smart context (specific files, not entire repo)

Prevention: Cost tracking + warnings for large context

---

6. Codex Hanging

Error: codex exec hangs waiting for approval

Fix: Always use --yolo flag

Prevention: Slash command includes --yolo by default (hard to miss)

---

7. JSON Parsing Fails (Gemini)

Error: jq: parse error

Fix: Check exit code before parsing, fall back to plain text

Prevention: Slash command checks exit code first

---

8. Not in Git Repo (Codex)

Warning: Codex warns if not in Git repo (safety feature)

Fix: Add --skip-git-repo-check flag if appropriate

Prevention: Slash command includes flag

---

Token Efficiency

Without This Skill

Typical scenario (stuck on bug):

1. Try approach 1 (~4k tokens)
2. Research CLI syntax (~3k tokens)
3. Format response (~2k tokens)
4. Try approach 2 (~4k tokens)
5. Research documentation (~3k tokens)
6. Try approach 3 (~4k tokens)

Total: ~20k tokens, 30-45 minutes

With This Skill

Same scenario:

1. Try approach 1 (~4k tokens)
2. Execute /consult-gemini (~1k tokens to execute)
3. Gemini finds issue (billed separately, but <5k tokens)
4. Implement fix (~3k tokens)

Total: ~8k tokens, 5-10 minutes

Savings: ~60% tokens, ~75% time

Value: Gemini's web search finds issue Claude couldn't

---

Success Metrics

Time Efficiency

• Without skill: 30-45 minutes (trial and error)
• With skill: 5-10 minutes (consultation + fix)
• Savings: ~75%

Token Efficiency

• Without skill: ~20k tokens (multiple attempts)
• With skill: ~8k tokens (one consultation)
• Savings: ~60%

Error Prevention

• Manual CLI use: 3-5 common errors (flags, parsing, privacy)
• With skill: 0 errors (all handled by commands)
• Prevention: 100%

Quality

• Manual: Risk of not synthesizing (just copying external AI)
• With skill: Forced synthesis via GEMINI.md/codex.md
• Improvement: Guaranteed value-add

---

Why CLI Approach (Not MCP)?

Aspect	MCP Server	CLI Approach
Setup time	4-6 hours	60-75 minutes
Complexity	High (MCP protocol)	Low (bash + CLIs)
Maintenance	Update MCP SDK	Update CLI (rare)
Flexibility	Locked to AIs	Any AI with CLI
Debugging	MCP protocol	Standard bash
Dependencies	MCP SDK, npm	Just CLIs

Winner: CLI approach - 80% less effort, same functionality

---

Troubleshooting

Gemini Not Working

# Check CLI installed
which gemini

# Check API key set
echo $GEMINI_API_KEY

# Test manually
gemini -p "test"

# Check API key valid
curl -H "Content-Type: application/json" \
  -d '{"contents":[{"parts":[{"text":"test"}]}]}' \
  "https://generativelanguage.googleapis.com/v1beta/models/gemini-pro:generateContent?key=$GEMINI_API_KEY"

Codex Not Working

# Check CLI installed
which codex

# Check API key set (if using API key auth)
echo $OPENAI_API_KEY

# Test manually
echo "test" | codex exec - --yolo

# Check OpenAI API (if installed)
openai api models.list

Fresh Claude Not Working

# Fresh Claude uses Task tool (built-in)
# If Task tool fails, check Claude Code CLI status
claude --version

Skill Not Discovered

# Check skill installed
ls -la ~/.claude/skills/multi-ai-consultant

# Check SKILL.md has YAML frontmatter
head -20 ~/.claude/skills/multi-ai-consultant/SKILL.md

# Ask Claude Code to list skills
# It should mention multi-ai-consultant

---

Contributing

Found an issue?

• Document it in Known Issues section
• Include fix/workaround
• Update slash commands to prevent

Adding new AI?

• Create new slash command: commands/consult-newai.md
• Add to router: Update commands/consult-ai.md
• Create template: templates/newai.md (if CLI supports system instructions)
• Update documentation

Improving synthesis?

• Edit templates: templates/GEMINI.md, templates/codex.md
• Test with real consultations
• Measure before/after quality

---

References

External Resources

• Gemini CLI: https://ai.google.dev/gemini-api/docs/cli
• OpenAI Codex: https://www.npmjs.com/package/codex
• OpenAI API: https://platform.openai.com/docs
• Gemini API Pricing: https://ai.google.dev/pricing
• OpenAI Pricing: https://openai.com/pricing

Internal Files

• Planning docs: planning/multi-ai-consultant-*.md
• Slash commands: commands/*.md
• Templates: templates/*
• Scripts: scripts/*

---

License

MIT License - See LICENSE file

---

Last Updated: 2025-11-07 Status: Production Ready Maintainer: Jeremy Dawes | jeremy@jezweb.net | https://jezweb.com.au