Prompt Engineering

When to Use This Skill

Use this skill when:

• Prompt Engineering tasks - Working on design, test, and version prompts with systematic evaluation and optimization strategies
• Planning or design - Need guidance on Prompt Engineering approaches
• Best practices - Want to follow established patterns and standards

Overview

Prompt engineering is the systematic design, testing, and optimization of prompts for large language models. Good prompts significantly impact model performance, reliability, and cost.

Prompt Design Patterns

Pattern Catalog

Prompt Structure Template

# Prompt: [Task Name]

## System Prompt
[Role definition and behavioral constraints]

## User Prompt Template
[Template with {placeholders}]

## Few-Shot Examples (if applicable)
### Example 1
Input: [Example input]
Output: [Expected output]

### Example 2
Input: [Example input]
Output: [Expected output]

## Output Format
[Expected structure, JSON schema if applicable]

## Guardrails
- [Constraint 1]
- [Constraint 2]

Prompt Components

System Prompt Best Practices

┌─────────────────────────────────────────────────────────────────┐
│                    SYSTEM PROMPT STRUCTURE                       │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  1. ROLE DEFINITION                                             │
│     "You are a [role] that [primary function]"                  │
│                                                                  │
│  2. CONTEXT                                                     │
│     Background information, domain knowledge                     │
│                                                                  │
│  3. INSTRUCTIONS                                                │
│     Clear, specific directions                                   │
│                                                                  │
│  4. OUTPUT FORMAT                                               │
│     Structure, schema, examples                                  │
│                                                                  │
│  5. CONSTRAINTS                                                 │
│     What NOT to do, boundaries                                  │
│                                                                  │
│  6. EXAMPLES (Optional)                                         │
│     Demonstrate expected behavior                                │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

Example System Prompt

You are a customer service assistant for TechCorp, specializing in product support.

CONTEXT:
- TechCorp sells software products (ProductA, ProductB, ProductC)
- Support hours are 9 AM - 5 PM EST
- Escalation is needed for billing issues and account closures

INSTRUCTIONS:
1. Greet the customer professionally
2. Identify their issue category
3. Provide relevant solutions or resources
4. Offer escalation if needed

OUTPUT FORMAT:
- Keep responses under 150 words
- Use bullet points for multiple items
- Include relevant KB article links

CONSTRAINTS:
- Never share internal pricing
- Don't make promises about refunds
- Don't provide legal advice
- Refer technical issues beyond scope to engineering

Chain-of-Thought Patterns

Standard CoT

Let's think through this step by step:

1. First, identify the key information...
2. Next, consider the constraints...
3. Then, apply the relevant formula...
4. Finally, calculate the answer...

Therefore, the answer is: [result]

Structured CoT

## Analysis

### Given Information
- [Fact 1]
- [Fact 2]

### Reasoning Steps
1. [Step 1 with explanation]
2. [Step 2 with explanation]
3. [Step 3 with explanation]

### Conclusion
[Final answer with confidence level]

Prompt Testing Framework

Test Categories

Evaluation Metrics

Test Case Template

# Prompt Test: [Test Name]

## Prompt Version
Version: 1.2.0
Git Hash: abc123

## Test Case
| ID | Input | Expected | Actual | Pass |
|----|-------|----------|--------|------|
| TC-001 | [Input] | [Expected] | [Actual] | ✓/✗ |
| TC-002 | [Input] | [Expected] | [Actual] | ✓/✗ |

## Metrics
| Metric | Target | Actual |
|--------|--------|--------|
| Accuracy | >90% | 92% |
| Latency | <2s | 1.5s |
| Tokens | <500 | 420 |

Prompt Versioning

Version Control Strategy

prompts/
├── customer-service/
│   ├── v1.0.0/
│   │   ├── system.txt
│   │   ├── template.txt
│   │   ├── examples.json
│   │   └── metadata.yaml
│   ├── v1.1.0/
│   │   └── ...
│   └── changelog.md
├── summarization/
│   └── ...
└── registry.yaml

Metadata Schema

# metadata.yaml
name: customer-service-prompt
version: 1.2.0
description: Customer service response generation
model:
  recommended: gpt-4o
  compatible:
    - gpt-4o
    - gpt-4o-mini
    - claude-3-5-sonnet
parameters:
  temperature: 0.7
  max_tokens: 500
  top_p: 1.0
metrics:
  accuracy: 0.92
  latency_p95_ms: 1500
  avg_tokens: 420
created: 2024-12-15
author: data-team
tags:
  - production
  - customer-facing

C# Prompt Management

public class PromptManager
{
    private readonly IPromptRepository _repository;

    public async Task<Prompt> GetPrompt(
        string name,
        string version = "latest",
        CancellationToken ct = default)
    {
        var prompt = await _repository.GetAsync(name, version, ct);

        return new Prompt
        {
            SystemMessage = prompt.System,
            Template = prompt.Template,
            Examples = prompt.Examples,
            Parameters = new PromptParameters
            {
                Temperature = prompt.Metadata.Parameters.Temperature,
                MaxTokens = prompt.Metadata.Parameters.MaxTokens
            }
        };
    }

    public string Render(Prompt prompt, Dictionary<string, string> variables)
    {
        var rendered = prompt.Template;

        foreach (var (key, value) in variables)
        {
            rendered = rendered.Replace($"{{{key}}}", value);
        }

        return rendered;
    }
}

// Usage with Semantic Kernel
var promptManager = new PromptManager(repository);
var prompt = await promptManager.GetPrompt("customer-service", "1.2.0");

var kernel = Kernel.CreateBuilder()
    .AddAzureOpenAIChatCompletion(deployment, endpoint, apiKey)
    .Build();

var function = kernel.CreateFunctionFromPrompt(
    prompt.Template,
    new OpenAIPromptExecutionSettings
    {
        Temperature = prompt.Parameters.Temperature,
        MaxTokens = prompt.Parameters.MaxTokens
    });

var result = await kernel.InvokeAsync(function, new KernelArguments
{
    ["customer_name"] = "John",
    ["issue"] = "Password reset"
});

Optimization Techniques

Token Optimization

Prompt Compression

public class PromptOptimizer
{
    public OptimizedPrompt Optimize(string prompt, OptimizationOptions options)
    {
        var optimized = prompt;
        var savings = 0;

        if (options.RemoveWhitespace)
        {
            var before = CountTokens(optimized);
            optimized = Regex.Replace(optimized, @"\s+", " ");
            savings += before - CountTokens(optimized);
        }

        if (options.UseAbbreviations)
        {
            optimized = ApplyAbbreviations(optimized);
        }

        if (options.CompressExamples)
        {
            optimized = CompressExamples(optimized, options.MaxExamples);
        }

        return new OptimizedPrompt
        {
            Content = optimized,
            OriginalTokens = CountTokens(prompt),
            OptimizedTokens = CountTokens(optimized),
            SavingsPercent = savings * 100.0 / CountTokens(prompt)
        };
    }
}

Validation Checklist

• Clear role and context defined
• Instructions are specific and unambiguous
• Output format specified
• Constraints and guardrails included
• Few-shot examples provided (if needed)
• Test cases cover happy path and edge cases
• Adversarial testing completed
• Version control in place
• Performance metrics measured
• Token usage optimized

Integration Points

Inputs from:

• Business requirements → Task definition
• ai-safety-planning skill → Guardrail requirements

Outputs to:

• token-budgeting skill → Cost estimation
• agentic-workflow-design skill → Agent prompts
• Application code → Prompt templates