Web Navigation Strategies for Playwright MCP

Strategic guide for systematic web exploration using Playwright MCP tools. This skill provides navigation patterns, not executable code.

Reading Depth Decision Framework

1. Analyze User Intent

Query Analysis Decision Tree:
├── Keywords: "간단히, 요약, 훑어, quick, summary, overview"
│   → **QUICK MODE**
│   → Extract: Titles + First paragraphs only
│   → Pages: 30-50 (fast scan)
│
├── Keywords: "자세히, 상세, 모든, 댓글, detailed, comprehensive, everything"
│   → **DEEP MODE**
│   → Extract: Full content + Comments + Metadata + Related links
│   → Pages: 5-10 (thorough read)
│
└── No specific keywords
    → **STANDARD MODE**
    → Extract: Main content + Basic metadata
    → Pages: 15-20 (balanced)

2. Execute Based on Depth

Quick Mode Strategy

// Get titles and summaries only
mcp_playwright.navigate(url)
links = mcp_playwright.query_selector_all('h2 a, h3 a')
for (link in links.slice(0, 50)) {
    mcp_playwright.click(link)
    title = mcp_playwright.get_text('h1')
    summary = mcp_playwright.get_text('p:first-of-type')
    mcp_playwright.go_back()
}

Standard Mode Strategy

// Get full main content
mcp_playwright.navigate(url)
links = mcp_playwright.query_selector_all('article a')
for (link in links.slice(0, 20)) {
    mcp_playwright.click(link)
    mcp_playwright.wait_for_load_state('networkidle')
    content = mcp_playwright.get_text('main, article, .content')
    mcp_playwright.go_back()
}

Deep Mode Strategy

// Get everything including discussions
mcp_playwright.navigate(url)
links = mcp_playwright.query_selector_all('article a')
for (link in links.slice(0, 10)) {
    mcp_playwright.click(link)
    mcp_playwright.wait_for_load_state('networkidle')
    
    // Main content
    content = mcp_playwright.get_text('article, main')
    
    // Comments
    comments = mcp_playwright.query_selector_all('.comment')
    for (comment in comments) {
        comment_text = mcp_playwright.get_text(comment)
    }
    
    // Metadata
    author = mcp_playwright.get_text('.author, .writer')
    date = mcp_playwright.get_text('time, .date')
    
    // Related links
    related = mcp_playwright.query_selector_all('.related a')
    
    mcp_playwright.go_back()
}

Core Navigation Patterns

Pattern 1: List-Detail Navigation

Use for: Blog indexes, search results, news listings

Step-by-Step Process:
1. Navigate to list page
2. Collect all article links
3. For each link:
   a. Click link
   b. Wait for page load
   c. Extract content
   d. Navigate back
   e. Continue with next

MCP Commands:

// Navigate to list
mcp_playwright.navigate('https://blog.com/posts')

// Get all article links
article_links = mcp_playwright.query_selector_all('article a[href]')

// Process each
for (link in article_links) {
    // Enter detail page
    mcp_playwright.click(link)
    mcp_playwright.wait_for_load_state('networkidle')
    
    // Extract content
    text = mcp_playwright.get_text('article')
    
    // Return to list
    mcp_playwright.go_back()
    mcp_playwright.wait_for_load_state('networkidle')
}

Pattern 2: Pagination Traversal

Use for: Multi-page results, forums, archives

Process:
1. Process current page
2. Find "Next" button
3. Click and repeat
4. Stop when no more pages

MCP Commands:

while (true) {
    // Process current page
    articles = mcp_playwright.query_selector_all('article')
    for (article in articles) {
        // Extract content
    }
    
    // Check for next page
    next_button = mcp_playwright.query_selector('a.next, [rel="next"]')
    if (!next_button) break
    
    // Go to next page
    mcp_playwright.click(next_button)
    mcp_playwright.wait_for_load_state('networkidle')
}

Pattern 3: Infinite Scroll

Use for: Social media, modern blogs, dynamic content

// Scroll and load pattern
current_height = 0
while (true) {
    // Get current height
    new_height = mcp_playwright.evaluate('document.body.scrollHeight')
    
    if (new_height == current_height) break
    
    // Scroll to bottom
    mcp_playwright.evaluate('window.scrollTo(0, document.body.scrollHeight)')
    mcp_playwright.wait_for_timeout(2000)  // Wait for content load
    
    current_height = new_height
}

// Then extract all loaded content
all_articles = mcp_playwright.query_selector_all('article')

Site-Specific Selector Patterns

Korean Portals

Naver (네이버)

// Search results
list_selector: '.blog_list li'
link_selector: '.api_txt_lines.total_tit'
content_selector: '.se-main-container'
comment_selector: '.u_cbox_contents'

// Blog specific
blog_content: '#postViewArea, .se-main-container'
blog_title: '.se-title-text, .pcol1'

Daum (다음)

// Search results  
list_selector: '.list_info'
link_selector: '.f_link_b, .tit_main'
content_selector: '.article_view'

// Tistory blogs
tistory_content: '.entry-content, .article-view'
tistory_comments: '.comment-list'

Brunch

article_list: '.wrap_article_list li'
article_link: 'a.link_post'
article_content: '.wrap_body'

Global Sites

Medium

article_list: 'article'
article_link: 'h2 a, h3 a'
article_content: 'section.pw-post-body'
clap_count: '[aria-label*="clap"]'

Reddit

post_list: '[data-testid="post-container"]'
post_content: '[data-click-id="text"]'
comments: '.Comment'
upvotes: '[aria-label*="upvote"]'

Generic Patterns

// Works on most blogs
article_containers: 'article, .post, .entry, .blog-post'
title_selectors: 'h1, .title, .post-title, .entry-title'
content_selectors: 'main, .content, .post-content, .entry-content'
comment_selectors: '.comments, .comment-list, #comments'
author_selectors: '.author, .by-author, .writer, [rel="author"]'
date_selectors: 'time, .date, .published, .post-date'

Intelligent Navigation Decisions

When to Use Each Pattern

Selector Priority Strategy

1. Try specific selector first
   → '.se-main-container' (Naver specific)
   
2. Fallback to semantic HTML
   → 'article', 'main'
   
3. Try common class patterns
   → '.content', '.post-content'
   
4. Last resort: get all text
   → 'body' (then clean up)

MCP Command Reference

Essential Commands

// Navigation
mcp_playwright.navigate(url)
mcp_playwright.go_back()
mcp_playwright.go_forward()
mcp_playwright.reload()

// Element Selection
mcp_playwright.query_selector(selector)        // First match
mcp_playwright.query_selector_all(selector)    // All matches

// Interaction
mcp_playwright.click(element_or_selector)
mcp_playwright.type(selector, text)
mcp_playwright.scroll_to(x, y)

// Content Extraction
mcp_playwright.get_text(selector)
mcp_playwright.get_attribute(selector, attribute)
mcp_playwright.get_url()
mcp_playwright.get_title()

// Waiting
mcp_playwright.wait_for_selector(selector)
mcp_playwright.wait_for_load_state('networkidle')
mcp_playwright.wait_for_timeout(milliseconds)

// JavaScript Execution
mcp_playwright.evaluate(javascript_code)

Practical Examples

Example 1: Quick News Scan

// User: "네이버 뉴스 제목만 빠르게 훑어줘"
mcp_playwright.navigate('https://news.naver.com')
headlines = mcp_playwright.query_selector_all('.news_tit')
for (headline in headlines.slice(0, 30)) {
    title = mcp_playwright.get_text(headline)
    // Store title
}

Example 2: Blog Deep Dive

// User: "이 블로그 최신글 5개 댓글까지 자세히 봐줘"
mcp_playwright.navigate('https://blog.example.com')
posts = mcp_playwright.query_selector_all('.post-link')

for (post in posts.slice(0, 5)) {
    mcp_playwright.click(post)
    
    // Get everything
    title = mcp_playwright.get_text('h1')
    content = mcp_playwright.get_text('article')
    author = mcp_playwright.get_text('.author')
    date = mcp_playwright.get_text('.date')
    
    // Get all comments
    comments = mcp_playwright.query_selector_all('.comment')
    for (comment in comments) {
        text = mcp_playwright.get_text(comment)
    }
    
    mcp_playwright.go_back()
}

Example 3: Forum Thread Analysis

// User: "레딧 스레드 전체 토론 분석해줘"
mcp_playwright.navigate(reddit_url)

// Main post
main_post = mcp_playwright.get_text('[data-click-id="text"]')
upvotes = mcp_playwright.get_text('[aria-label*="upvote"]')

// All comments with nested replies
all_comments = mcp_playwright.query_selector_all('.Comment')
for (comment in all_comments) {
    text = mcp_playwright.get_text(comment)
    // Parse nested structure
}

Error Handling Strategies

Common Issues and Solutions

1. Selector Not Found

   // Try multiple selectors
   content = mcp_playwright.query_selector('article') 
           || mcp_playwright.query_selector('main')
           || mcp_playwright.query_selector('.content')
           || mcp_playwright.query_selector('body')
   

2. Dynamic Content Not Loaded

   // Wait strategies
   mcp_playwright.wait_for_load_state('networkidle')
   mcp_playwright.wait_for_selector('.content', timeout=10000)
   mcp_playwright.wait_for_timeout(2000)  // Last resort
   

3. Navigation Failed

   try {
       mcp_playwright.go_back()
   } catch {
       // Fallback: navigate directly
       mcp_playwright.navigate(list_page_url)
   }
   

Best Practices

DO ✅

• Wait for content to load before extraction
• Use specific selectors when possible
• Handle pagination and infinite scroll
• Extract metadata (author, date) when available
• Respect rate limits with waits between requests

DON'T ❌

• Don't assume selectors work on all sites
• Don't skip wait_for_load_state
• Don't extract without checking element exists
• Don't ignore error handling
• Don't process too many pages in deep mode

Decision Flow Summary

User Request
    ↓
Analyze Keywords → Determine Depth (Quick/Standard/Deep)
    ↓
Identify Site Type → Choose Selectors
    ↓
Select Pattern → (List-Detail/Pagination/Scroll)
    ↓
Execute with MCP Commands
    ↓
Handle Errors → Fallback Strategies

Reference

For detailed patterns and selectors by site type, see:

• references/site-selectors.md - Comprehensive selector database
• references/navigation-patterns.md - Advanced navigation techniques