Python Coding

This skill is applicable for writing production intent python code

Core Principles

Applicable to production code. Less applicable to scripts, but use as much as it makes sense balancing cleanliness/simplicity with complexity/additional code

1. Prioritize KISS, SOLID and DRY principles
2. Utilize uv for all project management, venv, package installation, builds, execution etc.
3. Always use the latest Python practices (current version is 3.14) and packages.
4. Write module and function docstrings aiming for 100% coverage (verified by interrogate automatically in pre-commit)
5. Use type annotations maximally and run type checking always (with pyrefly)
6. Use logging (with structlog)
7. Fail Fast: Validate inputs and data (with pydantic) early at function boundaries
8. Write tests and aim for at least 70% coverage (with pytest and coverage)
9. Rely on ruff for formatting, import sorting and linting.
10. • Resource Management Use context managers (with statements) for files, connections, locks

Details on each below:

In-Line Documentation

• Follow Numpy docstring format for function docstrings. Except, do not add parameter types (as we already use type annotation and it would violate DRY causing confusion/errors)
• Module docstring to follow the format:

"""
Title:

Author: < AI Name >

Description:

Usage: <include basic examples>

Notes:

"""

• Be explicit and complete without being overly verbose.

Typing

• Maximize type annotations for all functions/methods
• Use pyrefly for type checking

Logging

• Use structured logging with structlog
• Centralize logging configuration with the dictConfig pattern
• Output to both file and terminal
• Follow the basic format: "%(asctime)s - %(levelname)s - %(message)s"
• Color the level name in outputs
  • Critical: Deep Red
  • Error: Light Red
  • Warning: Yellow
  • Info: Blue
  • Debug: Grey
• Rotate log files by time at the appropriate cadence
• Logger naming: create a hierarchy of loggers using dots in the names
• Logging Security
  • Never log sensitive data (passwords, tokens, PII)
  • Sanitize or redact sensitive fields before logging
  • Be careful with debug logs in production

Data Validation and Input Sanitization

• Treat all external input as untrusted (API requests, file uploads, user input)

• Use Pydantic models to validate and coerce types at system boundaries

• Sanitize inputs before using in:

  • SQL queries (use parameterized queries, ORMs)
  • Shell commands (avoid shell=True, use lists)
  • File paths (validate, use pathlib.Path.resolve())
  • HTML/JavaScript (escape properly)

• Aim for early Validation

• Validate at system boundaries (API endpoints, file inputs, user inputs). Don't let invalid data propagate through the system

• Make validators reusable and composable

• Fail fast with clear error messages

• Use assertions for internal consistency checks (developer errors, not user errors)

• Donts: Validate after processing/storage, Mixing validation with transformation. Over-validating trusted internal data

Error Handling

Use Standard Exceptions First

• Prefer built-in exceptions: ValueError, TypeError, KeyError, FileNotFoundError
• Let third-party exceptions bubble up: httpx.HTTPError, pydantic.ValidationError

Custom Exceptions: Only When Needed

Create custom exceptions only when:

• You need different handling logic at call sites (different catch blocks → different actions)
• The error represents a specific domain concept (e.g., DuplicateArticleError, RateLimitError)
• You're building a public API or library where users shouldn't catch internal exceptions
• Keep it simple - avoid deep hierarchies

When to Catch vs Propagate

• Catch when you can handle the error meaningfully (retry, fallback, user-friendly message)
• Propagate when the calling code is better positioned to handle it
• Re-raise with context using raise NewException(...) from original_exception to preserve stack trace

Error Messages

• Include specific field names and invalid values
• Provide actionable guidance when possible
• Example: ValueError(f"Invalid status '{status}'. Must be one of: {VALID_STATUSES}")

Error Context with Structured Logging

• Use structured logging to capture rich context instead of encoding it in exception types
• Include relevant fields: url, status_code, article_id, feed_name, etc.
• Set exc_info=True to capture full tracebacks

Testing

• Write tests as soon as possible, before code ideally (if following TDD) or after code is written.
• Name tests with the naming convention test_*.py
• Prefer when appropriate, Pytest with parametrized (@pytest.mark.parametrize) or randomized inputs( with hypothesis) complimentarily:
  • When to Use Each Use Parametrize For:
    • Known edge cases you must handle
    • Regression tests (specific bugs)
    • Documentation of expected behavior
    • Business rule examples
    • Quick debugging (run specific case) Use Hypothesis For:
    • Finding unknown edge cases
    • Testing invariants (properties that always hold)
    • Stress testing with many inputs
    • Discovering missing validation
  • Practical Workflow
    1. Start with parametrize for obvious cases
    2. Add hypothesis to find what you missed
    3. When hypothesis finds a bug, add it as a parametrized test for regression
    4. Keep both - parametrize documents intent, hypothesis keeps hunting
• Aim for Minimum 70% coverage (soft guideline), measure with pytest-cov
• Test file naming: test_<module>.py
• Test both valid and invalid cases
• Test boundary conditions explicitly

Security Best Practices

Secret Management

• Never hardcode secrets (API keys, passwords, tokens) in source code
• Use environment variables for configuration: os.getenv("API_KEY")
• Use .env files for local development (add to .gitignore)
• For production, use secret management services (Google Secret Manager, AWS Secrets Manager etc.)
• Validate that required secrets are present at startup

SQL Injection Prevention

• Always use parameterized queries or ORMs

• Never concatenate user input into SQL strings

• Example with parameterization:

  # UNSAFE
  cursor.execute(f"SELECT * FROM users WHERE id = {user_id}")
  
  # SAFE
  cursor.execute("SELECT * FROM users WHERE id = ?", (user_id,))
  

Command Injection Prevention

• Avoid subprocess.run(shell=True)
• Pass commands as lists: subprocess.run(["ls", "-l", directory])
• If shell=True is necessary, sanitize inputs rigorously

Development Tools

Prefer the following tools

Project management should be done declaratively through uv commands (which in turn will write into pyproject.toml and the uv lock file).

Command Reference

Available here .claude/skills/python-coding/references/commands.md

Package Preferences

Prefer:

• httpx over requests