PyPI Server Setup

This skill provides guidance for creating local PyPI servers to host and distribute Python packages.

When to Use This Skill

• Setting up a local PyPI repository or package index
• Building Python packages for distribution (wheel/sdist)
• Serving packages via HTTP for pip installation
• Testing package installation from custom index URLs

Environment Reconnaissance (First Step)

Before planning the approach, gather critical environment information:

1. Check Python version: Run python3 --version to determine compatibility constraints

   • Python 3.13+ removed the cgi module, breaking many older tools like pypiserver
   • Plan fallback strategies based on available Python version

2. Check available system tools: Verify what utilities exist

   • Process management: which ps pkill lsof kill
   • Network utilities: which curl wget nc
   • Package tools: pip list to see pre-installed packages

3. Identify port availability: Check if target ports are free before starting servers

Approach Selection

Decision Framework

Condition	Recommended Approach
Python 3.13+	Use python3 -m http.server with proper directory structure
Python 3.12 or earlier	Either pypiserver or http.server works
Simple single-package hosting	http.server is sufficient
Full PyPI mirroring needed	Consider pypiserver or devpi

Recommended Default: Python's Built-in HTTP Server

For most local PyPI hosting tasks, use python3 -m http.server with the correct directory structure. This approach:

• Has no external dependencies
• Works across all Python versions
• Is simpler to configure and debug

Directory Structure Requirements

Pip expects a specific directory structure when using --index-url:

server_root/
└── simple/
    └── <package-name>/
        └── <package-name>-<version>-py3-none-any.whl

Critical Details:

• The simple/ directory must be at the root of the served directory
• Package names in the directory should be normalized (lowercase, hyphens to underscores for some cases)
• The server must be started from the directory containing simple/, not from within it

Building Python Packages

Standard Package Structure

package_name/
├── setup.py
├── pyproject.toml (optional but recommended)
└── package_name/
    ├── __init__.py
    └── module.py

Minimal setup.py Example

from setuptools import setup, find_packages

setup(
    name="package-name",
    version="0.1.0",
    packages=find_packages(),
)

Build Commands

# Build wheel and source distribution
python3 -m pip install build
python3 -m build

# Output appears in dist/

Server Setup Steps

Step 1: Create Directory Structure

mkdir -p pypi-server/simple/packagename/
cp dist/*.whl pypi-server/simple/packagename/

Step 2: Start HTTP Server

cd pypi-server
python3 -m http.server 8080

Important: Start the server from the directory that contains simple/, not from within simple/.

Step 3: Verify Server

Test that the structure is correct:

curl http://localhost:8080/simple/
curl http://localhost:8080/simple/packagename/

Verification Strategy

Pre-Installation Checks

1. Verify server is running: curl -I http://localhost:PORT/
2. Verify simple index exists: curl http://localhost:PORT/simple/
3. Verify package directory exists: curl http://localhost:PORT/simple/packagename/
4. Verify wheel file is accessible: curl -I http://localhost:PORT/simple/packagename/file.whl

Installation Test

pip install --index-url http://localhost:PORT/simple packagename==version

Post-Installation Verification

import packagename
# Test core functionality

Common Pitfalls and Solutions

1. Port Already in Use

Symptom: OSError: [Errno 98] Address already in use

Solutions:

• Use a different port
• Find and kill the existing process:

  import socket
  sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
  sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
  

• Create a custom server script with SO_REUSEADDR option

2. Wrong Directory Structure

Symptom: 404 Not Found when pip tries to access /simple/

Cause: Server started from wrong directory or missing simple/ prefix

Solution: Verify the URL http://localhost:PORT/simple/ returns a directory listing

3. Python 3.13+ Compatibility

Symptom: ModuleNotFoundError: No module named 'cgi' when using pypiserver

Cause: Python 3.13 removed the deprecated cgi module

Solution: Use python3 -m http.server instead of pypiserver

4. Process Management Without Standard Tools

Symptom: Cannot find or kill background processes (ps/pkill not available)

Solutions:

• Use Python's process management:

  import subprocess
  proc = subprocess.Popen(['python3', '-m', 'http.server', '8080'])
  # Save proc.pid for later termination
  

• Use /proc filesystem on Linux: ls /proc/*/cmdline
• Track PIDs explicitly when starting background processes

5. Shell Compatibility

Symptom: source: not found when sourcing files

Cause: Using source in sh shell (source is a bash feature)

Solution: Use . instead of source for POSIX compatibility:

. ./venv/bin/activate

6. Package Name Normalization

Symptom: Package not found despite correct structure

Cause: Pip normalizes package names (underscores to hyphens, lowercase)

Solution: Use lowercase names and be consistent with hyphens/underscores

Custom Server Script (Robust Alternative)

For better process control and port reuse, consider a custom server script:

#!/usr/bin/env python3
import http.server
import socketserver
import os

PORT = 8080
DIRECTORY = "/path/to/pypi-server"

os.chdir(DIRECTORY)

class ReuseAddrServer(socketserver.TCPServer):
    allow_reuse_address = True

handler = http.server.SimpleHTTPRequestHandler
with ReuseAddrServer(("", PORT), handler) as httpd:
    print(f"Serving at port {PORT}")
    httpd.serve_forever()

Checklist Before Starting

• Verified Python version and tool compatibility
• Created correct directory structure with simple/ prefix
• Built package successfully (wheel exists in dist/)
• Copied wheel to correct location under simple/packagename/
• Confirmed target port is available
• Plan for process management (how to stop the server later)