name	comfyui
description	Generates images and videos using ComfyUI node-based workflows. Use when creating AI-generated assets, text-to-image, text-to-video, image-to-video, running Stable Diffusion, Flux, HunyuanVideo, or when user mentions "comfy," "ComfyUI," "generate image," "generate video," "AI art," "diffusion model," or needs visual content for courses/projects.

Generating Assets with ComfyUI

This skill helps you generate images and videos using ComfyUI, the powerful node-based generative AI platform.

Prerequisites

ComfyUI must be running locally or accessible via network. Default: http://127.0.0.1:8188

Quick Setup:

# Install comfy-cli
pip install comfy-cli

# Install ComfyUI
comfy install

# Launch ComfyUI server
comfy launch

Core Capabilities

Text-to-Image: Generate images from text prompts (Flux, SDXL, SD1.5)
Text-to-Video: Generate videos from text (HunyuanVideo, Wan, LTX-Video)
Image-to-Video: Animate images into video sequences
Image-to-Image: Transform/enhance existing images
Upscaling: Enhance resolution with ESRGAN/SwinIR
LoRA Integration: Apply style/concept fine-tunes

Workflow Execution Process

Step 1: Check ComfyUI Status

curl -s http://127.0.0.1:8188/system_stats | jq .

Step 2: Choose Workflow Template

Select based on the user's needs:

Image Generation: See ./workflow-templates.md#text-to-image
Video Generation: See ./workflow-templates.md#text-to-video
Image Enhancement: See ./workflow-templates.md#upscaling

Step 3: Submit Workflow via API

Use the provided Node.js scripts in ./scripts/ or submit directly:

// See ./scripts/comfy-api.mjs for full implementation
const response = await fetch('http://127.0.0.1:8188/prompt', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ prompt: workflowJson, client_id: clientId })
});

Step 4: Monitor Progress

Connect to WebSocket for real-time updates:

ws://127.0.0.1:8188/ws?clientId={client_id}

Step 5: Retrieve Results

# Get execution history
curl http://127.0.0.1:8188/history/{prompt_id}

# Download generated image
curl "http://127.0.0.1:8188/view?filename={filename}&subfolder={subfolder}&type=output" -o output.png

Quick Commands

Task	Script
Generate Image	`node ~/.claude/skills/generating-with-comfyui/scripts/comfy-api.mjs generate-image "prompt"`
Generate Video	`node ~/.claude/skills/generating-with-comfyui/scripts/comfy-api.mjs generate-video "prompt"`
Check Status	`node ~/.claude/skills/generating-with-comfyui/scripts/comfy-api.mjs status`
List Models	`node ~/.claude/skills/generating-with-comfyui/scripts/comfy-api.mjs list-models`

API Endpoints Reference

Endpoint	Method	Purpose
`/prompt`	POST	Submit workflow to queue
`/prompt`	GET	Get current queue status
`/queue`	GET	View execution queue
`/queue`	POST	Clear/manage queue
`/history`	GET	Get execution history
`/history/{id}`	GET	Get specific prompt history
`/interrupt`	POST	Stop current execution
`/object_info`	GET	List all available nodes
`/models`	GET	List available models
`/view`	GET	Retrieve generated outputs
`/upload/image`	POST	Upload input images

Model Recommendations

For Images

Flux.1 Dev/Schnell: Latest high-quality, fast generation
SDXL 1.0: Excellent quality, wide compatibility
SD 1.5: Fast, vast LoRA ecosystem

For Videos

HunyuanVideo 1.5: 8.3B params, consumer GPU friendly, 480p-1080p
Wan 2.1: High quality text-to-video
LTX-Video: Fast video generation

For Enhancement

4x-UltraSharp: Best general upscaler
RealESRGAN x4: Realistic photo enhancement
SwinIR: High-quality restoration

Directory Structure

ComfyUI expects models in specific locations:

ComfyUI/
├── models/
│   ├── checkpoints/     # Main model files (.safetensors)
│   ├── vae/             # VAE models
│   ├── loras/           # LoRA fine-tunes
│   ├── controlnet/      # ControlNet models
│   ├── clip/            # CLIP text encoders
│   ├── upscale_models/  # Upscaler models
│   └── diffusion_models/ # Flux/video models
├── input/               # Input images
├── output/              # Generated outputs
└── custom_nodes/        # Extensions

Installing Models

# Download checkpoint
comfy model download <url> models/checkpoints

# Or manually with curl/wget
curl -L "https://huggingface.co/model/file.safetensors" -o models/checkpoints/model.safetensors

Custom Nodes Installation

# Install via comfy-cli
comfy node install ComfyUI-VideoHelperSuite
comfy node install ComfyUI-Manager

# Or git clone to custom_nodes/
cd ComfyUI/custom_nodes
git clone https://github.com/kijai/ComfyUI-HunyuanVideoWrapper

Troubleshooting

Server Not Running

# Check if ComfyUI is running
curl -s http://127.0.0.1:8188/ > /dev/null && echo "Running" || echo "Not running"

# Start it
comfy launch

Out of Memory

Reduce resolution (512x512 for SD, 1024x1024 for SDXL)
Enable --lowvram or --cpu flags
Use FP8 quantized models

Missing Model

# List available models
curl http://127.0.0.1:8188/models | jq .

Best Practices

Always check server status before submitting workflows
Use unique client_id for WebSocket connections
Monitor queue to avoid duplicate submissions
Save successful workflows as JSON templates
Use seed values for reproducible results

Reference Documentation

Workflow templates: ./workflow-templates.md
Video generation specifics: ./video-generation-guide.md
Full API script: ./scripts/comfy-api.mjs

comfyui

Install Skill

SKILL.md