Vector Databases
When to Use This Skill
Use this skill when:
- Choosing between vector database options
- Designing semantic/similarity search systems
- Optimizing vector search performance
- Understanding ANN algorithm trade-offs
- Scaling vector search infrastructure
- Implementing hybrid search (vectors + filters)
Keywords: vector database, embeddings, vector search, similarity search, ANN, approximate nearest neighbor, HNSW, IVF, FAISS, Pinecone, Weaviate, Milvus, Qdrant, Chroma, pgvector, cosine similarity, semantic search
Vector Database Comparison
Managed Services
| Database |
Strengths |
Limitations |
Best For |
| Pinecone |
Fully managed, easy scaling, enterprise |
Vendor lock-in, cost at scale |
Enterprise production |
| Weaviate Cloud |
GraphQL, hybrid search, modules |
Complexity |
Knowledge graphs |
| Zilliz Cloud |
Milvus-based, high performance |
Learning curve |
High-scale production |
| MongoDB Atlas Vector |
Existing MongoDB users |
Newer feature |
MongoDB shops |
| Elastic Vector |
Existing Elastic stack |
Resource heavy |
Search platforms |
Self-Hosted Options
| Database |
Strengths |
Limitations |
Best For |
| Milvus |
Feature-rich, scalable, GPU support |
Operational complexity |
Large-scale production |
| Qdrant |
Rust performance, filtering, easy |
Smaller ecosystem |
Performance-focused |
| Weaviate |
Modules, semantic, hybrid |
Memory usage |
Knowledge applications |
| Chroma |
Simple, Python-native |
Limited scale |
Development, prototyping |
| pgvector |
PostgreSQL extension |
Performance limits |
Postgres shops |
| FAISS |
Library, not DB, fastest |
No persistence, no filtering |
Research, embedded |
Selection Decision Tree
Need managed, don't want operations?
├── Yes → Pinecone (simplest) or Weaviate Cloud
└── No (self-hosted)
└── Already using PostgreSQL?
├── Yes, <1M vectors → pgvector
└── No
└── Need maximum performance at scale?
├── Yes → Milvus or Qdrant
└── No
└── Prototyping/development?
├── Yes → Chroma
└── No → Qdrant (balanced choice)
ANN Algorithms
Algorithm Overview
Exact KNN:
• Search ALL vectors
• O(n) time complexity
• Perfect accuracy
• Impractical at scale
Approximate NN (ANN):
• Search SUBSET of vectors
• O(log n) to O(1) complexity
• Near-perfect accuracy
• Practical at any scale
HNSW (Hierarchical Navigable Small World)
Layer 3: ○───────────────────────○ (sparse, long connections)
│ │
Layer 2: ○───○───────○───────○───○ (medium density)
│ │ │ │ │
Layer 1: ○─○─○─○─○─○─○─○─○─○─○─○─○ (denser)
│││││││││││││││││││││││
Layer 0: ○○○○○○○○○○○○○○○○○○○○○○○○○ (all vectors)
Search: Start at top layer, greedily descend
• Fast: O(log n) search time
• High recall: >95% typically
• Memory: Extra graph storage
HNSW Parameters:
| Parameter |
Description |
Trade-off |
M |
Connections per node |
Memory vs. recall |
ef_construction |
Build-time search width |
Build time vs. recall |
ef_search |
Query-time search width |
Latency vs. recall |
IVF (Inverted File Index)
Clustering Phase:
┌─────────────────────────────────────────┐
│ Cluster vectors into K centroids │
│ │
│ ● ● ● ● │
│ /│\ /│\ /│\ /│\ │
│ ○○○○○ ○○○○○ ○○○○○ ○○○○○ │
│ Cluster 1 Cluster 2 Cluster 3 Cluster 4│
└─────────────────────────────────────────┘
Search Phase:
1. Find nprobe nearest centroids
2. Search only those clusters
3. Much faster than exhaustive
IVF Parameters:
| Parameter |
Description |
Trade-off |
nlist |
Number of clusters |
Build time vs. search quality |
nprobe |
Clusters to search |
Latency vs. recall |
IVF-PQ (Product Quantization)
Original Vector (128 dim):
[0.1, 0.2, ..., 0.9] (128 × 4 bytes = 512 bytes)
PQ Compressed (8 subvectors, 8-bit codes):
[23, 45, 12, 89, 56, 34, 78, 90] (8 bytes)
Memory reduction: 64x
Accuracy trade-off: ~5% recall drop
Algorithm Comparison
| Algorithm |
Search Speed |
Memory |
Build Time |
Recall |
| Flat/Brute |
Slow (O(n)) |
Low |
None |
100% |
| IVF |
Fast |
Low |
Medium |
90-95% |
| IVF-PQ |
Very fast |
Very low |
Medium |
85-92% |
| HNSW |
Very fast |
High |
Slow |
95-99% |
| HNSW+PQ |
Very fast |
Medium |
Slow |
90-95% |
When to Use Which
< 100K vectors:
└── Flat index (exact search is fast enough)
100K - 1M vectors:
└── HNSW (best recall/speed trade-off)
1M - 100M vectors:
├── Memory available → HNSW
└── Memory constrained → IVF-PQ or HNSW+PQ
> 100M vectors:
└── Sharded IVF-PQ or distributed HNSW
Distance Metrics
Common Metrics
| Metric |
Formula |
Range |
Best For |
| Cosine Similarity |
A·B / (||A|| ||B||) |
[-1, 1] |
Normalized embeddings |
| Dot Product |
A·B |
(-∞, ∞) |
When magnitude matters |
| Euclidean (L2) |
√Σ(A-B)² |
[0, ∞) |
Absolute distances |
| Manhattan (L1) |
Σ|A-B| |
[0, ∞) |
High-dimensional sparse |
Metric Selection
Embeddings pre-normalized (unit vectors)?
├── Yes → Cosine = Dot Product (use Dot, faster)
└── No
└── Magnitude meaningful?
├── Yes → Dot Product
└── No → Cosine Similarity
Note: Most embedding models output normalized vectors
→ Dot product is usually the best choice
Filtering and Hybrid Search
Pre-filtering vs Post-filtering
Pre-filtering (Filter → Search):
┌─────────────────────────────────────────┐
│ 1. Apply metadata filter │
│ (category = "electronics") │
│ Result: 10K of 1M vectors │
│ │
│ 2. Vector search on 10K vectors │
│ Much faster, guaranteed filter match │
└─────────────────────────────────────────┘
Post-filtering (Search → Filter):
┌─────────────────────────────────────────┐
│ 1. Vector search on 1M vectors │
│ Return top-1000 │
│ │
│ 2. Apply metadata filter │
│ May return < K results! │
└─────────────────────────────────────────┘
Hybrid Search Architecture
Query: "wireless headphones under $100"
│
┌─────┴─────┐
▼ ▼
┌───────┐ ┌───────┐
│Vector │ │Filter │
│Search │ │ Build │
│"wire- │ │price │
│less │ │< 100 │
│head- │ │ │
│phones"│ │ │
└───────┘ └───────┘
│ │
└─────┬─────┘
▼
┌───────────┐
│ Combine │
│ Results │
└───────────┘
Metadata Index Design
| Metadata Type |
Index Strategy |
Query Example |
| Categorical |
Bitmap/hash index |
category = "books" |
| Numeric range |
B-tree |
price BETWEEN 10 AND 50 |
| Keyword search |
Inverted index |
tags CONTAINS "sale" |
| Geospatial |
R-tree/geohash |
location NEAR (lat, lng) |
Scaling Strategies
Sharding Approaches
Naive Sharding (by ID):
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Shard 1 │ │ Shard 2 │ │ Shard 3 │
│ IDs 0-N │ │IDs N-2N │ │IDs 2N-3N│
└─────────┘ └─────────┘ └─────────┘
Query → Search ALL shards → Merge results
Semantic Sharding (by cluster):
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Shard 1 │ │ Shard 2 │ │ Shard 3 │
│ Tech │ │ Health │ │ Finance │
│ docs │ │ docs │ │ docs │
└─────────┘ └─────────┘ └─────────┘
Query → Route to relevant shard(s) → Faster!
Replication
┌─────────────────────────────────────────┐
│ Load Balancer │
└─────────────────────────────────────────┘
│ │ │
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│Replica 1│ │Replica 2│ │Replica 3│
│ (Read) │ │ (Read) │ │ (Read) │
└─────────┘ └─────────┘ └─────────┘
│ │ │
└───────────┼───────────┘
│
┌─────────┐
│ Primary │
│ (Write) │
└─────────┘
Scaling Decision Matrix
| Scale (vectors) |
Architecture |
Replication |
| < 1M |
Single node |
Optional |
| 1-10M |
Single node, more RAM |
For HA |
| 10-100M |
Sharded, few nodes |
Required |
| 100M-1B |
Sharded, many nodes |
Required |
| > 1B |
Sharded + tiered |
Required |
Performance Optimization
Index Build Optimization
| Optimization |
Description |
Impact |
| Batch insertion |
Insert in batches of 1K-10K |
10x faster |
| Parallel build |
Multi-threaded index construction |
2-4x faster |
| Incremental index |
Add to existing index |
Avoids rebuild |
| GPU acceleration |
Use GPU for training (IVF) |
10-100x faster |
Query Optimization
| Optimization |
Description |
Impact |
| Warm cache |
Keep index in memory |
10x latency reduction |
| Query batching |
Batch similar queries |
Higher throughput |
| Reduce dimensions |
PCA, random projection |
2-4x faster |
| Early termination |
Stop when "good enough" |
Lower latency |
Memory Optimization
Memory per vector:
┌────────────────────────────────────────┐
│ 1536 dims × 4 bytes = 6KB per vector │
│ │
│ 1M vectors: │
│ Raw: 6GB │
│ + HNSW graph: +2-4GB (M-dependent) │
│ = 8-10GB total │
│ │
│ With PQ (64 subquantizers): │
│ 1M vectors: ~64MB │
│ = 100x reduction │
└────────────────────────────────────────┘
Operational Considerations
Backup and Recovery
| Strategy |
Description |
RPO/RTO |
| Snapshots |
Periodic full backup |
Hours |
| WAL replication |
Write-ahead log streaming |
Minutes |
| Real-time sync |
Synchronous replication |
Seconds |
Monitoring Metrics
| Metric |
Description |
Alert Threshold |
| Query latency p99 |
99th percentile latency |
> 100ms |
| Recall |
Search accuracy |
< 90% |
| QPS |
Queries per second |
Capacity dependent |
| Memory usage |
Index memory |
> 80% |
| Index freshness |
Time since last update |
Domain dependent |
Index Maintenance
┌─────────────────────────────────────────┐
│ Index Maintenance Tasks │
├─────────────────────────────────────────┤
│ • Compaction: Merge small segments │
│ • Reindex: Rebuild degraded index │
│ • Vacuum: Remove deleted vectors │
│ • Optimize: Tune parameters │
│ │
│ Schedule during low-traffic periods │
└─────────────────────────────────────────┘
Common Patterns
Multi-Tenant Vector Search
Option 1: Namespace/Collection per tenant
┌─────────────────────────────────────────┐
│ tenant_1_collection │
│ tenant_2_collection │
│ tenant_3_collection │
└─────────────────────────────────────────┘
Pro: Complete isolation
Con: Many indexes, operational overhead
Option 2: Single collection + tenant filter
┌─────────────────────────────────────────┐
│ shared_collection │
│ metadata: { tenant_id: "..." } │
│ Pre-filter by tenant_id │
└─────────────────────────────────────────┘
Pro: Simpler operations
Con: Requires efficient filtering
Real-Time Updates
Write Path:
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Write │ │ Buffer │ │ Merge │
│ Request │───▶│ (Memory) │───▶│ to Index │
└─────────────┘ └─────────────┘ └─────────────┘
Strategy:
1. Buffer writes in memory
2. Periodically merge to main index
3. Search: main index + buffer
4. Compact periodically
Embedding Versioning
Version 1 embeddings ──┐
│
Version 2 embeddings ──┼──▶ Parallel indexes during migration
│
│ ┌─────────────────────┐
└───▶│ Gradual reindexing │
│ Blue-green switch │
└─────────────────────┘
Cost Estimation
Storage Costs
Cost = (vectors × dimensions × bytes × replication) / GB × $/GB/month
Example:
10M vectors × 1536 dims × 4 bytes × 3 replicas = 184 GB
At $0.10/GB/month = $18.40/month storage
Note: Memory (for serving) costs more than storage
Compute Costs
Factors:
• QPS (queries per second)
• Latency requirements
• Index type (HNSW needs more RAM)
• Filtering complexity
Rule of thumb:
• 1M vectors, HNSW, <50ms latency: 16GB RAM
• 10M vectors, HNSW, <50ms latency: 64-128GB RAM
• 100M vectors: Distributed system required
Related Skills
rag-architecture - Using vector databases in RAG systemsllm-serving-patterns - LLM inference with vector retrievalml-system-design - End-to-end ML pipeline designestimation-techniques - Capacity planning for vector systems
Version History
- v1.0.0 (2025-12-26): Initial release - Vector database patterns for systems design
Last Updated
Date: 2025-12-26