Open-source vector similarity search for SQL Server

A high-performance vector similarity search extension for SQL Server, inspired by pgvector and powered by FAISS (Facebook AI Similarity Search).

Project Name: sfvector = SQL Server + FAISS + Vector

Version: 0.1.0
Maintainer: robinson

This project brings advanced vector similarity search capabilities to SQL Server, enabling:

• Semantic search for AI/ML applications
• Recommendation systems using embeddings
• Image/document similarity search
• Anomaly detection using vector representations

SQL CLR (Common Language Runtime)

• Native integration with SQL Server
• Can call C++/FAISS via C++/CLI or P/Invoke
• User-defined types, functions, and stored procedures
• Security sandbox limitations

Approach: Hybrid - SQL CLR + Native C++ Library

• C# CLR layer for SQL Server integration
• C++ native library wrapping FAISS
• Clean separation of concerns

• User-defined vector type (VECTOR)
• SQL functions for vector operations
• Index management procedures
• Distance/similarity functions

• FAISS library wrapper
• Index serialization/deserialization
• High-performance search operations
• Memory management

• Unit tests
• Integration tests
• Performance benchmarks

CREATE TABLE documents (
    id INT PRIMARY KEY,
    content NVARCHAR(MAX),
    embedding VECTOR(1536)  -- OpenAI ada-002 dimension
);

• FLAT: Exact search (brute force)
• IVF: Inverted file index (clustering-based)
• HNSW: Hierarchical Navigable Small World graphs
• IVF_FLAT: IVF with flat quantization
• IVF_PQ: IVF with product quantization (compression)

• L2 (Euclidean distance)
• Inner Product (dot product)
• Cosine Similarity

-- Insert vectors
INSERT INTO documents (id, content, embedding)
VALUES (1, 'Hello world', VECTOR('[0.1, 0.2, ...]'));

-- Create FAISS index
EXEC sp_create_vector_index 
    @table = 'documents',
    @column = 'embedding',
    @index_type = 'HNSW',
    @metric = 'L2';

-- Similarity search
SELECT TOP 10 id, content, 
    vector_distance(embedding, VECTOR('[0.1, 0.2, ...]'), 'L2') as distance
FROM documents
ORDER BY embedding <-> VECTOR('[0.1, 0.2, ...]');  -- KNN operator

-- Or using function
SELECT * FROM vector_search(
    'documents',
    'embedding', 
    VECTOR('[0.1, 0.2, ...]'),
    10,  -- top k
    'L2'
);

Three Options for Vector Search:

Built-in VECTOR type introduced in SQL Server 2025.

Pros:

• ✅ Native integration (no CLR or external dependencies)
• ✅ Familiar SQL syntax
• ✅ Official Microsoft support
• ✅ HNSW and DiskANN indexes

Cons:

• ⚠️ Requires SQL Server 2025
• ⚠️ Limited index options vs FAISS
• ⚠️ No GPU support
• ⚠️ Newer technology (less mature)

FAISS-powered vector search for SQL Server 2019+.

Pros:

• ✅ Highest search throughput (~850 QPS on 10K dataset)
• ✅ Works on SQL Server 2019+
• ✅ Most index options (FLAT, HNSW, IVF, IVF-PQ)
• ✅ GPU acceleration available
• ✅ Advanced quantization (PQ, SQ)

Cons:

• ⚠️ Requires CLR and native library deployment
• ⚠️ Higher memory usage
• ⚠️ Community-maintained

Vector extension for PostgreSQL.

Pros:

• ✅ Fastest index builds
• ✅ Lowest memory usage
• ✅ Mature and stable
• ✅ Large community
• ✅ Simple installation

Cons:

• ⚠️ Requires PostgreSQL (not SQL Server)
• ⚠️ Lower search QPS than sfvector
• ⚠️ No GPU support

Choose SQL Server 2025 Native if:

• ✅ You're running SQL Server 2025 or newer
• ✅ You want native integration without CLR
• ✅ You prefer official Microsoft support
• ✅ Basic vector search is sufficient

Choose sfvector (this project) if:

• ✅ You need maximum search performance
• ✅ You're on SQL Server 2019/2022 (can't upgrade to 2025)
• ✅ You want advanced FAISS features (GPU, PQ)
• ✅ You need more index options
• ✅ Search throughput is critical

Choose pgvector if:

• ✅ You're using PostgreSQL
• ✅ You want the most mature solution
• ✅ Fast index builds are important
• ✅ Memory efficiency is critical
• ✅ You prefer simpler deployment

We provide comprehensive benchmarks comparing all three implementations:

# Three-way comparison
cd benchmarks
python run_sql2025_comparison.py \
    --sqlserver-conn "Server=localhost;Database=VectorDB;..." \
    --postgres-conn "host=localhost dbname=vectordb..." \
    --dataset-size 10000

# View results
cat results_sql2025/sql2025_comparison_report.md

See SQL2025_COMPARISON.md for detailed comparison documentation.

• C#: SQL CLR integration (.NET Framework 4.8 or .NET Core/5+)
• C++17: Native FAISS wrapper
• FAISS: Vector similarity search library
• CMake: Build system for native code
• MSBuild/dotnet: Build system for C# code

sfvector/
├── src/
│   ├── SqlServer.VectorSearch/          # C# SQL CLR project
│   │   ├── Types/
│   │   │   └── VectorType.cs           # UDT for VECTOR
│   │   ├── Functions/
│   │   │   ├── DistanceFunctions.cs
│   │   │   └── VectorOperations.cs
│   │   ├── Procedures/
│   │   │   └── IndexManagement.cs
│   │   └── Native/
│   │       └── FaissInterop.cs         # P/Invoke to native lib
│   ├── SqlServer.VectorSearch.Native/   # C++ FAISS wrapper
│   │   ├── include/
│   │   │   └── faiss_wrapper.h
│   │   ├── src/
│   │   │   └── faiss_wrapper.cpp
│   │   └── CMakeLists.txt
│   └── SqlServer.VectorSearch.Tests/
├── benchmarks/                          # Performance testing suite
│   ├── generate_test_data.py
│   ├── run_benchmarks.py
│   ├── sqlserver_benchmarks.sql
│   ├── pgvector_benchmarks.sql
│   └── BENCHMARK_GUIDE.md
├── docs/
│   ├── ARCHITECTURE.md
│   ├── API.md
│   └── DEPLOYMENT.md
├── examples/
│   └── semantic_search_example.sql
├── scripts/
│   ├── build.sh
│   └── deploy.sql
├── LICENSE
└── README.md

• SQL Server 2019+ (with CLR enabled)
• Visual Studio 2019+ or MSBuild
• CMake 3.15+
• FAISS library
• .NET Framework 4.8 or .NET 6+

# Build native library
cd src/SqlServer.VectorSearch.Native
mkdir build && cd build
cmake ..
cmake --build .

# Build CLR assembly
cd ../../SqlServer.VectorSearch
dotnet build

# Deploy to SQL Server
sqlcmd -S localhost -i scripts/deploy.sql

• Project setup and architecture design
• Implement vector UDT in C#
• Create C++ FAISS wrapper
• Implement distance functions (L2, Cosine, IP)
• Implement vector operations (15+ functions)
• Add FLAT index support
• Add HNSW index support
• Add IVF index support
• Implement KNN search
• Add batch operations
• Performance benchmarking suite
• Documentation and examples
• Deploy and test with real FAISS library
• GPU support (optional)
• Product quantization support
• Production deployment guide

• FAISS Documentation
• pgvector
• SQL Server CLR Integration

sfvector is inspired by pgvector and uses FAISS, both excellent open-source projects:

• pgvector: https://github.com/pgvector/pgvector
• FAISS: https://github.com/facebookresearch/faiss

Contributions welcome! Please read CONTRIBUTING.md for guidelines.