Multi-vector search
Problem
Hi, I'm wondering if pgVector support complex multi-vector search similar to what MS does in https://github.com/microsoft/MSVBASE? Our app is one of those that requires search across multiple vectors and scalers. We're researching new options in addition to Milvus hybrid search. Vbase from MS promises superior performance but it does not look mature enough.
Unverified for your environment
Select your OS to check compatibility.
1 Fix
Implement Multi-Vector Search with pgVector
pgVector, while capable of handling vector embeddings, lacks native support for complex multi-vector search operations that combine multiple vectors and scalars. This limitation arises from the design of pgVector, which primarily focuses on single vector operations. To achieve multi-vector search functionality, additional logic needs to be implemented to combine the results from multiple queries or to use a more advanced querying mechanism.
Awaiting Verification
Be the first to verify this fix
- 1
Define Vector and Scalar Structures
Create a schema in your PostgreSQL database that includes both vector and scalar fields. This will allow you to store and query the necessary data for multi-vector searches.
sqlCREATE TABLE items (id SERIAL PRIMARY KEY, vector_data VECTOR(128), scalar_value FLOAT); - 2
Insert Sample Data
Populate your database with sample data that includes both vector embeddings and scalar values. This will help in testing the multi-vector search functionality.
sqlINSERT INTO items (vector_data, scalar_value) VALUES ('[0.1, 0.2, 0.3, ...]', 1.5); - 3
Implement Multi-Vector Search Logic
Create a function that performs a search across multiple vectors and scalars. This function should combine the results of individual vector searches and apply any necessary filtering based on scalar values.
sqlSELECT * FROM items WHERE vector_data <-> '[0.1, 0.2, 0.3, ...]' < 0.5 AND scalar_value > 1.0; - 4
Optimize Query Performance
Consider adding indexes on the scalar fields and using materialized views to improve the performance of your multi-vector search queries.
sqlCREATE INDEX idx_scalar_value ON items (scalar_value); - 5
Test and Validate Results
Run test queries to ensure that the multi-vector search is returning the expected results. Adjust the logic as necessary based on the outcomes.
sqlSELECT * FROM items WHERE vector_data <-> '[0.1, 0.2, 0.3, ...]' < 0.5 AND scalar_value > 1.0;
Validation
Confirm that the multi-vector search returns accurate results by comparing the output against expected results based on known data. Additionally, monitor the performance of the queries to ensure they meet your application's requirements.
Sign in to verify this fix
Environment
Submitted by
Alex Chen
2450 rep