Zilliz Introduces VDBBench 1.0 for Realistic Vector Database Benchmarking

Zilliz has launched VDBBench 1.0, an open-source benchmarking platform designed to evaluate vector databases under realistic production conditions, announced in a press release. Unlike traditional benchmarks, VDBBench tests systems during streaming data ingestion, metadata filtering, and concurrent workloads, providing a more accurate reflection of enterprise deployments.

VDBBench 1.0 introduces several new features, including advanced filtering analysis and streaming read/write testing. It uses vectors from state-of-the-art embedding models like OpenAI and Cohere, with dimensions ranging from 768 to 1,536, to mirror current AI workloads. The platform also supports custom datasets, allowing organizations to benchmark using their production data and embedding models.

The platform prioritizes production-focused metrics such as P95/P99 tail latency and sustainable throughput under load, offering a redesigned dashboard with interactive visualizations to help engineers identify performance gaps. VDBBench is available on GitHub, providing organizations with the tools to move beyond misleading benchmarks and customize testing for specific requirements.