Title: Milvus GPU (CAGRA) benchmark stuck at “Milvus optimizing before search” Description #615
Description
I’m benchmarking Milvus GPU (CAGRA index) using VectorDBBench, and the benchmark seems to hang indefinitely during the “optimizing before search” phase.
Issue: Benchmark stuck at “Milvus optimizing before search”
Milvus log :
...
failed to create index, C Runtime Exception: => failed to build index, raft inner error at ...VectorMemIndex.cpp:276
segcore error[segcoreCode=2004]
The process freezes after showing this log message:
2025-10-16 07:08:58,237 | INFO: Milvus optimizing before search (milvus.py:145) (2752617)
It has been running for more than 2 hours without progressing to the search stage.
Observations:
GPU utilization stays low (nvidia-smi shows near-idle usage)
No new logs from Milvus container after this message
The benchmark does not terminate or error out
Steps to reproduce:
Start Milvus GPU standalone:
wget https://github.com/milvus-io/milvus/releases/download/v2.6.3/milvus-standalone-docker-compose-gpu.yml -O docker-compose.yml
docker compose up -d
Start the benchmark from VectorDBBench UI.
Observe that it stops at the “optimizing before search” phase and never completes.
Expected:
Benchmark should move from optimization to search and finish for all dataset sizes.
Actual:
It remains stuck in optimization indefinitely.
Environment
OS: Ubuntu 22.04
Python: 3.12 (venv)
VectorDBBench: 1.0.10
Milvus: 2.6.3
GPU: NVIDIA RTX (please specify your model)
Milvus Port: 19530
Datasets Tested: 100k, 250k, 1M, 5M