Skip to content

When using MultiQueryExpander, if the model's response contains empty newlines, the query expansion fails and returns the original query unchanged. The current implementation doesn't properly handle or filter out empty lines from the model's response. #3345

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kaijunyang opened this issue May 27, 2025 · 0 comments · May be fixed by #3347

Comments

@kaijunyang
Copy link

Bug description
When using MultiQueryExpander, if the model's response contains empty newlines, the query expansion fails and returns the original query unchanged. The current implementation doesn't properly handle or filter out empty lines from the model's response.

Environment

  • Spring AI version: 1.0.0
  • Java version: 17
  • Model/API used: deepseek-reasoner
    • Spring Boot version: 3.4.6

Steps to reproduce

  1. Configure a MultiQueryExpander with numberOfQueries(3)
  2. Call expand() with any query
  3. Have the AI model return a response that includes empty newlines between valid queries
  4. Observe that the original query is returned instead of the expanded queries

Expected behavior
The expander should:

  1. Filter out empty lines from the model's response
  2. Return the valid expanded queries as long as there are enough non-empty variants
  3. Only fall back to the original query if there aren't enough valid expanded queries

Actual behavior
The expander fails when encountering empty newlines in the response, even when there are enough valid query variants present.

Minimal Complete Reproducible example

MultiQueryExpander queryExpander = MultiQueryExpander.builder()
        .chatClientBuilder(this.chatClient.mutate())
        .includeOriginal(false)
        .numberOfQueries(3)
        .build();
return queryExpander.expand(new Query("How to run a Spring Boot app?"));

Proposed solution​
The split("\n") operation should be followed by filtering out empty strings. Here's the suggested fix:

var queryVariants = Arrays.stream(response.split("\n"))
        .filter(StringUtils::hasText)
        .toList();

if (CollectionUtils.isEmpty(queryVariants) || this.numberOfQueries > queryVariants.size()) {
    logger.warn(
            "Query expansion result does not contain the requested {} variants. Returning the input query unchanged.",
            this.numberOfQueries);
    return List.of(query);
}

Additional context​
This is particularly important because:

  1. LLMs often include formatting newlines in their responses
  2. The current behavior causes valid expansions to be discarded unnecessarily
  3. The fix would make the expander more robust to normal model output variations
kaijunyang pushed a commit to kaijunyang/spring-ai that referenced this issue May 27, 2025
…g-projects#3345

- Filter out empty strings from query expansion results
- Optimize the logic for checking the number of query variants
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant