Skip to content

Beam Search may generate more than need. #42

@TanZhendong

Description

@TanZhendong

Hi, thank you for your meticulous work.

In the loop of the beam search iterative process, we find that the condition for early stopping is only:

# Early stopping if all beams are completed
if len(active_beams) == 0:
break

However, doing so may likely result in the final number of generated completion_beams exceeding config.n, which is not used for aggregation.

Moreover, for the sake of fairness, the number of beams used for aggregation should be the same as in best of n. Otherwise, their generation budget cannot be considered same.

I believe an additional stopping condition should be added as follows:

        # Early stopping if all beams are completed
        if len(active_beams) == 0:
            break
        
        # Early stopping if completions >= config.n
        if len(completed_beams) >= config.n:
            break

On one hand, it speeds up the beam search through early stopping, and on the other hand, it ensures that the number of generated results does not exceed config.n.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions