-
Notifications
You must be signed in to change notification settings - Fork 128
Open
Description
Hi, thank you for your meticulous work.
In the loop of the beam search iterative process, we find that the condition for early stopping is only:
search-and-learn/src/sal/search/beam_search.py
Lines 151 to 153 in 1cc70e7
| # Early stopping if all beams are completed | |
| if len(active_beams) == 0: | |
| break |
However, doing so may likely result in the final number of generated
completion_beams exceeding config.n, which is not used for aggregation.
Moreover, for the sake of fairness, the number of beams used for aggregation should be the same as in best of n. Otherwise, their generation budget cannot be considered same.
I believe an additional stopping condition should be added as follows:
# Early stopping if all beams are completed
if len(active_beams) == 0:
break
# Early stopping if completions >= config.n
if len(completed_beams) >= config.n:
break
On one hand, it speeds up the beam search through early stopping, and on the other hand, it ensures that the number of generated results does not exceed config.n.
Metadata
Metadata
Assignees
Labels
No labels