add early stopping for hyperopt

[viotemp1]

Summary

Add early stopping for hyperopt

Quick changelog

• Add early stopping for hyperopt

What's new?

Will shorten the hyperopt time when no improvement in Objective

[xmatthias]

I'm sorry, i've somehow completely missed this PR.

For the CI, you'll need to run build_helpers/create_command_partials.py and commit the result - it'll auto-update all doc descriptions to keep (at least this part of) the docs in sync with reality.

i assume in terms of testing, this is best tested with a very small search space, so it exhausts pretty quickly?

[coveralls]

coverage: 94.349% (-0.01%) from 94.362%
when pulling 5a2b3d9 on viotemp1:optuna_addons
into c3f6aa1 on freqtrade:develop.

[xmatthias]

@viotemp1 i don't currently trust this functionality ...

with --epochs 100, --early-stop 5 - it stops after 21 epochs (same with --early-stop=2) - but the objective function is clearly still changing - significantly at that (used with --print-all to see all results)

[viotemp1]

I looked at their source code and it should work.

The default of min_n_trials is 20 in optuna.terminator.Terminator(improvement_evaluator=None, error_evaluator=None, min_n_trials=20).
We can add a warning message when lower than 20 or change the default.

[xmatthias]

well i think even enforcing a minimum of 20 (or ignoring values between 0 and 20) is reasonable.
if your result doesn't improve after 5 epochs - that won't tell you much 😆

[xmatthias]

Great enhancement 👍