Skip to content

Conversation

@vfdev-5
Copy link
Collaborator

@vfdev-5 vfdev-5 commented Dec 25, 2020

Description:

Will enable the following:

events = Events.STARTED(args.start_validation) | Events.COMPLETED | Events.EPOCH_COMPLETED(every=3)

trainer = ...
@trainer.on(events)
def run_validation(trainer):
    # do something

To simplify this implementation in a real use-case :

https://github.com/huggingface/transfer-learning-conv-ai/blob/d4c760731e8f062abd3d61ccdc8fa29ccb5e3485/train.py#L212-L215

trainer.add_event_handler(Events.EPOCH_COMPLETED, lambda _: evaluator.run(val_loader))
if args.n_epochs < 1:
    trainer.add_event_handler(Events.COMPLETED, lambda _: evaluator.run(val_loader))
if args.eval_before_start:
    trainer.add_event_handler(Events.STARTED, lambda _: evaluator.run(val_loader))

which can be

e = Events.EPOCH_COMPLETED
if args.n_epochs < 1:
    e |= Events.COMPLETED
if args.eval_before_start:
    e |= Events.STARTED
trainer.add_event_handler(e, lambda _: evaluator.run(val_loader))

and with the PR will be

e = Events.EPOCH_COMPLETED | Events.COMPLETED(args.n_epochs < 1) | Events.STARTED(args.eval_before_start)
@trainer.on(e):
    evaluator.run(val_loader)

TODO:

  • Maybe, implementation can be done differently without a custom event_filter.
  • How about Events.EPOCH_COMPLETED(args.condition, every=3) ?

Check list:

  • New tests are added (if a new feature is added)
  • New doc strings: description and/or example code are in RST format
  • Documentation is updated (if required)

@vfdev-5 vfdev-5 marked this pull request as draft December 25, 2020 10:52
@vfdev-5
Copy link
Collaborator Author

vfdev-5 commented Dec 25, 2020

@sdesrozis any comments on the idea ? At first I was thinking that this is good. Now, the implementation (not yet committed) would require some _SkippedEvent class and we can potentially do

Events.EPOCH_COMPLETED(args.condition)  -> if True: Events.EPOCH_COMPLETED else _SkippedEvent (which will be ignored at `Engine.add_event_handler`)
Events.EPOCH_COMPLETED(args.condition, every=3)  -> if True: Events.EPOCH_COMPLETED(every=3) else _SkippedEvent

and I'm hesitating about that ...

EDIT: anyway, this is not a priority ... we can wait with this PR.

@sdesrozis
Copy link
Contributor

I think that is a good PR and having a way to skip events is interesting and surely mandatory.

Btw the hard issue is to avoid handler to be triggered multiple times at the same event. Maybe this should help...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants