[P1] Using T5-family models

[tanguy8001]

Hi,

Thanks for your great work! I was wondering if T5 models are supported by pyreft, and if not, what suggestions would you have to integrate them.

[frankaging]

Hi, to use T5 or other new models, you need to specify the intervening component like
https://github.com/stanfordnlp/pyreft/blob/main/examples/loreft/train.py#L327

You can think of the intervention config as your lora config, which mounts the intervention to a relevant module.