Skip to content

[fbsync] Remove unneeded modules after using nn.Module for BetterTransformer #1696

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 3, 2022

Conversation

parmeet
Copy link
Contributor

@parmeet parmeet commented May 2, 2022

Summary:
Pull Request resolved: #1693

Remove unneeded modules after using nn.Module for BetterTransformer

Reviewed By: zrphercule

Differential Revision: D36038830

fbshipit-source-id: 1e0f5c7cf81096cf66cc1afcf15b5e0645c3da03

zrphercule2 and others added 4 commits May 1, 2022 17:00
Summary:
Pull Request resolved: pytorch#1690

This diff created a fast path of using better transformer (torch.nn.TransformerEncoderLayer), with a converter from the existing torchtext transformer encoder layer to better transforomer.
The related tests are added in the following diff.

Reviewed By: parmeet

Differential Revision: D35948440

fbshipit-source-id: e69e12f2dd28edfea3176a10ee3d7d321d50c897
Summary: Fully remove to_better method by rebuild torchtext TransformerEncoderLayer's load_from_state_dict and init. No more redundant params.

Reviewed By: parmeet

Differential Revision: D36020184

fbshipit-source-id: ccdd6da853a86034762b235cd7d5f793876d16c6
…ytorch#1693)

Summary:
Pull Request resolved: pytorch#1693

Remove unneeded modules after using nn.Module for BetterTransformer

Reviewed By: zrphercule

Differential Revision: D36038830

fbshipit-source-id: 1e0f5c7cf81096cf66cc1afcf15b5e0645c3da03
@parmeet parmeet requested a review from zrphercule May 2, 2022 13:38
Copy link
Contributor

@Nayef211 Nayef211 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like all unit tests were failing on this PR. I just kicked off another retry. Let's ensure everything passes before landing :)

@parmeet parmeet merged commit dccb84f into pytorch:main May 3, 2022
@parmeet parmeet deleted the fbsync_D36038830 branch May 3, 2022 03:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants