Skip to content

[Bug]: resize_token_embedding仍然存在bug,bart,pegasus无效,其他生成模型最后一层lm_head不会被resize #6008

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
1 task done
xiaotingyun opened this issue May 23, 2023 · 1 comment
Assignees
Labels
bug Something isn't working triage

Comments

@xiaotingyun
Copy link

软件环境

- paddlepaddle:2.4.0
- paddlepaddle-gpu: 2.4.0
- paddlenlp: 2.5.2

重复问题

  • I have searched the existing issues

错误描述

resize_token_embedding方法仍然存在问题,在bart和pegasus中无效,只更新share,但实际使用的encoder和decoder的embedding层不会更新,最后一层lm_head也不会更新,可能是因为bart和pegasus的encoder和decoder没有set_input_embeddings方法的原因,在t5和gpt中不会更新最后一层lm_head。

稳定复现步骤 & 代码

复现代码如下
https://aistudio.baidu.com/aistudio/projectdetail/6232319?contributionType=1&sUid=581675&shared=1&ts=1684831403394
版本号:resize_token bug

@xiaotingyun xiaotingyun added the bug Something isn't working label May 23, 2023
@sijunhe sijunhe self-assigned this May 29, 2023
@w5688414
Copy link
Contributor

w5688414 commented May 8, 2024

请升级一下paddle和paddlenlp的版本试试,现在已经支持了。

def resize_token_embeddings(self, new_num_tokens: Optional[int] = None) -> nn.Embedding:

@paddle-bot paddle-bot bot closed this as completed May 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage
Projects
None yet
Development

No branches or pull requests

4 participants