Skip to content

[Question]: 基于uie-base进行微调时，如何增加模型输入的最大长度，直接在微调命令中修改max_seq_length训练会直接报错 #6327

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
Spider4U opened this issue Jul 7, 2023 · 1 comment
Closed

[Question]: 基于uie-base进行微调时，如何增加模型输入的最大长度，直接在微调命令中修改max_seq_length训练会直接报错 #6327

Spider4U opened this issue Jul 7, 2023 · 1 comment
Assignees
@wawltor
Labels
question Further information is requested triage

Comments

@Spider4U
Copy link

Spider4U commented Jul 7, 2023

请提出你的问题

如何增加模型输入的最大长度，直接在微调命令中修改max_seq_length训练会直接报错

uie-max-seq
@Spider4U Spider4U added the question Further information is requested label Jul 7, 2023
@github-actions github-actions bot added the triage label Jul 7, 2023
@w5688414
Copy link
Contributor

w5688414 commented May 7, 2024

uie-base的默认最大长度是512，如果需要延长最大长度，需要修改max_position_embeddings，但这样的话超出512长度的position embeddings会被随机初始化，会影响效果。

https://github.com/PaddlePaddle/PaddleNLP/blob/ae0bea9e8580bbedea96b45d4153be7a8b884d74/paddlenlp/transformers/ernie/modeling.py#L71C20-L71C43

@paddle-bot paddle-bot bot closed this as completed May 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wawltor wawltor

Labels
question Further information is requested triage
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@w5688414 @wawltor @Spider4U