Skip to content

🐛 fix: fix reward model train seq_cls #3921

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 26, 2025
Merged

Conversation

gaohongkui
Copy link
Contributor

PR type

  • Bug Fix
  • New Feature
  • Document Updates
  • More Models or Datasets Support

PR information

修复了使用奖励模型(reward model)作为基础模型进行多分类训练时的问题。当 reward model 用于序列分类(seq_cls)任务且 num_labels > 1 时,由于模型结构差异可能导致尺寸不匹配的问题。此修复通过自动设置 ignore_mismatched_sizes=True 来解决这个问题。

主要改动:

  • get_model_tokenizer_from_local 函数中,当检测到是 reward model 且用于多分类任务(num_labels > 1)时,自动设置 ignore_mismatched_sizes=True(重新添加 分类头)
  • 添加了相应的警告日志,提醒用户正在使用该配置

Experiment results

不需要实验结果,这是一个功能性修复,确保了在以下场景可以正常工作:

  • 使用 reward model 作为基础模型
  • 进行多分类任务(num_labels > 1)
  • 模型加载时自动处理尺寸不匹配问题

@Jintao-Huang
Copy link
Collaborator

我理解这个修复了 感谢

@Jintao-Huang
Copy link
Collaborator

请merge main分支并执行以下命令进行代码整理

pip install pre-commit
pre-commit run --all-files

@gaohongkui
Copy link
Contributor Author

@Jintao-Huang 已 format 代码
image

@Jintao-Huang Jintao-Huang merged commit a2a858d into modelscope:main Apr 26, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants