ulysses raise NotImplementedError #4100

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

aishoot opened this issue May 7, 2025 · 0 comments

aishoot commented May 7, 2025 •

edited

Loading

使用 Qwen2.5-1.5B-Instruct 进行32k长序列训练测试，有报错如下：

ms-swift/swift/trainers/sequence_parallel/ulysses.py", line 192, in single_all_to_all
  5898	[rank4]:     raise NotImplementedError
  5899	[rank4]: NotImplementedError
  5900	[rank6]: Traceback (most recent call last):

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment