KTO训练，1张卡没问题，8张卡出现问题：Tensor Tensor dtypes: BFloat16vs Float

[qq941134965]

感觉是通信的问题？怎么解决

[qq941134965]

swift版本3.2.2
trl 0.16.1

[qq941134965]

看起来在进入训练之前报的错：
[rank7]: RuntimeError: Detected mismatch between collectives on ranks. Rank 7 is running collective: CollectiveFingerPrint(SequenceNumber=5, OpType=_ALLGATHER_BASE, TensorShape=[1], TensorDtypes=Float, TensorDeviceTypes=TensorOptions(dtype=float (default), device=cuda, layout=Strided (default), requires_grad=false (default), pinned_memory=false (default), memory_format=(nullopt))), but Rank 0 is running collective: CollectiveFingerPrint(SequenceNumber=5, OpType=_ALLGATHER_BASE, TensorShape=[1], TensorDtypes=BFloat16, TensorDeviceTypes=TensorOptions(dtype=float (default), device=cuda, layout=Strided (default), requires_grad=false (default), pinned_memory=false (default), memory_format=(nullopt))).Collectives differ in the following aspects: Tensor Tensor dtypes: Floatvs BFloat16

[hjh0119]

Please provide a minimal reproducible script.