Add bit redop test - #25
wangxicoding wants to merge 1 commit into
Conversation
|
Hi @wangxicoding, have you ever try to test this bitwise operation enabled NCCL with PyTorch distributed? Thank you! |
@GeKeShi Sorry, I didn't. I wanted to apply this to lookup table or sparse communication, but things I did later changed, so I didn't continue to do it. |
Thanks for your reply! I also want to apply this function to communicate sparse tensor. It is a common requirement in distributed setting, but NVIDIA still haven't developed the bitwise operation in All-Reduce. |
for NVIDIA/nccl#240