dmlcloud.gather_object

dmlcloud.gather_object(obj, dst=0, group=None)

Gathers objects from all ranks in the group to the destination rank.

This is a convenience wrapper around `torch.distributed.gather_object.

Parameters:

• obj (object) – The object to gather. Must be pickable. Only used on ranks that are not the destination rank.

• dst (int) – Destination rank to gather the object to. Default is 0.

• group (ProcessGroup, optional) – The process group to work on. If None, the default process group will be used. Default is None.

Returns:

A list of objects gathered from all ranks. None for ranks that are not the destination rank.

Return type:

List[object]