dmlcloud.scale_lr

dmlcloud.scale_lr(base_lr, world_size=None)

Scales the learning rate based on the world size.

Parameters:

base_lr (float) – The base learning rate.
world_size (int, optional) – The number of processes. Default is the global world size.

Returns:

The scaled learning rate.

Return type:

float