nncore.optim

Wrapper

class nncore.optim.lamb.Lamb(*args: Any, **kwargs: Any)[source]

Lamb Optimizer introduced in [1].

Parameters:

  • params (iterable) – Iterable of parameters to optimize or dicts defining parameter groups.

  • lr (float, optional) – The learning rate. Default: 1e-3.

  • betas (Tuple[float, float], optional) – The coefficients used for computing running averages of gradient and its square. Default: (0.9, 0.999).

  • eps (float, optional) – The term added to the denominator to improve numerical stability. Default: 1e-6.

  • weight_decay (float, optional) – The L2 normalization penalty. Default: 0.

References

  1. You et al. (https://arxiv.org/abs/1904.00962)

Builder

nncore.optim.builder.build_optimizer(cfg, *args, **kwargs)[source]

Build an optimizer from a dict. This method searches for optimizers in OPTIMIZERS first, and then fall back to torch.optim.

Parameters:

cfg (dict) – The config of the optimizer.

Returns:

The constructed optimizer.

Return type:

optim.Optimizer