verl/utils/megatron/optimizer.py

• get_megatron_optimizer · function · L26-L92 — def get_megatron_optimizer( model, config: OptimizerConfig, no_weight_decay_cond=None, scale_lr_cond=None, lr_mult=1.0, check_for_nan_in_loss_and_grad=False, overlap_param_gather=False # add for verl ): # Base optimizer.