← Về thư mục
verl/utils/megatron_utils.py
- get_model · function · L34-L116 — def get_model(model_provider_func, model_type=ModelType.encoder_or_decoder, wrap_with_ddp=True)
- unwrap_model · function · L122-L134 — def unwrap_model(model, module_instances=ALL_MODULE_WRAPPER_CLASSNAMES)
- convert_config · function · L140-L177 — def convert_config(hf_config: PretrainedConfig, megatron_config) -> TransformerConfig
- init_megatron_optim_config · function · L185-L195 — def init_megatron_optim_config(optim_config: Dict) -> OptimizerConfig
- init_model_parallel_config · function · L201-L212 — def init_model_parallel_config(config: DictConfig) -> ModelParallelConfig: # TODO(sgm): check how to disable megatron timers
- FakeTimers · class · L215-L223 — class FakeTimers
- init · method · L218-L220 — def init(self)
- call · method · L222-L223 — def call(self, args: Any, *kwds: Any) -> Any
- offload_megatron_param_and_grad · function · L226-L238 — def offload_megatron_param_and_grad(module_list: nn.ModuleList, offload_grad=False, hybrid_engine=None)
- load_megatron_param_and_grad · function · L241-L253 — def load_megatron_param_and_grad(module_list: nn.ModuleList, device_id, load_grad=False, hybrid_engine=None)