← Về thư mục
- AllGatherPPModel · class · L35-L208 — class AllGatherPPModel
- init · method · L37-L80 — def init(self, model_provider) -> None
- _build_param_buffer · method · L82-L86 — def _build_param_buffer(self, pp_rank)
- _build_param_references · method · L88-L90 — def _build_param_references(self, pp_rank, maintain_weight=False)
- _load_params_to_cuda · method · L92-L100 — def _load_params_to_cuda(self, pp_rank, to_empty=False)
- _offload_params_to_cpu · method · L102-L110 — def _offload_params_to_cpu(self, pp_rank, to_empty=False)
- load_params_to_cuda · method · L112-L116 — def load_params_to_cuda(self, to_empty=False)
- allgather_params · method · L118-L125 — def allgather_params(self)
- forward · method · L127-L144 — def forward(self, inputs, *kwargs)
- call · method · L146-L147 — def call(self, inputs, *kwargs)
- eval · method · L149-L151 — def eval(self)
- train · method · L153-L155 — def train(self)
- offload_params_to_cpu · method · L157-L161 — def offload_params_to_cpu(self, to_empty=False)
- get_all_params · method · L163-L184 — def get_all_params(self)
- update_this_rank_models · method · L186-L188 — def update_this_rank_models(self, new_models)
- this_rank_models · method · L191-L192 — def this_rank_models(self)
- pp_size · method · L195-L196 — def pp_size(self)
- pp_rank · method · L199-L200 — def pp_rank(self)
- pp_group · method · L203-L204 — def pp_group(self)
- pp_models · method · L207-L208 — def pp_models(self)
- MegatronVLLMShardingManager · class · L238-L410 — class MegatronVLLMShardingManager(BaseShardingManager)
- init · method · L240-L265 — def init(self, module: AllGatherPPModel, inference_engine: LLM, model_config, layer_name_mapping)
- default_tp_concat_fn · method · L267-L316 — def default_tp_concat_fn(self, name, param, infer_params, model_config)
- _post_process_params · method · L318-L343 — def _post_process_params(self, params)
- enter · method · L345-L358 — def enter(self): # create a new cuda space for parameters not in this pp rank
- exit · method · L360-L376 — def exit(self, exc_type, exc_value, traceback): # offload parameters doesn't belong to this pp rank
- preprocess_data · method · L378-L392 — def preprocess_data(self, data: DataProto) -> DataProto: # prompts are identical for each training tp. We select for each inference tp
- postprocess_data · method · L394-L410 — def postprocess_data(self, data: DataProto) -> DataProto
- get_micro_data_parallel_group · function · L418-L420 — def get_micro_data_parallel_group()
- get_micro_data_parallel_world_size · function · L423-L424 — def get_micro_data_parallel_world_size()
- get_micro_data_parallel_rank · function · L427-L428 — def get_micro_data_parallel_rank()