← Về thư mục
verl/utils/megatron/tensor_parallel.py
- update_kwargs_with_config · function · L27-L29 — def update_kwargs_with_config(dictionary: Dict, config: ModelParallelConfig)
- get_default_kwargs_for_model_parallel_config · function · L32-L40 — def get_default_kwargs_for_model_parallel_config()
- get_default_model_parallel_config · function · L43-L44 — def get_default_model_parallel_config()
- get_common_default_kwargs_for_parallel_linear · function · L47-L55 — def get_common_default_kwargs_for_parallel_linear()
- get_default_kwargs_for_column_parallel_linear · function · L58-L69 — def get_default_kwargs_for_column_parallel_linear()
- get_default_kwargs_for_row_parallel_linear · function · L72-L74 — def get_default_kwargs_for_row_parallel_linear()
- get_default_kwargs_for_parallel_embedding · function · L77-L83 — def get_default_kwargs_for_parallel_embedding()
- is_tensor_parallel_param · function · L86-L87 — def is_tensor_parallel_param(param)
- get_tensor_parallel_partition_dim · function · L90-L92 — def get_tensor_parallel_partition_dim(param)
- get_tensor_parallel_partition_stride · function · L95-L97 — def get_tensor_parallel_partition_stride(param)
- _VocabParallelEntropy · class · L100-L121 — class _VocabParallelEntropy(torch.autograd.Function)
- forward · method · L103-L115 — def forward(ctx, vocab_parallel_logits: torch.Tensor) -> torch.Tensor
- backward · method · L118-L121 — def backward(ctx, grad_output: torch.Tensor) -> torch.Tensor
- vocab_parallel_entropy · function · L124-L133 — def vocab_parallel_entropy(vocab_parallel_logits: torch.Tensor) -> torch.Tensor
- vocab_parallel_log_probs_from_logits · function · L136-L138 — def vocab_parallel_log_probs_from_logits(logits, labels)
- vocab_parallel_log_probs_from_logits_response_rmpad · function · L141-L165 — def vocab_parallel_log_probs_from_logits_response_rmpad(input_ids, attention_mask, logits_rmpad, response_length)
- vocab_parallel_compute_entropy_loss · function · L168-L184 — def vocab_parallel_compute_entropy_loss(logits, eos_mask)