← Về thư mục
verl/models/llama/megatron/layers/parallel_decoder.py
- ParallelLlamaDecoderLayer · class · L33-L96 — class ParallelLlamaDecoderLayer(nn.Module)
- init · method · L35-L42 — def init(self, config: LlamaConfig, megatron_config: ModelParallelConfig)
- forward · method · L44-L96 — def forward( self, hidden_states: torch.Tensor, attention_mask: Optional[torch.Tensor] = None, position_ids: Optional[torch.LongTensor] = None, ) -> Tuple[torch.FloatTensor, Optional[Tuple[torch.FloatTensor, torch.FloatTensor]]]
- ParallelLlamaDecoderLayerRmPad · class · L99-L146 — class ParallelLlamaDecoderLayerRmPad(nn.Module)
- init · method · L101-L110 — def init(self, config: LlamaConfig, megatron_config: ModelParallelConfig)
- forward · method · L112-L146 — def forward( self, hidden_states: torch.Tensor, position_ids: Optional[torch.LongTensor] = None, sequence_length: int = None, indices: torch.Tensor = None, cu_seqlens: int = None, max_seqlen_in_batch: int = None ) -> Tuple[torch.FloatTensor, Optional[Tuple[torch.FloatTensor, torch.FloatTensor]]]