verl/utils/dataset/rl_dataset.py

• collate_fn · function · L31-L55 — def collate_fn(data_list: list[dict]) -> dict
• RLHFDataset · class · L58-L155 — class RLHFDataset(Dataset)
• init · method · L63-L89 — def init(self, parquet_files: Union[str, List[str]], tokenizer: PreTrainedTokenizer, prompt_key='prompt', max_prompt_length=1024, filter_prompts=True, cache_dir='~/.cache/verl/rlhf', chat_template_func=None, return_raw_chat=False, truncation='error')
• _download · method · L91-L94 — def _download(self)
• _read_files_and_tokenize · method · L96-L115 — def _read_files_and_tokenize(self)
• len · method · L117-L118 — def len(self)
• getitem · method · L120-L155 — def getitem(self, item)