Search results

GRPOTrainer environment_factory / tools is broken for VLMs whose tools return images

Status: Open.

#5663 In huggingface/trl;

· evancasey opened on Apr 27, 2026

TPOTrainer.evaluate() returns NaN eval_loss while training loss is finite

Status: Open.

#5662 In huggingface/trl;

· emredeveloper opened on Apr 27, 2026

DPOTrainer does not work for multimodal Gemma 4

Status: Open.

#5646 In huggingface/trl;

· dgawlik opened on Apr 26, 2026

Does PPOTrainer use mini_batch_size to update parameters?

Status: Open.

#5645 In huggingface/trl;

· zhiang28 opened on Apr 26, 2026

DAPO loss is not normalized by gradient_accumulation_steps

Status: Open.

#5619 In huggingface/trl;

· haotang1995 opened on Apr 22, 2026

Feature Request: Allow passing dataset-provided sample weights to DPOTrainer

Status: Open.

#5613 In huggingface/trl;

· sirano1004 opened on Apr 21, 2026

[docs] GRPO model name mismatch across docs + missing GRPO tab in OOM troubleshooting

Status: Open.

#5611 In huggingface/trl;

· dhruvnigam93 opened on Apr 21, 2026

Training collapses when token embeddings are not padded to a multiple of 64

Status: Open.

#5608 In huggingface/trl;

· Xxxxsir opened on Apr 20, 2026

[Bug] Zero-std reward groups produce spurious KL gradients when beta > 0 in GRPO/RLOO

Status: Open.

#5588 In huggingface/trl;

· SwayamInSync opened on Apr 18, 2026

Add entropy regularization (e.g., initial_entropy_coef) to GRPOTrainer for improved exploration

Status: Open.

#5584 In huggingface/trl;

· lailanelkoussy opened on Apr 17, 2026

fix: DataCollatorForRewardModelingDataset checks margin only on examples[0], causing silent data loss after shuffle

Status: Open.

#5539 In huggingface/trl;

· kuishou68 opened on Apr 14, 2026

OnlineDPOTrainer._generate_vllm_server() flattens vllm-serve completion_ids twice

Status: Open.

#5514 In huggingface/trl;

· JohnGiorgi opened on Apr 10, 2026