Repository navigation

🧺 [4/N] Refactor `_generate` in GRPO/RLOO: Move `forward_kwargs` outside generation method - #4154

Merged

qgallouedec merged 125 commits into

mainfrom

refactor_generate_4

Oct 17, 2025

Merged

🧺 [4/N] Refactor `_generate` in GRPO/RLOO: Move `forward_kwargs` outside generation method#4154
qgallouedec merged 125 commits into
mainfrom
refactor_generate_4

Commits on Sep 19, 2025
- Refactor image handling: replace `image_split_sizes` with `image_grid_thw` in GRPO and RLOO trainers; update `split_pixel_values_by_grid` to use `image_grid_thw`
  qgallouedec
  committed
  552e899
- simpler
  qgallouedec
  committed
  449ef07
- gfpo
  qgallouedec
  committed
  c8933aa
- multi-image grpo
  qgallouedec
  committed
  229c554
- log with wandb
  qgallouedec
  committed
  3ca6ad5
Commits on Sep 20, 2025
- no vlm reward models
  qgallouedec
  committed
  dcf4b92
- rloo
  qgallouedec
  committed
  30ad7ca
- gfpo
  qgallouedec
  committed
  86cc30b
- fix
  qgallouedec
  committed
  088897b
- test peft
  qgallouedec
  committed
  d2adc63
- fix gfpo
  qgallouedec
  committed
  f4c82bf
- rloo test
  qgallouedec
  committed
  1257796
- peft rloo
  qgallouedec
  committed
  099a39b
- oops
  qgallouedec
  committed
  529add6
- update test
  qgallouedec
  committed
  fc6b11f
- generate method
  qgallouedec
  committed
  ae1f497
- debug
  qgallouedec
  committed
  f998432
- skip failing test
  qgallouedec
  committed
  fa73876
- Merge branch 'main' into drop-image_split_sizes
  qgallouedec
  authored
  52d8bd9
- Merge branch 'drop-image_split_sizes' into multi-image-support
  qgallouedec
  authored
  dfc0d38
- test fixed!
  qgallouedec
  committed
  fc52e68
- Merge branch 'multi-image-support' into generate-method
  qgallouedec
  authored
  4d12aeb
- gfpo
  qgallouedec
  committed
  4fc2b5b
- rm vllm
  qgallouedec
  committed
  b628744
- fix doc
  qgallouedec
  committed
  d3a769f
Commits on Sep 22, 2025
- Merge branch 'main' into drop-image_split_sizes
  qgallouedec
  authored
  e17ec42
- Merge branch 'drop-image_split_sizes' into multi-image-support
  qgallouedec
  authored
  efbb03a
- Merge branch 'main' into multi-image-support
  qgallouedec
  authored
  562c662
- Merge branch 'main' into multi-image-support
  qgallouedec
  authored
  485781c
- update layers to ignore
  qgallouedec
  committed
  05270f8
- clarify image column desc
  qgallouedec
  committed
  1c53094
Commits on Sep 23, 2025
- rm VLM x RM warning
  qgallouedec
  committed
  9b6652e
- Merge branch 'multi-image-support' into generate-method
  qgallouedec
  authored
  c500440
- Merge branch 'main' into generate-method
  qgallouedec
  authored
  a6a8c44
- Merge branch 'main' into generate-method
  qgallouedec
  authored
  d8665e1
- Merge branch 'main' into generate-method
  qgallouedec
  authored
  365d501
Commits on Sep 24, 2025
- Merge branch 'main' into generate-method
  qgallouedec
  authored
  cdb4c76
- same for rloo
  qgallouedec
  committed
  c83e710
- nits style and align
  qgallouedec
  committed
  ec6ad25
- Merge branch 'main' into generate-method
  qgallouedec
  authored
  b4cadde
Commits on Sep 25, 2025
- restart
  qgallouedec
  committed
  b0dceb9
- progress
  qgallouedec
  committed
  ebe32c2
- progress continues
  qgallouedec
  committed
  0213662
- progress again again
  qgallouedec
  committed
  8b3a724
- back to working point
  qgallouedec
  committed
  c1ae6aa
- revert chage data utils
  qgallouedec
  committed
  1a66b43
Commits on Sep 26, 2025
- Merge branch 'main' into generate-method
  qgallouedec
  authored
  2dc69a6
- refactor in grpo
  qgallouedec
  committed
  9435a94
- Merge branch 'main' into refactor_generate
  qgallouedec
  authored
  d3f1d3c
- wrong merge commit
  qgallouedec
  committed
  3d8ea27
- fix num_input_tokens_seen
  qgallouedec
  committed
  27dc958
- getting closer
  qgallouedec
  committed
  53772ef
- consistent naming
  qgallouedec
  committed
  8766fa5
- better
  qgallouedec
  committed
  236b78b
- simplify a bit + comment
  qgallouedec
  committed
  9da4830
- another one
  qgallouedec
  committed
  b3bd0b0
- get prompt ids from generation
  qgallouedec
  committed
  d79b9e1
- remove pad token removal
  qgallouedec
  committed
  8d34d54
- Merge branch 'refactor_generate' into refactor_generate_2
  qgallouedec
  authored
  e770efe
- rely on generator for prompt truncation
  qgallouedec
  committed
  0e2ae34
- revert
  qgallouedec
  committed
  46d8eb7
- rm enforce eager
  qgallouedec
  committed
  11acc75
- rm truncate_with_protected_tokens
  qgallouedec
  committed
  acee7d8
- ensure proper truncation and side
  qgallouedec
  committed
  0b5865e
- rm useless comment
  qgallouedec
  committed
  d8af003
- rm imports
  qgallouedec
  committed
  fc263a3
- requires padding
  qgallouedec
  committed
  35f99fd
- rm truncation test
  qgallouedec
  committed
  8149d05
- move forward_kwargs outside of generate
  qgallouedec
  committed
  9925199
- don't re-prepare data
  qgallouedec
  committed
  48a1c30
- rloo + doc
  qgallouedec
  committed
  55a2480
- Merge branch 'refactor_generate' into refactor_generate_2
  qgallouedec
  authored
  c8041e1
- Merge branch 'refactor_generate_2' into refactor_generate_3
  qgallouedec
  authored
  b8c0c9b
Commits on Sep 27, 2025
- test and doc
  qgallouedec
  committed
  7b7a11d
- gfpo
  qgallouedec
  committed
  c5064d6
- Merge branch 'main' into refactor_generate
  qgallouedec
  authored
  effb41b
- Merge branch 'main' into refactor_generate
  qgallouedec
  authored
  e82bfb4
- Merge branch 'refactor_generate' into refactor_generate_2
  qgallouedec
  authored
  4b9c126
- Merge branch 'refactor_generate_2' into refactor_generate_3
  qgallouedec
  authored
  3f02702
- Merge branch 'refactor_generate_3' into refactor_generate_4
  qgallouedec
  authored
  b0e0279
Commits on Sep 30, 2025
- Merge branch 'main' into refactor_generate_2
  qgallouedec
  authored
  f11759e
- fix vllm client server
  qgallouedec
  committed
  e7aa945
Commits on Oct 1, 2025
- repicate all_prompt_ids
  qgallouedec
  committed
  e164ec5
- Same for RLOO
  qgallouedec
  committed
  49577ad
- fix normal generation path
  qgallouedec
  committed
  5fca5b8
- Merge branch 'refactor_generate_2' into refactor_generate_3
  qgallouedec
  authored
  5cc6af5
- remove vision tokens
  qgallouedec
  committed
  4dce145
- same for rloo
  qgallouedec
  committed
  ddfd3b5
- truncation_side=left
  qgallouedec
  committed
  c434fa2
- rm test_training_vlm_and_prompt_truncation
  qgallouedec
  committed
  377b081
- Merge branch 'main' into refactor_generate_2
  qgallouedec
  authored
  d599c20
Commits on Oct 6, 2025
- 🔣 Fix test: replace `trainer.tokenizer` by `trainer.processing_class` (#4185)
  qgallouedec
  committed
  e82db74
- Fix CI ImportError: FlashAttention2 and decorator order for all parameterized tests (#4176)
  
  albertvillanova
  authored and
  qgallouedec
  committed
  192deb3
- Hotfix wrong formatting of docstrings with blockquote tips (#4187)
  
  albertvillanova
  authored and
  qgallouedec
  committed
  cf9d8e7
- 🌡️ Have vLLM return processed (temperature scaled) log probs (#4163)
  
  YonatanGideoni
  and
  qgallouedec
  committed
  f9c3c3c
- Replace remaining trainer.tokenizer with trainer.processing_class in GRPO test (#4192)
  
  albertvillanova
  authored and
  qgallouedec
  committed
  6489479
- [DOCS] Lora without regret (#4181)
  
  committed
  21a67fc
- [DOCS/FIX] lora without regrets - fix lr (#4207)
  
  burtenshaw
  authored and
  qgallouedec
  committed
  c1e7ad2
- Remove custome_container for building the docs (#4198)
  
  albertvillanova
  authored and
  qgallouedec
  committed
  5d34144
- Remove tokenizer creation from `sft` example script (#4197)
  
  sergiopaniego
  authored and
  qgallouedec
  committed
  ae2a0e7
- Hotfix: Exclude transformers 4.57.0 for Python 3.9 (#4209)
  
  authored and
  qgallouedec
  committed
  6543f51
- Replace unittest with pytest (#4188)
  
  albertvillanova
  authored and
  qgallouedec
  committed
  8319ce0
- Updated vLLM integration guide (#4162)
  
  sergiopaniego
  and
  qgallouedec
  committed
  4fdaa4c
- Remove `Optional` from `processing_class` in `PPOTrainer` (#4212)
  
  sergiopaniego
  authored and
  qgallouedec
  committed
  d258e36
- Replace setup with pyproject and fix packaging unintended modules (#4194)
  
  albertvillanova
  authored and
  qgallouedec
  committed
  7f5b499
- Merge branch 'main' into refactor_generate_2
  qgallouedec
  authored
  df386f9
- Merge branch 'main' into refactor_generate_2
  qgallouedec
  authored
  5b9a6ab
- Merge branch 'refactor_generate_2' into refactor_generate_3
  qgallouedec
  authored
  766bbce
- Merge branch 'refactor_generate_3' into refactor_generate_4
  qgallouedec
  authored
  ac2717f
- Merge branch 'main' into refactor_generate_2
  qgallouedec
  authored
  4a274d5
- Merge branch 'refactor_generate_2' into refactor_generate_3
  qgallouedec
  authored
  db552be
- Merge branch 'refactor_generate_3' into refactor_generate_4
  qgallouedec
  authored
  2c012dc
- style
  qgallouedec
  committed
  a84325c
- Merge branch 'refactor_generate_3' into refactor_generate_4
  qgallouedec
  authored
  34034e7
- token_type_ids and RLOO
  qgallouedec
  committed
  2ce6c1f
- gfpo
  qgallouedec
  committed
  ddf3405
- style
  qgallouedec
  committed
  e3c679c
Commits on Oct 7, 2025
- remove test case for prompt truncation
  qgallouedec
  committed
  ee03478
- Merge branch 'refactor_generate_3' into refactor_generate_4
  qgallouedec
  authored
  ed54e2a
- Merge branch 'main' into refactor_generate_3
  qgallouedec
  authored
  45290c9
- Merge branch 'refactor_generate_3' into refactor_generate_4
  qgallouedec
  authored
  a0ee1e6
Commits on Oct 10, 2025
- Merge branch 'main' into refactor_generate_4
  qgallouedec
  authored
  010fd41
- Merge branch 'main' into refactor_generate_4
  qgallouedec
  authored
  7c55c1d
Commits on Oct 15, 2025
- Merge branch 'main' into refactor_generate_4
  qgallouedec
  authored
  aede65d
Commits on Oct 17, 2025
- Merge branch 'main' into refactor_generate_4
  qgallouedec
  authored
  505bc5a

Commits

Commits on Sep 19, 2025

Commits on Sep 20, 2025

Commits on Sep 22, 2025

Commits on Sep 23, 2025

Commits on Sep 24, 2025

Commits on Sep 25, 2025

Commits on Sep 26, 2025

Commits on Sep 27, 2025

Commits on Sep 30, 2025

Commits on Oct 1, 2025

Commits on Oct 6, 2025

Commits on Oct 7, 2025

Commits on Oct 10, 2025

Commits on Oct 15, 2025

Commits on Oct 17, 2025