Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learning
Yibin Wang et al.
arXiv:2508.20751
In short
RL for text-to-image models gets “reward hacked” when tiny score differences are amplified by normalisation. Pref-GRPO instead compares images in pairs and uses win rate as the reward, and the paper adds UniGenBench, a finer-grained text-to-image benchmark.
Why it matters
More stable RL for image generators, plus a better yardstick for them.
Read first
The 4 Field Guide ideas this paper leans on.
Starting from scratch? The full route 9 ideas · basics first
- Machine Learning ✓ understood
Building systems that learn patterns from data instead of following hand-written rules, getting better at a task as they see more examples.
- Neural Network ✓ understood
A model built from layers of simple units, each taking a weighted sum of its inputs and applying a nonlinearity; training adjusts the weights so the whole stack maps inputs to useful outputs.
- Deep Learning ✓ understood
A subset of machine learning that uses neural networks with multiple layers (deep neural networks) to learn hierarchical representations of data.
- Computer Vision ✓ understood
The field of AI that gets computers to extract meaning from images and video: what is in them, where it is, and how it moves.
- Image Generation · read first ✓ understood
Creating new images, often from a text prompt, with generative models such as GANs, VAEs and diffusion models that learn to turn random noise into realistic pictures.
- Reinforcement Learning · read first ✓ understood
Learning through interaction with an environment, receiving rewards or penalties to learn optimal behavior policies.
- Reward · read first ✓ understood
The single number an environment sends back after each action, telling a reinforcement learning agent how good that step was; the agent learns to maximize its long-run total.
- Dataset ✓ understood
A collection of data examples used for training, validating, or testing machine learning models.
- Benchmark · read first ✓ understood
A standardized dataset and task used to compare model performance across different approaches (ImageNet, GLUE, SuperGLUE).
In the frontier
- Rank
- #58 of 100
- Citations
- 90
- as of Aug 9, 2026
- Published
- Aug 2025
Topics: RL for reasoning , Image generation and editing , Agent benchmarks and computer use
Selection: 1kpapers.com by Together AI, most-cited as of Aug 9, 2026