Vision & Multimodal Aug 2025 · #58 most cited · 90 citations

Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learning

Yibin Wang et al.

arXiv:2508.20751

In short

RL for text-to-image models gets “reward hacked” when tiny score differences are amplified by normalisation. Pref-GRPO instead compares images in pairs and uses win rate as the reward, and the paper adds UniGenBench, a finer-grained text-to-image benchmark.

Why it matters

More stable RL for image generators, plus a better yardstick for them.

Read first

The 4 Field Guide ideas this paper leans on.

Vision & Multimodal · Landmark Image Generation Creating new images from scratch or from text descriptions using generative models (GANs, diffusion models, VAEs). Agents & RL · Landmark Reinforcement Learning Learning through interaction with an environment, receiving rewards or penalties to learn optimal behavior policies. Agents & RL · Landmark Reward A scalar feedback signal indicating how good an action was, used to train reinforcement learning agents. Evaluation · Standard Benchmark A standardized dataset and task used to compare model performance across different approaches (ImageNet, GLUE, SuperGLUE).

Starting from scratch? The full route 9 ideas · basics first

Machine Learning ✓ understood
Building systems that learn patterns from data instead of following hand-written rules, getting better at a task as they see more examples.
Neural Network ✓ understood
A computational model inspired by biological neural networks, consisting of interconnected nodes (neurons) organized in layers that process information through weighted connections.
Deep Learning ✓ understood
A subset of machine learning that uses neural networks with multiple layers (deep neural networks) to learn hierarchical representations of data.
Computer Vision ✓ understood
The field of AI that gets computers to extract meaning from images and video: what is in them, where it is, and how it moves.
Image Generation · read first ✓ understood
Creating new images from scratch or from text descriptions using generative models (GANs, diffusion models, VAEs).
Reinforcement Learning · read first ✓ understood
Learning through interaction with an environment, receiving rewards or penalties to learn optimal behavior policies.
Reward · read first ✓ understood
A scalar feedback signal indicating how good an action was, used to train reinforcement learning agents.
Dataset ✓ understood
A collection of data examples used for training, validating, or testing machine learning models.
Benchmark · read first ✓ understood
A standardized dataset and task used to compare model performance across different approaches (ImageNet, GLUE, SuperGLUE).

In the frontier

Rank: #58 of 100
Citations: 90; as of Aug 9, 2026
Published: Aug 2025

Topics: RL for reasoning , Image generation and editing , Agent benchmarks and computer use

Selection: 1kpapers.com by Together AI, most-cited as of Aug 9, 2026

Nearby papers

Canon · 2009 ImageNet: A Large-Scale Hierarchical Image Database ImageNet and its annual challenge gave deep learning the benchmark it needed to prove itself. Canon · 2013 Playing Atari with Deep Reinforcement Learning It started deep reinforcement learning: one learner, raw perception, many tasks. Canon · 2016 Mastering the Game of Go with Deep Neural Networks and Tree Search It proved that learned intuition plus search can master a problem long thought a decade away. Canon · 2017 Proximal Policy Optimization Algorithms PPO became the default RL algorithm, including for RLHF on the first ChatGPT-era models.