Vision & Multimodal Aug 2025 · #58 most cited · 90 citations

Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learning

Yibin Wang et al.

arXiv:2508.20751

In short

RL for text-to-image models gets “reward hacked” when tiny score differences are amplified by normalisation. Pref-GRPO instead compares images in pairs and uses win rate as the reward, and the paper adds UniGenBench, a finer-grained text-to-image benchmark.

Why it matters

More stable RL for image generators, plus a better yardstick for them.

Read first

The 4 Field Guide ideas this paper leans on.

Starting from scratch? The full route 9 ideas · basics first
  1. Machine Learning ✓ understood

    Building systems that learn patterns from data instead of following hand-written rules, getting better at a task as they see more examples.

  2. Neural Network ✓ understood

    A model built from layers of simple units, each taking a weighted sum of its inputs and applying a nonlinearity; training adjusts the weights so the whole stack maps inputs to useful outputs.

  3. Deep Learning ✓ understood

    A subset of machine learning that uses neural networks with multiple layers (deep neural networks) to learn hierarchical representations of data.

  4. Computer Vision ✓ understood

    The field of AI that gets computers to extract meaning from images and video: what is in them, where it is, and how it moves.

  5. Image Generation · read first ✓ understood

    Creating new images, often from a text prompt, with generative models such as GANs, VAEs and diffusion models that learn to turn random noise into realistic pictures.

  6. Reinforcement Learning · read first ✓ understood

    Learning through interaction with an environment, receiving rewards or penalties to learn optimal behavior policies.

  7. Reward · read first ✓ understood

    The single number an environment sends back after each action, telling a reinforcement learning agent how good that step was; the agent learns to maximize its long-run total.

  8. Dataset ✓ understood

    A collection of data examples used for training, validating, or testing machine learning models.

  9. Benchmark · read first ✓ understood

    A standardized dataset and task used to compare model performance across different approaches (ImageNet, GLUE, SuperGLUE).

In the frontier

Rank
#58 of 100
Citations
90
as of Aug 9, 2026
Published
Aug 2025

Topics: RL for reasoning , Image generation and editing , Agent benchmarks and computer use

Selection: 1kpapers.com by Together AI, most-cited as of Aug 9, 2026

Nearby papers

Summary in our own words; read the paper for the details. ← All papers