SCOPE: Semantic Bounded Credit Optimization for Prompt Engineering in Text-To-Image Generation

Yufei Jiao, Jinfa Huang, Zeyue Xue, Tianyu Xie, Yuexiao Ma, Rongrong Ji, Xiawu Zheng, Jiebo Luo

SCOPE improves prompt engineering by aligning reinforcement learning updates with semantic phrase boundaries.

How can we improve prompt rewriting for text-to-image models by assigning credit to specific semantic spans rather than treating the entire prompt as a single unit?

Reinforcement learning (RL) for prompt optimization often treats every subword token as an independent unit, fragmenting coherent compositional edits—like object counts or spatial relations—across arbitrary tokenizer boundaries. SCOPE resolves this by having the rewriter generate both a prompt and its semantic partition, then pooling policy log-probability shifts within each span to derive shared update weights. This modulates the completion-level advantage across tokens without requiring separate rewards or additional image renders. Across three frozen text-to-image backbones, SCOPE consistently outperforms uniform token-level baselines, achieving average relative gains of 9.48% on GenEval and 7.62% on T2I-CompBench.

Paper Primer

The core mechanism is a credit modulation layer that sits atop Group Relative Policy Optimization (GRPO). It acts like a traffic controller: it reads the semantic "address" of each token span and routes the shared completion advantage proportionally, ensuring that a single compositional edit is updated as a unified block rather than a collection of disconnected subwords.

SCOPE significantly improves compositional adherence in frozen text-to-image generators.

Performance gains on GenEval and T2I-CompBench across Qwen-Image, SD3, and FLUX.1-dev. Up to 45.6% relative improvement in spatial positioning accuracy on SD3 compared to GRPO.

Semantic boundary alignment is the primary driver of performance, not just generic reweighting.

Ablation studies comparing semantic partitions against random boundaries or fixed 3-token windows. 8–9% performance drop when using non-semantic boundaries.

Why does uniform token-level updating fail for prompt engineering?

Uniform updates fragment coherent compositional edits—such as "four handbags"—across subword boundaries, causing the model to inadvertently distort fine-grained descriptive phrases during exploration.

Does SCOPE require extra computational overhead during training?

No; it uses the existing rendering and reward pipeline and requires no separate span rewards, additional image renders, or learned critics.

Introduction and Motivation

Uniform token rewards fragment credit across semantic spans, limiting prompt‑rewriting RL.

Reinforcement‑learning prompt rewriting receives a single image‑level reward, which is then applied uniformly to every subword token. This uniform treatment fragments credit for compositional edits that naturally belong to coherent semantic spans, preventing the agent from learning which prompt parts truly drive image quality. Like grading each word in isolation versus grading whole sentences, uniform token weighting is cheaper but fails to capture the meaning that spans convey.

SCOPE tackles the credit‑assignment bottleneck by pooling policy‑log‑probability shifts inside each semantic span, producing a shared coefficient that reweights token losses while keeping a single advantage per image.

**Figure 1.** Comparison of uniform token weighting in GRPO and semantic span weighting in SCOPE. A T2I evaluator provides a single reward for the image. Standard GRPO derives an advantage for the completion and applies it across token positions. However, compositional prompt edits are naturally organized into semantic phrases whose boundaries need not align with tokenizer positions. SCOPE preserves the same advantage for the completion and derives a shared coefficient for each semantic span from shifts in policy log probabilities. This coefficient modulates all token updates within the span.

The uniform reward signal is the core limitation that prevents RL from correctly crediting the most impactful prompt segments.

Related Work and Baseline

Summarizes prior credit‑assignment work and outlines SCOPE’s methodological context.

Credit‑assignment for sequence generation has long struggled with per‑token credit granularity. GRPO [refs] optimizes autoregressive policies without a learned critic and shares a single advantage across all tokens in each completion. Recent work explores finer granularity via eligibility traces, counterfactual evaluation, and tree‑structured estimation (e.g., SPO and FIPO), but SCOPE retains a single advantage while modulating token updates through shared information within each semantic span.

Prompt optimization for text‑to‑image (T2I) generation adapts the textual input to elicit desired visual outputs. Prior approaches include search‑based refinement, learned prompt adapters, Promptist [refs] (supervised init + RL), PAG (GFlowNets for diverse prompts), and RePrompt [refs] (GRPO‑driven RL with image‑preference signals). Our SCOPE method incorporates the semantic structure of compositional edits into token weighting while preserving a shared advantage for the complete rewrite.

Reinforcement learning for T2I generation treats image synthesis as a sequential decision process. Earlier work optimizes diffusion trajectories (Flow‑GRPO [refs]), masked generation steps (Mask‑GRPO [refs]), or uses backward‑progressive training to address sparse rewards (B2‑DiffuRL [refs]). SCOPE applies RL to the prompt‑rewriting policy, keeping the image generator frozen, and leverages semantic spans to distribute credit within the rewrite.

GRPO treats every token in a generated sequence as receiving the same advantage, updating the whole sequence uniformly based on a group‑level signal.

How does GRPO differ from the classic REINFORCE algorithm that assigns a separate advantage to each token?

REINFORCE computes a token‑wise advantage (often the raw reward minus a baseline) and updates each token independently, which can introduce high variance when tokens have disparate contributions. GRPO instead aggregates rewards across a batch of rewrites, standardizes them, and applies the resulting single advantage uniformly to all tokens, thereby reducing variance and eliminating the need for a per‑token baseline.

The SCOPE Mechanism

SCOPE reweights token credit by semantic spans to focus learning on impactful prompt parts.

Standard RL for prompt rewriting treats every generated token identically, so the optimizer cannot tell which parts of the prompt actually improve the image.

Instead of giving each token the same credit, SCOPE groups tokens into meaningful spans and rescales the advantage inside each span according to how the policy’s probability changed for that span.

How does SCOPE differ from the GRPO baseline that gives every token the same advantage?

GRPO multiplies the shared advantage A by the same scalar for all tokens, so a token’s loss contribution depends only on its probability ratio. SCOPE first aggregates $\Delta$ within each semantic span, creates a span‑specific coefficient $w$, and then rescales A separately for each span. Tokens inside a high‑impact span receive a larger (or smaller) advantage, while the overall sum of advantages stays the same.

Compute the span means: b₁ = (0.05+0.04)/2 = 0.045, b₂ = (‑0.02)/1 = ‑0.02.

Exponentiate and clip: v₁ = clip(exp(0.045), 0.8, 1.2) ≈ 1.046, v₂ = clip(exp(‑0.02), 0.8, 1.2) ≈ 0.980.

Normalize by token count: denominator = |Span 1|·v₁ + |Span 2|·v₂ = 2·1.046 + 1·0.980 = 3.072. Then w₁ = 1.046 / 3.072 ≈ 0.341, w₂ = 0.980 / 3.072 ≈ 0.319.

Apply to the shared advantage A (assume A=1.0): ˜A for tokens in Span 1 = 0.341, for token in Span 2 = 0.319.

These weights sum to (2·0.341 + 1·0.319) ≈ 1.001 ≈ 1, preserving the total credit.

Span‑wise averaging removes the raw token‑count bias, and the detached normalization guarantees that the overall advantage magnitude is unchanged while still redistributing credit according to each span’s improvement.

**Require:** minibatch $\mathcal{B}$ from $\mathcal{D}$; rewriter $\pi_\theta$; frozen renderer $f_\phi$ 1: On the first step set $\pi_{\text{old}} \leftarrow \pi_\theta$. Afterward $\pi_{\text{old}}$ is the pre-update snapshot. 2: **for** training step $s = 1, 2, \dots$ **do** 3: $\quad$ **for** each prompt $x_g \in \mathcal{B}$ **do** 4: $\quad \quadSampley_{g,1}, \dots, y_{g,N-1} \sim \pi_\theta(\cdot \mid x_g)$ 5: $\quad \quadSety_{g,N} \leftarrow x_g$, so $c(y_{g,N}) = x_g$, and $A_{g,N} \leftarrow 0$ 6: $\quad \quad$ Share one rendering seed across the $N$ group members 7: $\quad \quad$ **for** $i = 1, \dots, N$ **do** 8: $\quad \quad \quad r_{g,i} \leftarrow w_{t2i} r_{t2i} + w_{\text{fmt}} r_{\text{fmt}}$ $\quad$ \# Eq. (2) 9: $\quad \quad$ **end for** 10: $\quad \quad A_{g,i}$ from $b_g = r_{g,N}$, with $\mu_g$ and $\hat{\sigma}_g$ over $i < N$ $\quad$ \# Eq. (3) 11: $\quad \quad$ **for** $i = 1, \dots, N-1$ **do** 12: $\quad \quad \quad \Delta_{g,i,t} \leftarrow \log \pi_\theta - \log \pi_{\text{old}}$ $\quad$ \# Eq. (6) 13: $\quad \quad \quad \rho_{g,i,t} \leftarrow \exp(\Delta_{g,i,t})$ $\quad$ \# Eq. (4) 14: $\quad \quad \quad$ **if** the answer partition aligns to the rollout tokens **then** 15: $\quad \quad \quad \quad b_{g,i,s} \leftarrow \text{mean}_{t \in \mathcal{T}_{g,i,s}} \Delta_{g,i,t}$ $\quad$ \# Eq. (7) 16: $\quad \quad \quad \quad v_{g,i,s} \leftarrow \text{clip}(\exp(\beta b_{g,i,s}), 0.8, 1.2)$ $\quad$ \# Eq. (8) 17: $\quad \quad \quad \quad w_{g,i,s} \leftarrow v_{g,i,s}/\text{mean}(v)$ $\quad$ \# Eq. (9) 18: $\quad \quad \quad \quad \tilde{A}_{g,i,t} \leftarrow A_{g,i} \text{stopgrad}(w_{g,i,s})$ $\quad$ \# Eq. (10) 19: $\quad \quad \quad \quad \ell_{g,i}^{\text{scope}}$ $\quad$ \# Eq. (11) 20: $\quad \quad \quad$ **else** 21: $\quad \quad \quad \quad \ell_{g,i}^{\text{uni}}$ $\quad$ \# Eq. (5) 22: $\quad \quad \quad$ **end if** 23: $\quad \quad$ **end for** 24: $\quad$ **end for** 25: $\quad \mathcal{L}_{\text{pol}} \leftarrow |\mathcal{B}|^{-1} \sum_g (N-1)^{-1} \sum_{i < N} \ell_{g,i}$ $\quad$ \# Eq. (12) 26: $\quad$ Add the optional KL penalty against the frozen reference, which is distinct from $\pi_{\text{old}}$, and update $\theta$ 27: $\quadSet\pi_{\text{old}} \leftarrow \pi_\theta$ for the next step 28: **end for** 29: **return** $\pi_\theta$

**Figure 2. Overview of SCOPE Pipeline.** The rewriter generates a prompt together with its semantic spans. After removing the span separators, the frozen renderer generates an image whose score contributes to the reward for the completion. SCOPE aligns the spans to rollout tokens and pools changes in log probability between the current and old policies. The resulting coefficients reweight the shared advantage within each span. Gradients flow through the clipped GRPO surrogate while the coefficients are detached.

Experimental Results

SCOPE beats baselines on GenEval and T2I‑CompBench, delivering sizable score gains.

Recall that standard RL treats all prompt tokens equally, while SCOPE assigns credit to semantic spans, focusing learning on the most impactful segments.

Uniform token‑level updates blur fine‑grained details, much like repainting an entire wall with a single brushstroke erases subtle features.

SCOPE dramatically improves spatial positioning on SD3, raising accuracy from 0.57 to 0.83 (+45.6%).

Table 2 shows the SD3 Position score jumps from 0.57 (GRPO) to 0.83 (SCOPE).

**Table 1.** SCOPE achieves the highest overall GenEval score on each frozen backbone. RePrompt$^\dagger$ is our Qwen-Image reproduction, while its SD3 and FLUX.1-dev scores follow Wu et al. [55]. Single and Two denote one and two objects. Cnt., Clr., Pos., and Attr. denote counting, color, position, and attribute binding. Bold marks the best score per metric and backbone.

**Table 1.** Quantitative comparison of different methods on the GenEval benchmark across three frozen backbones (Qwen-Image, SD3, and FLUX.1-dev). SCOPE (Ours) consistently outperforms baseline methods across all categories and backbones.

**Table.** CompBench scores for the Z-Image rewriter by backbone. Subset of Table.

**Figure 4.** Qualitative comparison on four prompts. From left to right: images from the original prompt (Qwen-Image), GRPO, and gpt-image 2, followed by the rewrite snippet and the SCOPE output (Qwen-Image).

SCOPE consistently outperforms baselines across all backbones.

Ablations and Analysis

Ablations confirm that span‑pooled credit drives performance while random boundaries and token‑level noise hurt.

We now dissect how each design choice contributes to SCOPE’s superiority over the uniform GRPO baseline.

**Figure 3. Ablation and mechanistic diagnostics.** (a) Geneval averages across backbones: SCOPE consistently outperforms uniform GRPO, whereas unpooled token-level shifts degrade performance. (b) Semantic boundaries substantially outperform random partitions and fixed 3-token windows on Qwen-Image. (c) SCOPE remains robust across advantage formulations on FLUX.1-dev, with all variants surpassing vanilla GRPO (0.760) and the hybrid rule achieving the peak score. (d) Training reward dynamics on Qwen-Image: SCOPE breaks through GRPO’s early saturation plateau to achieve higher asymptotic reward.

Span‑pooled credit weighting yields higher scores than uniform GRPO and token‑level shifts.

SCOPE attains 0.95, 0.84, 0.84 across Qwen‑Image, SD3, and FLUX.1‑dev, whereas the token‑local control reaches 0.90, 0.75, 0.76.

Replacing semantic span boundaries with random partitions or fixed three‑token windows reduces performance.

Random boundaries score 0.86 and fixed three‑token windows 0.87 on Qwen‑Image, an 8–9 % drop relative to SCOPE’s 0.95.

All advantage‑rule variants outperform the uniform GRPO baseline, with the hybrid rule adding a modest extra boost.

$\Delta$bg yields 0.827, z‑norm 0.832, and the hybrid rule 0.841, each surpassing GRPO’s 0.760.

SCOPE maintains a higher training‑reward trajectory than GRPO throughout the early training window.

Smoothed SCOPE rewards stay in the 0.86–0.90 range, while GRPO remains between 0.83 and 0.85 over the first 300 steps.

SCOPE achieves a 9.48 % relative gain on the GenEval benchmark.

Across Qwen‑Image, SD3, and FLUX.1‑dev the average score rises from the GRPO baseline to the SCOPE result, yielding the reported percentage improvement.

SCOPE achieves a 7.62 % relative gain on the T2I‑CompBench benchmark.

The same three backbones show consistent uplift over GRPO, reflected in the reported percentage.

Implementation Details

Implementation details, hyperparameters, and extra results for reproducibility.

A.1 distributes policy training across workers: the Qwen3-8B rewriter runs on policy nodes while separate reward and rendering workers host frozen T2I backbones. The rewriter emits spans directly during rollout, eliminating any dedicated span parser.

A.2 lists the hyperparameters used throughout the experiments (see Table 3). Key settings include a learning rate of 1e‑6, KL coefficient 0.01, and a maximum completion length of 512 tokens.

**Table 3.** Optimization configuration for prompt rewriting, span weighting, and normalization.

A.3 defines the rendering setup (Table 4). All three backbones render 512×512 images; Qwen‑Image and FLUX.1‑dev use 20 diffusion steps with CFG scale 4.0 and 3.5 respectively, while SD3 runs 28 steps at CFG 7.5.

**Table 4.** Fixed rendering configuration for training rewards and benchmark evaluation in our runs.

A.4 describes how spans are extracted and how fallbacks are handled. The rewrite instruction forces non‑empty spans separated by “|”, and token offsets are aligned to guarantee a valid partition. Malformed outputs trigger uniform GRPO weighting as a fallback.

**Table 5.** Fallback frequencies in the logged training rollouts by benchmark and frozen image backbone.

A.5 presents Z‑Image rewriter scores (Tables 6 and 7). Across the three frozen backbones, Qwen‑Image consistently outperforms SD3 and FLUX.1‑dev on both GenEval and T2I‑CompBench metrics.

**Table 4.**

A.6 notes that policy optimization and auxiliary jobs run on NVIDIA H200 GPUs; exact node counts vary per deployment but follow the sampling schedule in Table 4.

A.7 details environment variables exposed by reward services, including generation count (default 8) and the separate $\beta$ parameters for span weighting versus KL regularization.

A.8 specifies the rewrite instruction format: the system message expects “span 1 | span 2 | span 3”. After extraction, delimiters are removed and spans concatenated into a natural‑language prompt for rendering.

A.9 clarifies that advantages follow the hybrid rule (Eq. 3), with groups of N = 8 prompts sharing a random diffusion seed to control renderer noise during training.

A.10 explains that RePrompt results for Qwen‑Image are reproduced from Wu et al. [55] using the same reward clients, rendering seeds, and benchmark harness as our GRPO and SCOPE runs.

A.11 outlines the SCOPE training algorithm (Algorithm 1). The procedure keeps the renderer frozen, updates the policy via Eq. 1–12, and uses the span‑weighted loss ℓ_scope when the answer partition aligns; otherwise it falls back to the uniform loss ℓ_uni.

A.12 provides additional qualitative comparisons (Figure 5). Across four prompts, the SCOPE outputs align more closely with the detailed rewrite rationales than GRPO or gpt‑image 2 generations.

**Figure 5.** Additional qualitative comparisons on four prompts, using the same five columns as Figure 4.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers