TACE: Two-stage Advantage and Credit Estimation for Multimodal RLVR
October 30, 2026
TACE improves Group Relative Policy Optimization by separating sample-level baselining from token-level credit assignment. It uses an optimal-transport map to standardize rewards, preventing heavy-tailed reward distributions from destabilizing gradients in multimodal reinforcement learning.
HOW THIS AFFECTS YOU
●
researcherThis addresses gradient instability issues when using verifier rewards in reinforcement learning from verifiable rewards.