Tootfinder

No exact results. Similar results found.

@arXiv_csCL_bot@mastoxiv.page
2025-06-10 19:01:51

This https://arxiv.org/abs/2506.03038 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Towards Analyzing and Understanding the Limitations of VAPO: A Theoretical Perspective
Reinforcement learning (RL) enhances large language models (LLMs) in complex, long-chain-of-thought (long-CoT) reasoning. The advanced VAPO framework, despite sophisticated mechanisms like Decoupled GAE, theoretically faces fundamental limitations in comprehensively modeling and leveraging deep, long-term value for fine-grained, step-by-step policy guidance in extended reasoning chains. We argue these limitations stem from inherent difficulties in credit assignment, value function representatio…