AGT: Efficient Offline Reinforcement Learning With Advantage-Guided Transformer
Jiaye Wei , Xin Xu , Yixing Lan , Tenglong Liu , Yueying Wang
CAAI Transactions on Intelligence Technology ›› 2026, Vol. 11 ›› Issue (4) : 1127 -1143.
Offline reinforcement learning (RL) is a paradigm that seeks to train policies directly based on fixed datasets derived from previous interactions with the environment. However, offline RL faces critical challenges in environments characterised by sparse rewards and datasets dominated by suboptimal trajectories. In this article, we propose a simple and efficient offline RL framework via advantage-guided transformer (AGT). Different from previous works, AGT promotes an advantage-guided policy constraint into the loss function during policy training that adaptively learns the features of trajectories with high advantage values. Furthermore, AGT samples actions with high advantage values during action generation, replacing manual designs with advantage values to update the return-to-go (RTG) as conditional inputs, to ensure action consistency with expected returns. Theoretical analysis shows that AGT's policy improvement guarantees convergence to near-optimal policies in environments with low stochasticity. Extensive experiments on D4RL benchmarks, including robotic locomotion control, dexterous manipulation and maze navigation tasks, show AGT outperforms state-of-the-art (SOTA) offline RL methods.
artificial intelligence / artificial neural network / learning (artificial intelligence) / machine learning
| [1] |
|
| [2] |
|
| [3] |
|
| [4] |
|
| [5] |
|
| [6] |
|
| [7] |
|
| [8] |
|
| [9] |
|
| [10] |
|
| [11] |
|
| [12] |
|
| [13] |
|
| [14] |
|
| [15] |
|
| [16] |
|
| [17] |
|
| [18] |
|
| [19] |
|
| [20] |
|
| [21] |
|
| [22] |
|
| [23] |
|
| [24] |
|
| [25] |
|
| [26] |
|
| [27] |
|
| [28] |
|
| [29] |
|
| [30] |
|
| [31] |
|
| [32] |
|
| [33] |
|
| [34] |
|
| [35] |
|
| [36] |
|
| [37] |
|
| [38] |
|
| [39] |
|
| [40] |
|
| [41] |
|
| [42] |
|
| [43] |
|
| [44] |
|
| [45] |
|
| [46] |
|
| [47] |
|
| [48] |
|
| [49] |
|
| [50] |
|
| [51] |
|
| [52] |
|
| [53] |
|
| [54] |
|
/
| 〈 |
|
〉 |