沙丘社区
论文

ROME：以开放训练生态与交互片段信用分配训练Agent

Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem

Weixun Wang, XiaoXiao Xu, Wanhe An, Fangwen Dai, Wei Gao, Yancheng He, Ju Huang, Qiang Ji, Hanqi Jin, Xiaoyang Li, Yang Li, Zhongwen Li, Shirong Lin, Jiashun Liu, Zenan Liu, Tao Luo, Dilxat Muhtar, Yuanbin Qu, Jiaqiang Shi, Qinghui Sun, Yingshui Tan, Hao Tang, Runze Wang, Yi Wang, Zhaoguo Wang, Yanan Wu, Shaopan Xiong, Binchen Xu, Xander Xu, Yuchi Xu, Qipeng Zhang, Xixia Zhang, Haizhou Zhao, Jie Zhao, Shuaibing Zhao, Baihui Zheng, Jianhui Zheng, Suhang Zheng, Yanni Zhu, Mengze Cai, Kerui Cao, Xitong Chen, Yue Dai, Lifan Du, Tao Feng, Tao He, Jin Hu, Yijie Hu, Ziyu Jiang, Cheng Li, Xiang Li, Jing Liang, Chonghuan Liu, ZhenDong Liu, Haodong Mi, Yanhu Mo, Junjia Ni, Shixin Pei, Jingyu Shen, XiaoShuai Song, Cecilia Wang, Chaofan Wang, Kangyu Wang, Pei Wang, Tao Wang, Wei Wang, Ke Xiao, Mingyu Xu, Tiange Xu, Nan Ya, Siran Yang, Jianan Ye, Yaxing Zang, Duo Zhang, Junbo Zhang, Boren Zheng, Wanxi Deng, Ling Pan, Lin Qu, Wenbo Su, Jiamang Wang, Wei Wang, Hu Wei, Minggang Wu, Cheng Yu, Bing Zhao, Zhicheng Zheng, Bo Zheng
模型训练AI 基础设施强化学习SandboxAgent Sandbox

摘要

Agentic构建要求LLM在真实环境中多轮采取行动、观察结果并迭代改进产物，但开源社区仍缺乏有原则的端到端Agent开发生态。本文提出Agentic Learning Ecosystem（ALE），用于优化Agent LLM的生产管线。ALE由三个部分组成：负责权重优化的后训练框架ROLL、负责轨迹生成的沙箱环境管理器ROCK，以及用于高效上下文工程的Agent框架iFlow CLI。作者发布基于ALE、以百万余条轨迹训练的开源Agent模型ROME。方法包括合成复杂行为的数据组合协议，以及交互式策略对齐算法Interaction-based Policy Alignment（IPA）；IPA按语义交互片段而非单个Token分配信用，以改善长程训练稳定性。作者在结构化环境中评估ROME，并提出扩大规模、加强污染控制的Terminal Bench Pro。ROME在SWE-bench Verified、Terminal Bench等基准中取得较好表现，支持ALE基础设施的有效性。

ROME：以开放训练生态与交互片段信用分配训练Agent 配图
图1：Agentic Learning Ecosystem（ALE）及ROME性能概览。