沙丘社区
论文

Hunyuan-A13B技术报告

Hunyuan-A13B Technical Report

Tencent Hunyuan Team, Ao Liu, Botong Zhou, Can Xu, Chayse Zhou, ChenChen Zhang, Chengcheng Xu, Chenhao Wang, Decheng Wu, Dengpeng Wu, Dian Jiao, Dong Du, Dong Wang, Feng Zhang, Fengzong Lian, Guanghui Xu, Guanwei Zhang, Hai Wang, Haipeng Luo, Han Hu, Huilin Xu, Jiajia Wu, Jianchen Zhu, Jianfeng Yan, Jiaqi Zhu, Jihong Zhang, Jinbao Xue, Jun Xia, Junqiang Zheng, Kai Liu, Kai Zhang, Kai Zheng, Kejiao Li, Keyao Wang, Lan Jiang, Lixin Liu, Lulu Wu, Mengyuan Huang, Peijie Yu, Peiqi Wang, Qian Wang, Qianbiao Xiang, Qibin Liu, Qingfeng Sun, Richard Guo, Ruobing Xie, Saiyong Yang, Shaohua Chen, Shihui Hu, Shuai Li, Shuaipeng Li, Shuang Chen, Suncong Zheng, Tao Yang, Tian Zhang, Tinghao Yu, Weidong Han, Weijie Liu, Weijin Zhou, Weikang Wang, Wesleye Chen, Xiao Feng, Xiaoqin Ren, Xingwu Sun, Xiong Kuang, Xuemeng Huang, Xun Cao, Yanfeng Chen, Yang Du, Zhen Yang, Yangyu Tao, Yaping Deng, Yi Shen, Yigeng Hong, Yiqi Chen
模型训练模型架构MoE预训练强化学习Agentic RL

摘要

我们提出Hunyuan-A13B，一个基于混合专家（MoE）架构的开源大语言模型。它总参数量为800亿，但推理时仅激活130亿，在模型能力、计算效率与部署成本之间取得平衡。模型在经过严格过滤的20T token语料上预训练，并强化了STEM数据策展，提升了事实可靠性与推理能力。高质量监督微调与大规模强化学习进一步增强了其整体性能。Hunyuan-A13B还引入双模式思维链框架，使推理深度适配任务复杂度：常规查询用快思考，复杂多步问题用慢思考。评测显示其在数学、科学、编程、通用语言理解与Agent任务上表现有竞争力，常接近大得多的模型。其高推理吞吐使其适合延迟敏感的应用。我们开源Hunyuan-A13B以支持开放研究与LLM实际部署。