论文
Nemotron 3 Nano Omni：高效、开放的多模态智能
Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligence
NVIDIA; :; Amala Sanjay Deshmukh; Kateryna Chumachenko; Tuomas Rintamaki; Matthieu Le; Tyler Poon; Danial Mohseni Taheri; Ilia Karmanov; Guilin Liu; Jarno Seppanen; Arushi Goel; Mike Ranzinger; Greg Heinrich; Guo Chen; Lukas Voegtle; Philipp Fischer; Timo Roman; Karan Sapra; Collin McCarthy; Shaokun Zhang; Fuxiao Liu; Hanrong Ye; Yi Dong; Mingjie Liu; Yifan Peng; Piotr Zelasko; Zhehuai Chen; Nithin Rao Koluguri; Nune Tadevosyan; Lilit Grigoryan; Ehsan Hosseini Asl; Pritam Biswas; Leili Tavabi; Yuanhang Su; Zhiding Yu; Peter Jin; Alexandre Milesi; Netanel Haber; Yao Xu; Sarah Amiraslani; Nabin Mulepati; Eric Tramel; Jaehun Jung; Ximing Lu; Brandon Cui; Jin Xu; Zhiqi Li; Shihao Wang; Yuanguo Kuang; Shaokun Zhang; Huck Yang; Boyi Li; Hongxu Yin; Song Han; Bilal Kartal; Pavlo Molchanov; Adi Renduchintala; Charles Wang; David Mosallanezhad; Soumye Singhal; Luis Vega; Katherine Cheung; Sreyan Ghosh; Yian Zhang; Alexander Bukharin; Venkat Srinivasan; Johnny Greco; Andre Manoel; Maarten Van Segbroeck; Suseella Panguliri; Rohit Watve; Divyanshu Kakwani; Shubham Pachori; Jeffrey Glick; Radha Sri-Tharan; Aileen Zaman; Khanh Nguyen; Shi Chen; Jiaheng Fang; Qing Miao; Wenfei Zhou; Yu Wang; Zaid Pervaiz Bhat; Varun Praveen; Arihant Jain; Ramanathan Arunachalam; Tomasz Kornuta; Ashton Sharabiani; Amy Shen; Wei Huang; Yi-Fu Wu; Ali Roshan Ghias; Huiying Li; Brian Yu; Nima Tajbakhsh; Chen Cui; Wenwen Gao; Li Ding; Terry Kong; Manoj Kilaru; Anahita Bhiwandiwalla; Marek Wawrzos; Daniel Korzekwa; Pablo Ribalta; Grzegorz Chlebus; Besmira Nushi; Ewa Dobrowolska; Maciej Jakub Mikulski; Kunal Dhawan; Steve Huang; Jagadeesh Balam; Yongqiang Wang; Nikolay Karpov; Valentin Mendelev; George Zelenfroynd; Meline Mkrtchyan; Qing Miao; Omri Almog; Bhavesh Pawar; Rameshwar Shivbhakta; Sudeep Sabnis; Ashrton Sharabiani; Negar Habibi; Geethapriya Venkataramani; Pamela Peng; Prerit Rodney; Serge Panev; Richard Mazzarese; Nicky Liu; Michael Fukuyama; Andrii Skliar; Roger Waleffe; Duncan Riach; Yunheng Zou; Jian Hu; Hao Zhang; Binfeng Xu; Yuhao Yang; Zuhair Ahmed; Alexandre Milesi; Carlo del Mundo; Chad Voegele; Zhiyu Cheng; Nave Assaf; Andrii Skliar; Daniel Afrimi; Natan Bagrov; Ran Zilberstein; Ofri Masad; Eugene Khvedchenia; Natan Bagrov; Borys Tymchenko; Tomer Asida; Daniel Afrimi; Parth Mannan; Victor Cui; Michael Evans; Katherine Luna; Jie Lou; Pinky Xu; Guyue Huang; Negar Habibi; Michael Boone; Pradeep Thalasta; Adeola Adesoba; Dina Yared; Christopher Parisien; Leon Derczynski; Shaona Ghosh; Wes Feely; Micah Schaffer; Radha Sri-Tharan; Jeffrey Glick; Barnaby Simkin; George Zelenfroynd; Tomasz Grzegorzek; Rishabh Garg; Aastha Jhunjhunwala; Sergei Kolchenko; Farzan Memarian; Haran Kumar; Shiv Kumar; Isabel Hulseman; Anjali Shah; Kari Briski; Padmavathy Subramanian; Joey Conway; Udi Karpas; Jane Polak Scowcroft; Annie Surla; Shilpa Ammireddy; Ellie Evans; Jesse Oliver; Tom Balough; Chia-Chih Chen; Sandip Bhaskar; Alejandra Rico; Bardiya Sadeghi; Seph Mard; Katherine Cheung; Meredith Price; Laya Sleiman; Saori Kaji; Wesley Helmholz; Wendy Quan; Michael Lightstone; Jonathan Cohen; Jian Zhang; Oleksii Kuchaiev; Boris Ginsburg; Jan Kautz; Eileen Long; Mohammad Shoeybi; Mostofa Patwary; Oluwatobi Olabiyi; Andrew Tao; Bryan Catanzaro; Udi Karpas·
模型训练预训练
摘要
我们推出 Nemotron 3 Nano Omni，这是 Nemotron 多模式系列中的最新型号，也是第一个原生支持音频输入以及文本、图像和视频的型号。凭借架构、训练数据和配方方面的进步，Nemotron 3 Nano Omni 在所有模式下均比其前身 Nemotron Nano V2 VL 提供了一致的精度改进。特别是，Nemotron 3 在现实世界文档理解、长音频-视频理解和代理计算机使用方面提供了领先的结果。 Nemotron 3 Nano Omni 基于高效的 Nemotron 3 Nano 30B-A3B 主干网络构建，进一步结合了创新的多模态词元缩减技术，与其他类似尺寸的模型相比，可显着降低推理延迟并提高吞吐量。我们将发布 BF16、FP8 和 FP4 格式的模型检查点以及部分训练数据和代码库，以促进进一步的研究和开发。