论文
最后的翻译基准
Last Translation Benchmark
Vilém Zouhar; Niyati Bafna; Mukund Choudhary; Maike Züfle; Sara Rajaee; Pinzhen Chen; Jannis Vamvas; Sara Papi; Ona de Gibert; Bhavitvya Malik; Eliya Habba; Orfeas Menis Mastromichalakis; Patrícia Schmidtová; Michelle Wastl; Sheriff Issaka; Leshem Choshen; Stella Biderman; Antonis Anastasopoulos; Jan Niehues; Rico Sennrich; Mrinmaya Sachan; Ondřej Bojar; Kenton Murray; Jörg Tiedemann; Alham Fikri Aji; Philipp Koehn; Christof Monz; Alexandra Birch; Sowmya Vajjala; Chalamalasetti Kranti; Cristina España-Bonet; Nobin Sarwar; David Kaczér; Shunta Asano; Malik Marmonier; Daban Q. Jaff; Vaisakhi Mishra; Hend Al- Khalifa; Gabriele Sarti; Sourajit Saha; Nils Rehlinger; Juan Daniel Cuervo Villa; Jonathan Tonglet; Saugata Purkayastha; Dominik Macháček; Jagannathan Ramanujam; Heejin Do; Zuzana Nadova; Fred Philippy; Fabian Retkowski; Maria Lymperaiou; Silvia Casola; Hanna Yukhymenko; Shubhashis Roy Dipta; Sangwon Ryu; Andrés Jerez; Ron Keinan; Shuaib Shuaib Yusuf; Avantica Vempati; Maria Carmen Staiano; Sukannya Purkayastha; Adrian Cosma; Vitalii Babenko; Erivan Inan; Aviral Nigam; Wafa Aissa; Fatima Haouari; Venkata Prasanth Kumar Gummadi; Mehdi Jafarzadeh; Valentin Scourneau; Lukas Edman; Kaiser Sun; Shaomu Tan; Mohammad Sadegh Gholizadeh; Johannes-Rudolf David; Dipankar Srirag; Javier García Gilabert; Ruta Binkyte; Manar Ali; Ana-Maria Bucur; Sabry E. Farrag; Youssef Saber; Yihong Liu; Jean Maillard; Cojocaru Nicoleta; Xiaochuang Yuan; Sina Ahmadi; Philipp Mondorf; Kaustubh Dhole; Roman Wixinger; Shenbin Qian; Manuel Tuor; Sergey Troshin; Jonathan Yahav; Fida Mohammad Thoker; Amir Arsalan Rezapour; Lance Calvin Lim Gamboa; Manon Reusens; Kätriin Kukk; Koel Dutta Chowdhury; Giuseppe Gallipoli; Christian Hoang; Shaswati Saha; Seth Aycock; Jan Kocoń; Bo Chen; Linh Vu; Vatsal Venkatkrishna; Arafat Ahsan; Luan Thanh Nguyen; Hassan Soliman; Daryna Dementieva; Theresia Veronika Rampisela; Ngoc Quynh Tram Do; Marius Huber; Kazuki Egashira; Azmine Toushik Wasi; Vladislav Poritski; Mike Zhang; Deep Shah; Paul Gavrikov; Luis Frentzen Salim; David Africa; R. Damanhuri; Bello Umar Bello; Anumit Garg; Gengyu Rao; Pawan Sasanka Ammanamanchi; Kamile Dementaviciute; Andrianos Michail; L D M S Sai Teja; Dawei Zhu; Yi Fan; Wei Liu; Farhan Farsi; Elias Herranen; Sankalan Pal Chowdhury; Karen Sanchez; Farzad Shami; Ashok Urlana; Zimu Wang; Tomasz Limisiewicz; Priyaranjan Pattnayak; Marii Ojastu; Hongbin Na; Emilian Radoi; Chenyi Zhao; Carlos Hinojosa; Andrea Gregor de Varda; Zaid Alyafeai; Reem Alzahrani; Nehal Kathrotia; Alex Flückiger; Ulysses Sekai Tully Carr; Jimson Paulo Layacan; Guy Kaplan; Ritwik Tiwari; Rishit Dagli; Oksana Volchek; Isaac R Caswell; Bowen Yi; Blanka Kövér; Amir Hossein Yari; Aicha Chorana; Zhengxiang Wang; Selja Keränen; Samuel Simko; Joy Olusanya; Jenny Chim; Enzo Doyen; Vivek Harsha Lakkamaneni; Sophia Conrad; Pouya Sadeghi; Panayiotis Panayiotou; Luis Lara; Jannatul Nayem; Eran Yahav; Debanshu Das; Antonia Karamolegkou; Anmol Goel; Aishik Mandal; Tommaso Cerruti; Raoyuan Zhao; Mykola Haltiuk; Thura Aung; Naser Almousa; Amir Hossein Kargaran; Rachel Bawden; Qiaoyuan Zheng; Mateusz Lango; Beni Egressy; Fidel Rodríguez Velásquez; Natchapon Jongwiriyanurak; Minh Ngoc Do; Marco Gaido; Lena Libon; Dzmitry Kuzmin; Badal Nyalang; Antoine Taroni; Andrei Niculae; Abdulaziz Nura Kani; Rushikesh Zawar; Marek Šuppa; Beatrice Savoldi; Andreas Simons; Rayyan Merchant; Ilai Yaron Levy; Francesco Pinto; Ziyi Yang; Yolanda Xavier; Samuel Frontull; Muhammad Ravi Shulthan Habibi; Kenneth Enevoldsen; Harris Abdul Majid; Francesca Padovani; Tim Graf; Tatiana Bielakova; Sharifa Djurabaeva; Shaoxiong Ji; Raia Abu Ahmad; Pavel Stepachev; Jirui Qi; Ayush Sunil Munot; Alireza Pakniat; Ayla Rigouts Terryn; Yuxing Lu; Yurii Paniv; Xiyan Fu; Tosin Adewumi; Sunisth Kumar; Stéphane J. P. S. Thunus; Shree Harsha Bokkahalli Satish; Shayan Bali; Prakhar Gupta; Papa Abdou Karim Karou Diallo; Matija Akrap; Marko Culjak; Kristýna Onderková; Joseph Attieh; Esrael Teferi Tensay; Elisabeth Fittschen; Benoît Sagot; Jingwei Ni; Yu Fan·
模型评测基准与评测资源
摘要
为了科学进步，我们需要测试最先进模型极限的基准，以及告诉我们失败案例的评估方法。随着模型变得越来越强大，机器翻译的标准基准正在接近饱和。此外，自动翻译指标不可靠，容易受到 奖励作弊 的影响，并且提供无法操作的评估。即使是人工评估金标准也不是没有问题的，因为它往往缺乏可重复性、客观性和可扩展性。总体而言，这使我们无法跟踪该领域的客观进展并确定改进途径。我们推出了 Last Translation Benchmark，这是一组由人类创作和同行评审的示例（文本、图像、音频、视频），这些示例打破了领先的机器翻译模型。我们还提出了一种新的评估方法：每个示例都带有手工制作的验证规则，描述该示例的具体失败案例，因此允许可靠且可操作的未来评估。最后的翻译基准是一个实时数据集，接受持续的贡献。最新版本是 LTBv1，包含 2026 年 9 月 1 日之前已接受的贡献，随着不断收集新数据，计划未来发布。