November 20, 2022
SUMEval 2022
Scaling Up Multilingual Evaluation Workshop @ AACL 2022
Location: Online only
SUMEval2022 will be held (starting) on 20th November 2022 and will be merged with the Eval4NLP workshop (opens in new tab) this year. Please find the combined schedule in Taipei time below.
| Session | Time (Taipei time, UTC+8) | Paper ID | Paper Title | Authors |
| 1 | 18:30:00 – 18:45 | Opening Presentation | ||
| 1 | 18:45:00 – 19:30 | Invited Talk | Anya Belz | |
| 1 | 19:30:00 – 19:45 | Why sentence similarity benchmark is not predictive of application-oriented task performance? | Kaori Abe, Sho Yokoi, Tomoyuki Kajiwara, Kentaro Inui | |
| 1 | 19:45:00 – 20:00 | Better Smatch = Better Parser? AMR evaluation is not so simple anymore | Juri Opitz, Anette Frank | |
| 1 | 20:00:00 – 20:15 | Chat Translation Error Detection for Assisting Cross-lingual Communications | Yunmeng Li, Jun Suzuki, Makoto Morishita, Kaori Abe, Ryoko Tokuhisa, Ana Brassard, Kentaro Inui | |
| 2 | 21:15:00 – 22:00 | Invited Talk | Reinald Kim Amplayo | |
| 2 | 22:00:00 – 22:15 | A Japanese Corpus of Many Specialized Domains for Word Segmentation and Part-of-Speech Tagging | Shohei Higashiyama, Masao Ideuchi, Masao Utiyama, Yoshiaki Oida, Eiichiro Sumita | |
| 2 | 22:15:00 – 22:30 | Evaluating the role of non-lexical markers in GPT-2’s language modeling behavior | Roberta Rocca | |
| 2 | 22:30:00 – 22:45 | From COMET to COMES – Can Summary Evaluation Benefit from Translation Evaluation? | Mateusz Krubiński, Pavel Pecina | |
| 2 | 22:45:00 – 23:00 | Random Text Perturbations Work, but not Always | Zhengxiang Wang | |
| 3 | 23:30:00 – 00:15 | Invited Talk | Maxime Peyrard | |
| 3 | 00:15:00 – 00:30 | A Comparative Analysis of Stance Detection Approaches and Datasets | Parush Gera, Tempestt Neal | |
| 3 | 00:30:00 – 00:45 | Assessing Neural Referential Form Selectors on a Realistic Multilingual Dataset | Guanyi Chen, Fahime Same, Kees Van Deemter | |
| 3 | 00:45:00 – 01:00 | Assessing Resource-Performance Trade-off of Natural Language Models using Data Envelopment Analysis | Zachary Zhou, Alisha Zachariah, Devin Conathan, Jeffery Kline | |
| 3 | 01:00:00 – 01:15 | GLARE: Generative Left-to-right AdversaRial Examples | Ryan Andrew Chi, Nathan Kim, Patrick Liu, Zander Lack, Ethan A Chi | |
| 4 | 01:30-02:10 | Invited Talk Masakhane | Masakhane | |
| 4 | 02:10-02:20 | SUMEval shared task overview | Kabir Ahuja, Antonios Anastasopoulos, Barun Patra, Graham Neubig, Monojit Choudhury, Sandipan Dandapat, Sunayana Sitaram and Vishrav Chaudhary | |
| 4 | 02:20-02:30 | SUMEval-2 | To Train or Not to Train: Predicting the Performance of Massively Multilingual Models | Shantanu Patankar, Omkar Gokhale, Onkar Litake, Aditya Mandke and Dipali Kadam |
| 4 | 02:30-02:40 | SUMEval-1 | The GMU System Submission for the SUMEval 2022 Shared Task | Syeda Sabrina Akter and Antonios Anastasopoulos |
| 4 | 02:40-02:55 | SUMEval-5 | NTREX-128 — News Test References for MT Evaluation of 128 Languages | Christian Federmann, Tom Kocmi, Ying Xin |
| 4 | 02:55-03:10 | SUMEval-6 | IndoRobusta: Towards Robustness Against Diverse Code-Mixed Indonesian Local Languages | Muhammad Farid Adilazuarda, Samuel Cahyawijaya, Genta Indra Winata, Pascale Fung, Ayu Purwariant |
| 4 | 03:10-03:20 | Closing |