November 20, 2022
SUMEval 2022
Scaling Up Multilingual Evaluation Workshop @ AACL 2022
Location: Online only
SUMEval2022 will be held (starting) on 20th November 2022 and will be merged with the Eval4NLP workshop this year. Please find the combined schedule in Taipei time below.
| Session | Time (Taipei time, UTC+8) | Paper ID | Paper Title | Authors |
| 1 | 18:30:00 – 18:45 | Opening Presentation | ||
| 1 | 18:45:00 – 19:30 | Invited Talk | Anya Belz | |
| 1 | 19:30:00 – 19:45 | Why sentence similarity benchmark is not predictive of application-oriented task performance? | Kaori Abe, Sho Yokoi, Tomoyuki Kajiwara, Kentaro Inui | |
| 1 | 19:45:00 – 20:00 | Better Smatch = Better Parser? AMR evaluation is not so simple anymore | Juri Opitz, Anette Frank | |
| 1 | 20:00:00 – 20:15 | Chat Translation Error Detection for Assisting Cross-lingual Communications | Yunmeng Li, Jun Suzuki, Makoto Morishita, Kaori Abe, Ryoko Tokuhisa, Ana Brassard, Kentaro Inui | |
| 2 | 21:15:00 – 22:00 | Invited Talk | Reinald Kim Amplayo | |
| 2 | 22:00:00 – 22:15 | A Japanese Corpus of Many Specialized Domains for Word Segmentation and Part-of-Speech Tagging | Shohei Higashiyama, Masao Ideuchi, Masao Utiyama, Yoshiaki Oida, Eiichiro Sumita | |
| 2 | 22:15:00 – 22:30 | Evaluating the role of non-lexical markers in GPT-2’s language modeling behavior | Roberta Rocca | |
| 2 | 22:30:00 – 22:45 | From COMET to COMES – Can Summary Evaluation Benefit from Translation Evaluation? | Mateusz Krubiński, Pavel Pecina | |
| 2 | 22:45:00 – 23:00 | Random Text Perturbations Work, but not Always | Zhengxiang Wang | |
| 3 | 23:30:00 – 00:15 | Invited Talk | Maxime Peyrard | |
| 3 | 00:15:00 – 00:30 | A Comparative Analysis of Stance Detection Approaches and Datasets | Parush Gera, Tempestt Neal | |
| 3 | 00:30:00 – 00:45 | Assessing Neural Referential Form Selectors on a Realistic Multilingual Dataset | Guanyi Chen, Fahime Same, Kees Van Deemter | |
| 3 | 00:45:00 – 01:00 | Assessing Resource-Performance Trade-off of Natural Language Models using Data Envelopment Analysis | Zachary Zhou, Alisha Zachariah, Devin Conathan, Jeffery Kline | |
| 3 | 01:00:00 – 01:15 | GLARE: Generative Left-to-right AdversaRial Examples | Ryan Andrew Chi, Nathan Kim, Patrick Liu, Zander Lack, Ethan A Chi | |
| 4 | 01:30-02:10 | Invited Talk Masakhane | Masakhane | |
| 4 | 02:10-02:20 | SUMEval shared task overview | Kabir Ahuja, Antonios Anastasopoulos, Barun Patra, Graham Neubig, Monojit Choudhury, Sandipan Dandapat, Sunayana Sitaram and Vishrav Chaudhary | |
| 4 | 02:20-02:30 | SUMEval-2 | To Train or Not to Train: Predicting the Performance of Massively Multilingual Models | Shantanu Patankar, Omkar Gokhale, Onkar Litake, Aditya Mandke and Dipali Kadam |
| 4 | 02:30-02:40 | SUMEval-1 | The GMU System Submission for the SUMEval 2022 Shared Task | Syeda Sabrina Akter and Antonios Anastasopoulos |
| 4 | 02:40-02:55 | SUMEval-5 | NTREX-128 — News Test References for MT Evaluation of 128 Languages | Christian Federmann, Tom Kocmi, Ying Xin |
| 4 | 02:55-03:10 | SUMEval-6 | IndoRobusta: Towards Robustness Against Diverse Code-Mixed Indonesian Local Languages | Muhammad Farid Adilazuarda, Samuel Cahyawijaya, Genta Indra Winata, Pascale Fung, Ayu Purwariant |
| 4 | 03:10-03:20 | Closing |