Submissions were assessed using the subjective video quality assessment method, specifically the Comparison Category Rating (CCR) test from the crowdsourcing implementation of ITU-T Rec. P.910. In CCR, subjects view the source (ground truth) and processed clips, rating the quality of the second clip compared to the first. The presentation order is randomized, and average ratings, presented as CMOS, indicate the processed clip’s quality relative to the source. Ratings range from -3 (much worse) to +3 (much better), with 0 indicating no difference.
The average subjective scores in terms of CMOS, multiple objective metrics, and the ranking of models evaluated in the challenge are presented in Table II. Two consecutive models are considered to have tied ranks when there is no significant difference between the distributions of CMOS values in tracks 1 and 2.
Citation
Please find the description of the challenge, methods and results in our publication:
ICME 2025 Grand Challenge on Video Super-Resolution for Video Conferencing (opens in new tab), ICME 2025.
@article{naderi2025icme,
title={ICME 2025 Grand Challenge on Video Super-Resolution for Video Conferencing},
author={Naderi, Babak and Cutler, Ross and Cho, Juhee and Khongbantabam, Nabakumar and Ivkovic, Dejan},
journal={arXiv preprint arXiv:2506.12269},
year={2025}
}
