Learning-based image codecs produce different compression artifacts when compared to the blocking and blurring degradations introduced by conventional block-based image codecs such as JPEG, JPEG 2000, and HEVC Intra.
For this reason, a crowdsourcing-based subjective quality dataset was created to benchmark a representative set of end-to-end deep learning-based image codecs, submitted to the MMSP’2020 Grand Challenge on Learning-Based Image Coding. For the first time, a double stimulus methodology with a continuous quality scale (DSCQS) was applied to evaluate this type of image codecs. The subjective experiment is one of the largest ever reported including more than 240 pair-comparisons evaluated by 118 naïve subjects [1].
The same dataset has also been used to assess the performance of a large number of objective quality metrics, both on conventional and learning-based compression artifacts [2].
More detailed information on the dataset can be found in [1], or you can refer to [2] for more information about the objective-objective correlation experiment.
Contact: Michela Testolina ([email protected]).
License
The data are licensed under a Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/).
Download
The dataset can be downloaded through FTP by using dedicated FTP clients, such as FileZilla or FireFTP (we recommend to use FileZilla):
Protocol: FTP
FTP address: tremplin.epfl.ch
Username: [email protected]
Password: ohsh9jah4T
FTP port: 21
After you connect, choose the JPEG_AI_subjective folder from the remote site, and download the relevant material. The total size of the provided data is ~735 MB.
Please read the README files for further information on the structure and the usage of the material.
Conditions of use
The data are licensed under a Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/). You are free to share (— copy and redistribute the material in any medium or format) and adapt (— remix, transform, and build upon the material for any purpose, even commercially) the provided data.
In no event shall the Ecole Polytechnique Fédérale de Lausanne (EPFL) be liable to any party for direct, indirect, special, incidental, or consequential damages arising out of the use of the data and its documentation. The Ecole Polytechnique Fédérale de Lausanne (EPFL) specifically disclaims any warranties. The data provided hereunder is on an “as is” basis and the Ecole Polytechnique Fédérale de Lausanne (EPFL) has no obligation to provide maintenance, support, updates, enhancements, or modifications.
If you wish to use any of the provided material in your research, we kindly ask you to cite [1] and [2].
References
[1] Evgeniy Upenik, Michela Testolina, João Ascenso, Fernando Pereira and Touradj Ebrahimi, “Large-Scale Crowdsourcing Subjective Quality Evaluation of Learning-Based Image Coding,” 2021 IEEE Visual Communications and Image Processing (VCIP). IEEE, 2021.
[2] Michela Testolina, Evgeniy Upenik, João Ascenso, Fernando Pereira, and Touradj Ebrahimi. “Performance Evaluation of Objective Image Quality Metrics on Conventional and Learning-Based Compression Artifacts.” In 13th International Conference on Quality of Multimedia Experience (QoMEX). 2021.