Scalable Video Database ‒ MMSPG ‐ EPFL

Introduction

Scalable video coding schemes offer an efficient alternative to simulcast for applications where content needs to be distributed over many heterogeneous clients with different network conditions and decoding/display capabilities. Three-dimensional scalability inherent in scalable video coding, i.e. spatial, temporal, and quality scalability, enables such adaptive content distribution. An important issue for effective exploitation of scalability is to understand subjective preference and priority between different scalability configurations in the three-dimensional scalability space.

For this, we conducted extensive subjective tests for scalable video coding, where two different scalable video codecs and three different high definition (HD) contents were considered. The video material used in our experiments and the subjective data obtained by using the paired comparison methodology, named Scalable Video Database, are made publicly available in order to provide researchers with a benchmark database for reproducible research.

Scalable Video Database description

A brief description of the database is given below. For more details, please refer to our paper [3,4].

Video data

The video data are summarized as:

2 scalable video codecs: SVC [1] and wavelet-based codec [2]
3 HD contents: DucksTakeOff, IntoTree, and ParkJoy (originally from The SVT High Definition Multi Format Test Set: available at http://www.its.bldrdoc.gov/vqeg; documentation at ftp://vqeg.its.bldrdoc.gov/HDTV/SVT_MultiFormat/SVT_MultiFormat_v10.pdf)
Bit-rates ranging between 300 kbps and 4 Mbps
3 spatial resolutions: 320×180, 640×360, and 1280×720
4 temporal resolutions: 6.25 fps, 12.5 fps, 25 fps, 50 fps

In total, 28 and 44 video sequences were considered for each codec, respectively. The scalable bit streams and the decoders are provided, from which the video data in the yuv 4:2:0 format can be extracted by using the accompanied batch files.

Subjective rating data

Two different test methodologies were used in our experiments: paired comparison and single stimulus methodologies. The test methodology and environment are described in the papers [3,4] in details.

In paired comparison, two different video sequences with different scalability configurations were shown in a side-by-side manner and the subjects were asked to give their preference between 3 possible choices: left, same, right. The excel table containing the number of votes between the 3 choices for 16 subjects are provided here. Software for processing the ratings and converting them into quality scores will also be available at a later date.

In single stimulus tests, each video sequence was shown to the subjects and rated within a range of [0, 100]. The excel table provided contains the rating values of 16 subjects.

Download

You can use the results in the database in your research without any conditions, as long as you clearly mention and acknowledge the following paper :

J.-S. Lee, F. De Simone, T. Ebrahimi, “Subjective quality evaluation via paired comparison: application to scalable video coding,” IEEE Transactions on Multimedia, vol. 13, no. 5, pp. 882-893, Oct. 2011. [detail] [pdf]

J.-S. Lee, F. De Simone, N. Ramzan, Z. Zhao, E. Kurutepe, T. Sikora, J. Ostermann, T. Ebrahimi, “Subjective evaluation of scalable video coding for content distribution,” in Proc. ACM Multimedia, Firenze, Italy, pp. 65-72, Oct. 2010. [detail] [pdf]

Also, if you download the data, it is assumed that you agree to the copyright notice below.

The zip files are available for download:

Video data produced by SVC (bit streams and decoder executable under Linux)
Video data produced by wavelet-based codec (bit streams and decoder executable under MS-Windows)
Subjective rating data based on paired comparison tests (excel file)
Subjective rating data based on single stimulus tests (excel file- small error corrected on 8 Feb. 2012)
This page in pdf

The files are protected by a password. If you want to obtain it, please send an email to [email protected].

No guarantee or support is provided for the data provided, and by downloading it you explicitly accept that its use is at your own risks and any eventual direct or indirect damages of any form are not under our responsibility.

Copyright notice

Permission is hereby granted, without written agreement and without license or royalty fees, to use, copy, modify, and distribute the data provided and its documentation for research purpose only. The data provided may not be commercially distributed.

In no event shall the Ecole Polytechnique Fédérale de Lausanne (EPFL) be liable to any party for direct, indirect, special, incidental, or consequential damages arising out of the use of the data and its documentation.

The Ecole Polytechnique Fédérale de Lausanne (EPFL) specifically disclaims any warranties. The data provided hereunder is on an “as is” basis and the Ecole Polytechnique Fédérale de Lausanne (EPFL) has no obligation to provide maintenance, support, updates, enhancements, or modifications.

The use of the data is conditional to the users explicitly and clearly mentioning and acknowledging the following publication :

Contact

If you have any question, please contact Jong-Seok Lee ([email protected]).

Acknowledgment

This work was supported by the European Community’s Seventh Framework Programme (FP7/2007-2011) under grant agreement no. 21644 (PetaMedia) and the Swiss National Foundation for Scientific Research in the framework of the NCCR Interactive Multimodal Information Management (IM2).

References

[1] J. Reichel, H. Schwarz, and M. Wien, “Joint scalable video model (JSVM),” Joint Video Team, doc. JVT-X202, Jul. 2007.

[2] N. Ramzan, T. Zgaljic, and E. Izquierdo, “An efficient optimization scheme for scalable surveillance centric video communications,” Signal Processing: Image Communication, vol. 24, no. 6, pp. 510-523, 2009.

[3] J.-S. Lee, F. De Simone, N. Ramzan, Z. Zhao, E. Kurutepe, T. Sikora, J. Ostermann, T. Ebrahimi, “Subjective evaluation of scalable video coding for content distribution,” in Proc. ACM Multimedia, Firenze, Italy, pp. 65-72, Oct. 2010.

[4] J.-S. Lee, F. De Simone, T. Ebrahimi, “Subjective quality evaluation via paired comparison: application to scalable video coding,” IEEE Transactions on Multimedia, vol. 13, no. 5, pp. 882-893, Oct. 2011.