Quality assessment for JPEG images based on difference of power spectrum distribution

Abstract

No-reference quality assessment aims at designing objective assessment criteria consistent to subjective perceived quality without any knowledge about reference image. This paper proposes a no-reference quality assessment algorithm specific to JPEG images. Blocking artifact in JPEG images is caused by the block based quantization of frequency coefficients, which is equivalent to applying low pass filtering in each block. In view of this idea, the algorithm in this paper was used to realize the quality assessment of JPEG images by quantizing the difference of power spectrum distribution between inner-block and inter-block. The assessment method proposed in this paper owns low algorithm complexity, clear physical meanings, free from learning and training and other advantages. Compared with most presented algorithms, the assessment results of proposed algorithm demonstrate a higher correlation to the subjective perceived quality.

Key words： image quality assessment; blocking artifact; power spectrum distribution

Introduction

No-reference assessment method does not require any knowledge about the reference images in the assessment process, and it can adapt to almost all the application occasions [ 1, 2]. Therefore, no-reference assessment methods have been paid much more attention. Since the performance of most general-purpose methods is quite limited for JPEG images [ 3− 5], some algorithms specific to JPEG images have also been proposed [ 6− 10]. This paper proposed a no-reference assessment method based on difference of power spectrum distribution (DPSD), called DPSD method. This algorithm can be adopted to effectively assess the blocking artifact in JPEG images. Different from most presented algorithms, this algorithm focuses on the influences of blocking artifact on the power spectrum distribution (PSD), instead of directly focusing on the boundary effects caused by blocking artifact.

Algorithm

Block extending

JPEG compression algorithm uses color sampling, frequency coefficients quantization, predictive coding and Huffman coding to realize data compression. In these four processes, only the former two processes may lead to information loss, which will further result in the impairment of image quality. Due to limited space, the influence of color sampling on image quality is not discussed here. Only the influence of blocking artifact caused by quantization process on image quality is considered.

The primary cause of blocking artifact is block based quantization of frequency coefficients in the compression process. The quantization equals to implementing a low pass filter in image block. Low pass filtering not only contributes to the homogeneity of inner-block pixels, but also leads to the discontinuity of inter-block pixels. Figure 1 shows the blocking artifacts in JPEG images. As is seen, the pixel gray level of inner-block tends to unification. At the same time, at the boundary of the block, obvious gray difference has formed.

Fig.1 Blocking artifacts in JPEG image

Full size|PPT slide

In view of the frequency domain of signal, low pass filtering makes the power distribution of inner-block pixels highly centralized in the low-frequency. However, once the block boundary is crossed, the power distribution will immediately spread from low frequency to high frequency. In consideration of this thought, further analysis is made on the power spectrum distribution characteristic of image block in different cases.

According to the block scheme of JPEG compression algorithm, images are divided into certain blocks of 8 × 8 (For the convenience of illustration, it is called regular block), noted as B_i, i=1,2,...,N, which represents the number of regular block in images. To compare the different characteristics of power spectrum distribution between inner-block and inter-block, naturally-extending block and edge-extending block are defined. In the scope of images, a regular block is extended by 1 pixel all around, forming an image area of 10 × 10, which is called naturally-extending block. The naturally-extending block derived from a regular block B_i is denoted as {\tilde{B}}_i . A naturally-extending block contains the corresponding regular block and its block boundaries. Therefore, its power spectrum distribution represents the inter-block spectrum characteristic. Furthermore, to maintain the energy consistency, boundary pixels of a regular block B_i are copied and extended by 1 pixel all around, forming one image area of 10 × 10, which is called edge-extending block. The edge-extending block derived from a regular block B_i is denoted as {\tilde{B}}_i . An edge-extending block does not contain block boundary. Thus, its power spectrum distribution represents the inner-block spectrum characteristic. Figure 2 shows an example.

Fig.2 Naturally-extending block and edge-extending block. (a) 10 × 10 naturally-extending block; (b) 10 × 10 edge-extending block

Full size|PPT slide

In Fig. 2, the rectangle area marked by a thicker border is a regular block in image. Figure 2(a) refers to 10 × 10 naturally-extending block that naturally extends in image scopes; and Fig. 2(b) refers to 10 × 10 edge-extending block, which is formed by copying and extending around the boundary of regular block. It can be seen that when a regular block extends into naturally-extending block, the pixel gray level has a significant change at the block boundary due to blocking artifact. However, when one regular block extends to edge-extending block, the overall distribution of pixel gray level has no significant change due to local autocorrelation of image content.

Power spectrum is a distribution of signal power according to frequency. To simplify calculation, discrete cosine transform (DCT), instead of discrete Fourier transform (DFT), is used to calculate the power spectrum of images. Suppose the space domain distribution of block B_i is f_i(x,y) , then its frequency spectrum and power spectrum can respectively be expressed as

(1) F_i(u,v)=DCT\mathrm{2}\{f_i(x,y)\},\hspace{1em}i=\mathrm{1},\mathrm{2},\cdots ,N,

(2) P_i(u,v)={\left|F_i(u,v)\right|}^{\mathrm{2}},\hspace{1em}i=\mathrm{1},\mathrm{2},\cdots ,N.

According to these formulas, we can calculate the corresponding power spectrums of naturally-extending blocks {\tilde{B}}_i and edge-extending blocks {\bar{B}}_i of regular block B_i, respectively noted as {\tilde{P}}_i(u,v) and {\bar{P}}_i(u,v) . According to the above explanation, {\bar{P}}_i(u,v) is called inner-block power spectrum and {\tilde{P}}_i(u,v) is called inter-block power spectrum. After zigzag scanning, one dimensional form can be obtained to present the inner-block power spectrum, {\bar{P}}_i(k) and inter-block power spectrum, {\tilde{P}}_i(k) .

(3) {\tilde{P}}_i(k)=ZIGZAG\left\{{\tilde{P}}_i(u,v)\right\},\hspace{1em}k=\mathrm{1},\mathrm{2},\cdots \mathrm{100},

(4) {\bar{P}}_i(k)=ZIGZAG\left\{{\bar{P}}_i(u,v)\right\},\hspace{1em}k=\mathrm{1},\mathrm{2},\cdots \mathrm{100.}

Some sample images are selected and several regular blocks are randomly selected for test. Among the power spectrums of naturally-extending blocks and edge-extending blocks of these regular blocks, qualitative analysis and comparison are made. Figure 3 shows an example. Among them, Fig. 3(a) refers to the comparison chart between inner-block power spectrum and inter-block power spectrum in images without blocking artifacts; and Fig. 3(b) refers to the comparison chart between inner-block power spectrum and inter-block power spectrum in images with blocking artifacts. The curves of inner-block power spectrum distribution and inter-block power spectrum distribution are respectively drawn by red line and blue line. Since direct current (DC) coefficients have much larger values than alternating current (AC) coefficients in general, plotting DC coefficients in the same axis as AC coefficients will result in very small amplitude for most AC coefficients and so they are difficult to observe. In the other hand, most of the AC coefficients with a higher index beyond about 40 have zero values, so they can be omitted in the plot. To highlight the distribution characteristic of AC coefficients, the power spectrum distribution is drawn only from the 2nd to the 40th frequency points.

Fig.3 Comparison chart of power spectrum distribution curves. (a) Without blocking artifacts; (b) with blocking artifacts

Full size|PPT slide

We can draw the following conclusion based on the results described as above. According to the different image contents, their power spectrum distribution differs from each other. However, the difference between inner-block power spectrum distribution (red curve) and inter-block power spectrum distribution (blue curve) become significant due to blocking artifacts.

Measurement of blocking artifact

In the above section, qualitative analysis is made on difference of power spectrum distribution caused by blocking artifact. In this section, the impairment of blocking artifact on image quality will be quantitatively assessed. The power spectrum distribution characteristic function can be defined as follows.

(5) Q_i={\displaystyle \sum _{k=T}^{\mathrm{100}}{P}_i(k)}/{\displaystyle \sum _{k=\mathrm{1}}^T{P}_i(k)},

where T represents the index thresholds of high frequency and low frequency. Power spectrum distribution characteristic function is the ratio of high frequency energy to low frequency energy, reflecting the concentration level of power spectrum distribution to low-frequency part. Difference of power spectrum distribution function is defined as follows.

(6) S_i={\displaystyle \frac{{\bar{Q}}_i-{\tilde{Q}}_i}{{\tilde{Q}}_i}},

(7) S=\Vert S_i\Vert =\Vert {\displaystyle \frac{{\bar{Q}}_i-{\tilde{Q}}_i}{{\tilde{Q}}_i}}\Vert =\sqrt[\beta ]{{\displaystyle \frac{\mathrm{1}}{N}}{\displaystyle \sum _{i=\mathrm{1}}^N{\left|{\displaystyle \frac{{\bar{Q}}_i-{\tilde{Q}}_i}{{\tilde{Q}}_i}}\right|}^{\beta }}},

(8) DPSD=\alpha S^{\beta }+\gamma ,

where α=163.37, β=0.2238 and γ=−98.7501. The transformation expressed by Eq. (8) is used to perform a regression analysis, so that the result is more significantly correlated to the subjective ratings. The values of \alpha , \beta , and \gamma can be calculated by well-known linear regression analysis methods.

Human vision model

Studies have demonstrated that blocking artifact intensity has direct and significant correlation with image perceived quality. However, its impairment on the image quality is still related to the characteristic of image content. These characters include texture and luminance [ 11, 12]. Therefore, after the measurement of blocking artifact, most algorithms also utilize texture mask coefficient and luminance mask coefficient to amend the measurement results to obtain a quality index which can accurately match the subjective perceived quality. However, after the test, it is found that the proposed algorithm can obtain satisfying assessment results without this step. The reason for this is that the above-mentioned two factors have been considered in the assessment function.

First, Eq. (6) is applied to evaluate blocking artifact intensity of one regular block. It will be noticed that {\tilde{Q}}_i is a value dependent on low frequency energy. Suppose, the denominator represents DC component of the image block, namely, mean luminance. This means this index has considered the luminance mask coefficient of common blocks.

Secondly, S_i can also be written into

(9) S_i={\displaystyle \frac{{\bar{Q}}_i-{\tilde{Q}}_i}{{\tilde{Q}}_i}}={\left({\bar{Q}}_i-{\tilde{Q}}_i\right)}^{\alpha }{\left({\displaystyle \frac{\mathrm{1}}{\tilde{Q}}}\right)}^{\beta }={[S_i^{(\mathrm{1})}]}^{\alpha }{[S_i^{(\mathrm{2})}]}^{-\beta },

where, S_i^{(\mathrm{1})} represents the physical intensity of blocking artifact, S_i^{(\mathrm{2})} represents the power spectrum distribution of naturally-extending blocks. According to human visual system (HVS), the blocking artifact in smooth area has more significant impairment on image quality, so lower scores or higher values of S_i should be obtained for the blocks in smooth area. Compared to texture area, the naturally-extending blocks in smooth area have a relatively centralized distribution of power spectrum. This means the {\tilde{Q}}_i of these blocks are smaller than those in texture area. Consequently, a larger factor S_i^{(\mathrm{2})} is weighted to S_i^{(\mathrm{1})} , which represents the physical intensity of blocking artifact, and a higher values of S_i is obtained. So S_i^{(\mathrm{2})} reflects the texture mask coefficient of blocking artifact.

Experiment

To test the performance of the algorithm proposed in this paper, LIVE database [ 13] is selected as the test image source. LIVE database contains 982 test images generated from 29 reference images and their difference mean opinion scores (DMOS). Among them, 233 JPEG distorted images are selected to test the algorithm in this paper. Figure 4 provides the assessment results and the scatter chart of DMOS.

Fig.4 Scatter chart of DPSD and DMOS

Full size|PPT slide

It can be found from the figure that there is a significant correlation between the objective assessment score DPSD and subjective assessment score DMOS.

Meanwhile, the algorithm proposed in this paper is compared with several presented algorithms, including classic full-reference assessment algorithms (for example, peak signal noise ratio (PSNR) and structural similarity (SSIM)) and no-reference assessment algorithms specific to JPEG images. Pearson correlation coefficient and Spearman rank correlation coefficient are used to assess the performance of these algorithms. The Pearson correlation coefficient is the most widely used because it can measure the strength of the linear relationship between normally distributed variables. However, when the variables are not normally distributed or the relationship between the variables is not linear, it may be more appropriate to use the Spearman rank correlation method. Here, both correlation coefficients are calculated and listed in Table 1.

Tab.1 Correlation coefficients between subjective and objective assessment scores

	Pearson	Spearman
full-reference	PSNR	0.903	0.883
SSIM	0.946	0.947
no-reference	GBIM [14]	0.736	0.912
	LABM [7]	0.834	0.832
	NPBM [12]	0.900	0.904
	BAM_SGP [15]	0.941	0.925
	DPSD	0.955	0.931

It can be seen that the algorithm proposed in this paper gets the highest value of Pearson coefficient. That means it has better performance than others in term of ‘linear correlation’. With respect to Spearman coefficient, which is a measure of ‘monotonic correlation’, the proposed algorithm has the second best performance after the full-reference algorithm- SSIM.

Conclusions

This paper proposed one no-reference quality assessment algorithm based on difference of power spectrum distribution. This algorithm can effectively assess the blocking artifacts of JPEG images. The assessment method proposed in this paper owns low algorithm complexity, clear physical meanings, free from learning and training and other advantages. Compared with most presented full-reference assessment algorithms and JPEG-specific assessment algorithms, this algorithm demonstrated a higher performance in term of correlation to the subjective perceived quality.

References

1 Wang Z, Bovik A C. Modern image quality assessment. Modern Image Quality Assessment, 2006, 2(1): 1−156

2 Wang Z, Bovik A C. Reduced- and no-reference image quality assessment. Signal Processing Magazine, IEEE, 2011, 28(6): 29−40

3 Saad M A, Bovik A C, Charrier C. A DCT statistics-based blind image quality index. Signal Processing Letters, IEEE, 2010, 17(6): 583−586

4 Mittal A, Moorthy A K, Bovik A C. No-reference image quality assessment in the spatial domain. IEEE Transactions on Image Processing, 2012, 21(12): 4695−4708

5 Jiao S, Qi H, LinW, Shen W. Fast and efficient blind image quality index in spatial domain. Electronics Letters, 2013, 49(18): 1137−1138

6 Wang Z, Sheikh H R, Bovik A C. No-reference perceptual quality assessment of jpeg compressed images. In: Proceedings of International Conference on Image Process. 2002, 1: 477−480

7 Pan F, Lin X, Rahardja S, Lin W. A locally adaptive algorithm for measuring blocking artifacts in images and videos. In: Proceedings of the 2004 International Symposium on Circuits and Systems. 2004, 3: 925−928

8 Pan F, Lin X, Rahardja S, Ong E P, Lin W S. Using edge direction information for measuring blocking artifacts of images. Multidimensional Systems and Signal Processing, 2007, 18(4): 297−308

9 Perra C, Massidda F, Giusto D D. Image blockiness evaluation based on Sobel operator. In: Proceedings of IEEE International Conference on Image Processing. 2005, 1: 389−392

10 Chen C, Bloom J A. A blind reference-free blockiness measure. In: Advances in Multimedia Information Processing- PCM 2010. 2010, 6297: 112−123

11 Liu H, Heynderickx I. A simplified human vision model applied to a blocking artifact metric. Computer Analysis of Images and Patterns, 2007, 4673: 334−341

12 Liu H, Heynderickx I. A no-reference perceptual blockiness metric. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. 2008, 1: 865−868

13 Sheikh H R, Wang Z, Cormackl L, Bovik A C. Live image quality assessment databases release2. http://live.ece.utexas.edu/research/quality/

14 Wu H R, Yuen M. A generalized block-edge impairment metric for video coding. Signal Processing Letters, IEEE, 1997, 4(11): 317−320

15 Chen J, Zhang Y, Liang L, Ma S, Wang R, Gao W. A no-reference blocking artifacts metric using selective gradient and plainness measures. In: Advances in Multimedia Information Processing- PCM 2008. 2008, 5353: 894−897