Quality assessment method for geometrically distorted images

Abstract

The objective assessment of image quality is important for image processing, which has been paid much attention to in recent years. However, there were few reports about objective quality assessment methods for geometrically distorted images. Different from the routine image degradation processing (for example, noise addition, contrast change and lossy compression), the geometric distortion results in the changes of the spatial relationship of image pixels, which makes the traditional quality assessment algorithms, such as mean square error (MSE) and peak signal to noise ratio (PSNR) failure to obtain expected assessment results. In this paper, a full reference image quality assessment algorithm is proposed specifically for the quality evaluation of geometrically distorted images. This assessment algorithm takes into account three key factors, such as distortion intensity, distortion change rate and line feature index for perceptual quality assessment of images. Experimental results in this study show that the proposed assessment algorithm not only is significantly better than those of the traditional objective assessment methods such as PSNR and structural similarity index measurement (SSIM), but also has significant correlation with human subjective assessment.

Key words： image quality assessment; geometrical distortion; displacement vector field; line feature index; Hough transform

Introduction

Objective image quality assessment (IQA) has been widely applied in many fields, such as image quality online monitoring, image automatic classification and image watermarking performance assessment [1-3]. For example, for parameterized image restoration, the parameters of the image restoration algorithm need to be automatically adjusted according to actual restoration effects. It can be solved through the feedback system including objective quality assessment algorithm module, as shown in Fig. 1.

Fig.1 Typical application block diagram of IQA

Full size|PPT slide

In traditional solution, the evaluation on the performance of image restoration algorithms is carried out by observing output image subjectively. In above system, the evaluation task is implemented by IQA module instead. The evaluating result is employed to tune the parameters of image restoration module until satisfied image quality is obtained.

The traditional assessment algorithms, such as mean square error (MSE) and peak signal to noise ratio (PSNR), are widely used, and some other objective quality assessment algorithms have also been proposed in recent years, for example, modified PSNR [4], unique quality index (UQI) [5 ] and structural similarity index measurement (SSIM) [6]. These assessment results not only have been verified to be feasible and effective, but also have distinct correlation with the subjective quality assessment results in most cases [4-6].

However, most of previous quality assessment methods cannot deal with geometric distortion of images, while it needs to be considered in some practical applications, such as video transmission system and image scanning system. Especially, the geometric attack to the digital watermarking also becomes a hot topic, and assessment of image degradation caused by the geometric attacks is also an urgent technical problem to be solved [7-9].

Though geometric distortion does not directly affect the brightness and chrominance of image pixels, it does change the spatial relationship of pixels. Furthermore, its mathematical model is difficult to express analytically. Therefore, it is more difficult than other distortion models. What’s more, the perceptive distortion caused by geometric distortion is relevant not only to the intensity of the geometric distortion, but also to the structural features of the target images. Therefore, if we neglect the structural features of the target images, the physical distortion in the images may be far from the perceptive distortion.

For this reason, some quality assessment algorithms specific to geometrically distorted images have been developed in recent years. Wang and Simoncelli [10], Sampat et al. [11] presented the image similarity assessment algorithms based on complex wavelet domain. The small displacement in spatial domain (which can be regarded as the simplest geometric distortion) only manifests phase change in wavelet domain, so these algorithms are not sensitive to the spatial displacement. But they still failed to effectively deal with more complex geometric distortion.

Setyawan et al. also reported a quality assessment algorithm specific to geometric distortion [12]. In this algorithm, complex geometric distortion model is deemed as a combination of some simple distotion models, which can be described by paramerized affine transforms, thus the image quality can be evaluated by pooling all these simple distotion models. Since most of geometric distortions have strong randomness, this modeling method has its inherent limitations. At the same time, this method ignores the feature information of the target images, which leads to the great performance differences of the algorithm when dealing with the images of different types.

Then, D’Angelo et al. obtained better experimental effects based on two-dimensional Garbo transform algorithm [13]. This algorithm not only takes the physical intensity of the geometric distortion into consideration, but also covers the influence of distortion on the structural information in the images. However, this proposed algorithm still neglects some influence factors on the perceptive quality, for example, distortion change rate (reflecting the change degree of distortion intensity) and line features of the target images [13].

In this paper, we have proposed a new quality assessment algorithm, which considers the distortion intensity, distortion change rate and line feature index as three key factors of perceptive quality. Experimental results suggested that the problem of the mapping from physical distortion to perceptive distortion had been effectively solved.

Mathematical model of geometric distortion

Mathematical models of geometric distortion include local affine transform, Markov Random Field (MRF), Local Permutation with Cancelation and Duplication (LPCD) [14] and so on. Because most of these models are difficult to express analytically, displacement vector field (DVF) is often used to describe those geometric distortion models and can be obtained through general registration algorithm according to reference images and distorted images [15].

To generate geometric distortion artificially, we proposed one DVF model based on smoothened white random field (SWRF). SWRF takes white noise as two components of the vector field, and it utilizes Gaussian filter to obtain smooth displacement components. When SWRF model is employed, the strength and inregularity of distortion can be effectively controlled, and it also has lower computational complexity than MRF. Due to limitations of space, the details about this topic are left out in this paper. Figure 2 shows the displacement field models of MRF and SWRF respectively. Like MRF, this model can generate smooth displacement vector field and thus is suitable for building image database which is used for later experiments.

Fig.2 Complex DVF. (a) MRF; (b) SWRF

Full size|PPT slide

Assessment algorithms

It was thought before that it is difficult to make breakthrough in the assessment algorithm of geometric distortion, because there are various factors influencing the image quality evaluation of human eyes. Different from blurring, noise and other image degradation processes, geometric distortion changes the spatial relationship of the image pixels. The existing quality assessment algorithms or their derivative algorithms usually are based on ‘pixel level comparison’, so the small changes of the pixel position in the image may have significant effects on the assessment results, but it has little influence on perceptual quality for human eyes. Figure 3 shows one reference image (Fig. 3(a)) and two distorted images (Figs. 3(b) and 3(c)) derived from Fig. 3(a). Figure 3(b) is applied with the slight local random geometric transform and Fig. 3(c) is applied with random salt and pepper noises of certain intensity. By comparison, we can find that the perceptive quality of Fig. 3(b) is obviously better than that of Fig. 3(c), but their PSNR are 17.18 and 19.06 dB respectively. This indicates that PSNR cannot exactly reflect the geometric distortion in the images.

Fig.3 Influences of geometric distortion and additive noise on PSNR. (a) Reference image; (b) geometrical distorted image (PSNR: 17.18 dB); (c) image with noise (PSNR: 19.06 dB)

Full size|PPT slide

To be good correlation with the human subjective assessment results (MOS), we consider three key factors, such as distortion intensity, distortion change rate and line feature index, in the objective assessment algorithm, which are related to the perceptual distortion.

Distortion intensity

The distortion intensity of each pixel can be represented by the length of displacement vector of this pixel. Meanwhile, the distortion intensity of a whole image can be represented by the normalized average of the distortion intensities of all pixels. In our proposed algorithm, the actually-applied distortion intensity is the ratio of normalized average to the image size. The image size is represented by half diagonal length of images, noted as

Z=\sqrt{M^{\mathrm{2}}+N^{\mathrm{2}}}/\mathrm{2}.

Therefore, the calculation formula of distortion intensity Q_s can be expressed as

(1)Q_s={\displaystyle \frac{\sqrt{{\displaystyle \frac{\mathrm{1}}{MN}}{\displaystyle \sum _{x,y\in I}{\left|\mathit{d}(x,y)\right|}^{\mathrm{2}}}}}{Z}}={\displaystyle \frac{\mathrm{2}}{\sqrt{M^{\mathrm{2}}+N^{\mathrm{2}}}}}\sqrt{{\displaystyle \frac{\mathrm{1}}{MN}}{\displaystyle \sum _{x,y\in I}{\left|\mathit{d}(x,y)\right|}^{\mathrm{2}}}},

where M and N respectively represent the width and height of the image; \mathit{d}(x,y) represents the displacement vector field. This paper supposes the value range of length of displacement vector as [\mathrm{0},Z], and thus the value range of Q_s falls [\mathrm{0},\mathrm{1}].

Distortion change rate

Distortion change rate is the variation of DVF, which can be solved through variance analysis or Fourier analysis. Variance only embodies the dispersion degree of DVF amplitude value, while it fails to reflect the intensity changes in space. Therefore, it cannot correctly reflect the influence of DVF on the perceptive quality. Fourier transform is adopted to calculate the distortion change rate. First, the Fourier transform of the DVF of the image can be realized through Clifford algebra [16]. For simplification, we replace it with component based Fourier transforms, and then calculate the energy proportion of top 25% coefficient in the spectral matrix. The calculation formula of distortion change rate can be expressed as

(2)Q_r={\displaystyle \frac{{\displaystyle \sum _{u,v\in \omega }{\left|\mathit{D}(u,v)\right|}^{\mathrm{2}}}}{{\displaystyle \sum _{u,v\in \mathrm{\Omega }}{\left|\mathit{D}(u,v)\right|}^{\mathrm{2}}+ϵ}}},

where \mathit{D}(u,v)represents Fourier transform of displacement vector field \mathit{d}(x,y);

ϵ is a very small constant, which is introduced to avoid the denominator gets zero value. The value range of Q_r falls [\mathrm{0},\mathrm{1}].

Line feature index

From HVS, the human eye is more sensitive to distortion of straight lines, and this point also can be proved through MOS test. When there are many line features in images, the influence of geometric distortion on perceptive quality is more significant. To evaluate the line feature intensity in images, we proposed the line feature index defined as the total length of all straight-line segments in the images.

The calculation of the line feature index is involved in the issue of straight-line detection in images. There are many straight-line detection methods in the images. The typical ones include the Hough transform [17] and the Ridgelet transform [18]. The coefficient intensity of parameter space of Hough transform directly reflects the straightness in the spatial domain, and the straight line objects can be selected and filtered through parameter adjustment, so the Hough transform is used here to calculate the line feature index of the image. The calculation process is described as following:

1) Extract image edges by Canny operator;

2) Apply Hough transform on the image containing edge information and obtain coefficient map image in parameter space;

3) Extract K most significant line objects;

4) Calculate the total length of all line objects.

The final line feature index can be expressed with the following equation.

(3)L={\displaystyle \sum _{i=\mathrm{1}}^{K}L(i)},

where L(i)represents the line length and K represents the number of the investigated lines. In this algorithm, K is assigned to 10.

Figure 4 shows the calculation process. Figure 4(a) shows the edge image obtained through Canny operator. Figure 4(b) shows the coefficient map image in parameter space through Hough transform. Figure 4(c) shows the detected lines.

Fig.4 Line features obtained from Hough transform. (a) Edge image obtained through Canny operator; (b) coefficient map image in parameter space through Hough transform; (c) detected lines

Full size|PPT slide

Objective assessment score

In conclusion, the assessment algorithm can be expressed by the following equations.

(4)\{\begin{array}{l}\tilde{Q}=\alpha {\displaystyle \frac{\mathrm{2}}{\sqrt{M^{\mathrm{2}}+N^{\mathrm{2}}}}}\sqrt{{\displaystyle \frac{\mathrm{1}}{MN}}{\displaystyle \sum _{x,y\in I}{\left|\mathit{d}(x,y)\right|}^{\mathrm{2}}}}+\beta {\displaystyle \frac{{\displaystyle \sum _{u,v\in \omega }{\left|\mathit{D}(u,v)\right|}^{\mathrm{2}}}}{{\displaystyle \sum _{u,v\in \mathrm{\Omega }}{\left|\mathit{D}(u,v)\right|}^{\mathrm{2}}+ϵ}}},\hfill \\ \widehat{Q}={\displaystyle \frac{L}{\max \{M,N\}}}\tilde{Q}+\mathrm{1}-{\displaystyle \frac{L}{\max \{M,N\}}},\hfill \\ Q=\mathrm{4}\widehat{Q}+\mathrm{1},\hfill \end{array}.

where,

\alpha +\beta =\mathrm{1};

\tilde{Q} represents the distortion assessment index determined by displacement vector field, which is superposed of distortion intensity and distortion change rate. The weight of superposition is controlled by \alpha and \beta. \widehat{Q} represents the amended index on \tilde{Q} by line feature index L. In this way, it can embody the influence of image line feature on perceptive quality. Q represents the final assessment score. Through a simple linear transform on \widehat{Q}, the value range falls [\mathrm{1},\mathrm{5}].

Experimental results and discussion

Most image databases [19] have been designed for noise, compression, blurring and other degradation processes rather than for geometric distortion. Therefore, these image databases are not suitable for the test of this algorithm. To test the validity of this algorithm in experiments, 20 volunteers have been recruited for MOS test for geometrically distorted images. Procedures for MOS test have been designed by following the ITU-T Recommendation P.910,13 which suggests standard viewing conditions, criteria for the selection of observers and test material, assessment procedures, and data analysis methods [20]. After training, all the volunteers are left in completely same objective environment, and required to give their subjective assessment of image in fixed time. Different reference images are used for this test, including figure, fruit, buildings, natural landscape, and so on. SWRF model is used to generate geometric distortions with 20 different parameters and scales for each reference image, and finally 200 test images are obtained. Those 200 images are distributed to 20 volunteers for MOS test, and 200 MOS scores are presented by those volunteers. At the same time, objective assessment algorithm proposed in this paper is used to assess those images and also 200 objective assessment scores are obtained. Finally, hash chart, polynomial trendline and correlation coefficient are drawn from these scores, as shown in Fig. 5. Results of three reported objective assessment algorithms including PSNR, SSIM [6] and assessment algorithm based on Gabor [13] are also shown in Fig. 5

Fig.5 Hash chart of subjective and objective assessment scores. (a) PSNR; (b) SSIM; (c) Gabor; (d) our metric

Full size|PPT slide

From Figs. 5(a) and 5(b), it can be observed that there is no correlation between MOS and the assessment results of PSNR and SSIM. The reason is that the influence of geometric distortion on image quality is not taken into account in the traditional assessment algorithms of PSNR and SSIM. The assessment results of the assessment algorithm based on Gabor is shown in Fig. 5(c), which demonstrate a certain correlation with MOS. However, the algorithm proposed in this paper has significant correlation with the human subjective assessment MOS.

To assess the validity of the proposed algorithm, three kinds of correlation coefficients between subjective scores and objective scores, including Pearson, Spearman and Kendall, are shown in Table 1. Since there is no correlation between MOS and the assessment results of PSNR and SSIM, only the coreelation coefficients of algorithm based on Garbo and our algorithm are calculated and listed in the table. It can be found that our algorithm gets higher values than the algorithm based on Garbo on all three kinds of coreelation coefficients.

Tab.1 Correlation coefficients between subjective and objective assessment scores

	Pearson	Spearman	Kendall
Gabor based metric	0.569	0.733	0.573
our metric	0.869	0.839	0.677

Conclusions

Through consideration of distortion intensity, distortion change rate and image line features, the objective assessment results by proposed algorithm in this paper have good consistency with human subjective perception and significant correlation with MOS. Moreover, the proposed algorithm suitable to geometrically distorted images has better performance than that of the traditional quality assessment algorithm.

Acknowledgements

The authors would like to thank Professor Zhenyu Yang for insightful comments as well as all the volunteers for participating in the MOS test.

References

1 Supriyanto E, Lau E X, Seow S C. Automatic image quality monitoring system for low cost ultrasound machine. In: Proceedings of the International Conference on Information Technology and Applications in Biomedicine (ITAB 2008), 2008, 183–186

2 Nezhadarya E, Wang Z J, Ward R K. Image quality monitoring using spread spectrum watermarking. In: Proceedings of 16th IEEE International Conference on Image Processing (ICIP 2009), 2009, 2233–2236

3 Wang Z. Applications of objective image quality assessment methods. IEEE Signal Processing Magazine, 2011, 28(6): 137–142

4 Gupta P, Srivastava P, Bhardwaj S, Bhateja V. A modified PSNR metric based on HVS for quality assessment of color images. In: Proceedings of the International Conference on Communication and Industrial Application (ICCIA), 2011, 1-4

5 Wang Z, Bovik A C. A universal image quality index. IEEE Signal Processing Letters, 2002, 9(3): 81–84

6 Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13(4): 600–612

PMID

7 Xie R S, Zhou M, Huang C L, Li Y M. Anti-geometrical attacks image watermarking scheme based on template watermark. In: Proceedings of International Symposium on Computer Network and Multimedia Technology (CNMT 2009), 2009, 1–4

8 Kim H S, Lee H K. Invariant image watermark using zernike moments. IEEE Transactions on Circuits and Systems for Video Technology, 2003, 13(8): 766–775

9 Mishra A, Jain A, Narwaria M, Agarwal C. An experimental study into objective quality assessment of watermarked images. International Journal of Image Processing, 2011, 5(2): 199–219

10 Wang Z, Simoncelli E P. Translation insensitive image similarity in complex wavelet domain. In: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’05), 2005, 573–576

11 Sampat M P, Wang Z, Gupta S, Bovik A C, Markey M K. Complex wavelet structural similarity: a new image similarity index. IEEE Transactions on Image Processing, 2009, 18(11): 2385–2401

PMID

12 Setyawan I, Delannay D, Macq B M M, Lagendijk R L. Perceptual quality evaluation of geometrically distorted images using relevant geometric transformation modeling. In: Proceedings of SPIE Security and Watermarking of Multimedia Contents, 2003, 5020: 85–94

13 D’Angelo A, Zhaoping L, Barni M. A full-reference quality metric for geometrically distorted images. IEEE Transactions on Image Processing, 2010, 19(4): 867–881

PMID

14 D’Angelo A, Menegaz G, Barni M. Perceptual quality evaluation of geometric distortions in images. In: Proceedings of SPIE Human Vision and Electronic Imaging, 2007, 6492: 64920J-12

15 Periaswamy S, Farid H. Medical image registration with partial data. Medical Image Analysis, 2006, 10(3): 452–464

PMID

16 Ebling J, Scheuermann G. Clifford Fourier transform on vector fields. IEEE Transactions on Visualization and Computer Graphics, 2005, 11(4): 469–479

PMID

17 Ji J, Chen G, Sun L. A novel Hough transform method for line detection by enhancing accumulator array. Pattern Recognition Letters, 2011, 32(11): 1503–1510

18 Terrades O R, Valveny E. A new use of the Ridgelets transform for describing linear singularities in images. Pattern Recognition Letters, 2006, 27(6): 587–596

19 Sheikh H R, Wang Z, Cormackl L, Bovik A C. Live image quality assessment databases release2. http://live.ece.utexas.edu/research/quality/

20 International Telecommunication Union. Subjective Video Quality Assesment Methods for Multimedia Applications. Recommendation P.910. Geneva, Switzerland, 1996