A new formant feature and its application in Mandarin vowel pronunciation quality assessment

Abstract

In order to improve the Mandarin vowel pronunciation quality assessment, a novel formant feature was proposed and applied to formant classification for Chinese Mandarin vowel pronunciation quality evaluation. Formant candidates of each frame were plotted on the time-frequency plane to form a bitmap, and its Gabor feature was extracted to represent the formant trajectory. The feature was then classified by using GMM model and the classification posterior probability was mapped to pronunciation quality grade. The experiments of comparing the Gabor transformation based formant trajectory feature with several other kinds of traditionally used features show that with this method, a human-machine scoring correlation coefficient (CC) of 0.842 can be achieved, which is better than the result of 0.832 by traditional speech recognition techniques. At the same time, considering that the long-term information of formant classification and the short-term information of speech recognition technique are complementary to each other, it is investigated to combine their results with linear or nonlinear methods to further improve the evaluation performance. As a result, experiments on PSK show that the best CC of 0.913, which is very close to the correlation of inter-human rating of 0.94, is gotten by using neural network.

Keywords

computer assisted language learning / speech recognition / Gaussian mixture model / formant / Gabor feature / neural network

References

[1] FrancoH, NeumeyerL. Automatic pronunciation scoring for language instruction [C]. Proceedings of International Conference of Acoust, Speech and Signal Processing, 1997MunichIEEE1471-1474

[2] NeumeyerL, FrancoH. Automatic scoring of pronunciation quality [J]. Speech Communication, 2000, 30(2): 83-93

[3] StrikH, TruongK, WetF D, CucchiariniC. Comparing different approaches for automatic pronunciation error detection [J]. Speech Communication, 2009, 51(10): 1-14

[4] TatsuyaK, MasatakeD, YasushiT. Practical use of English pronunciation system for Japanese students in the CALL classroom [C]. Proceedings of 8th International Conference on Spoken Language Processing, Jeju Island, 2004KoreaISCA1689-1692

[5] WittS M, YoungS J. Phone-level pronunciation scoring and assessment for interactive language learning [J]. Speech Communication, 2000, 30(2/3): 95-108

[6] ChenJ-c, JangJ-s R, TsaiT-lu. Automatic pronunication assessment for mandarin Chinese: Approaches and system overview [J]. Computational Linguistics and Chinese Language Processing, 2007, 12(4): 443-458

[7] TruongK, NeriA, CucchiariniC, StrikH. Automatic pronunciation error detection: an acoustic-phonetic approach [C]. Proceedings of the InSTIL/ICALL Symposium on NLP and Speech Technologies in Advanced Language Learning Systems, 2004Venice, ItalySpringger-Verlag135-138

[8] TruongKAutomatic pronunciation error detection in Dutch as a second language: an acoustic-phonetic approach [D], 2004The NetherlandsUtrecht University

[9] XieS, KeelanE. Gaussian mixture modeling of vowel durations for automated assessment of non-native speech [C]. Proceedings of International Conference of Acoust, Speech and Signal Processing, 2011PragueIEEE5716-5719

[10] YusofS A M, PaulrajM, YaacobS. Classification of malaysian vowels using formant based feature [J]. Journal of ICT, 2009, 7(2): 27-40

[11] ZhandosY, MuslimaK, AltynbekS. Formant analysis and mathematical model of kazakh vowels [C]. Proceedings of International Conference on Modeling and Simulation, 2012UKSimIEEE427-431

[12] SchmidP, BanardE. Explicit, n-best formant features for vowel classification [C]. Proceedings of International Conference of Acoust, Speech and Signal Processing, 1997MunichIEEE21-24

[13] LeeM, VansantenJ, MobiusB, OliveJ. Formant tracking using context-dependent phonemic information [J]. IEEE Transactions on Speech and Audio Processing, 2005, 13(5): 741-750

[14] GrigorescuS E, PetkovN, KruizingaP. Comparison of texture features based on Gabor filters [J]. IEEE Transactions on Image Processing, 2002, 11(10): 1160-1167

[15] PetkovN. Biologically motivated computationally intensive approaches to image pattern recognition [J]. Future Generation Computer Systems, 1995, 11(4): 451-465

[16] JonesJ P, PalmerL A. An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex [J]. Journal of Neurophysiology, 1987, 58(6): 1233-1258

[17] MorganN, ZhuQ-feng. Pushing the envelope aside: Beyond the spectral envelope as the fundamental representation for speech recognition [J]. IEEE Signal Processing Magazine, 2005, 22(5): 81-88

[18] XiaoD-r, HouJ-min. Application research of neural network in fault diagnosis [J]. Journal of Central South University of Technology: Natural Science, 2003, 34(1): 206-208

[19] DingD-x, ZhangZ-jun. Artificial neural network based on inverse design method for circular sliding slopes [J]. Journal of Central South University of Technology, 2004, 11(1): 89-92

[20] BishopC MNeural networks for pattern recognition [M], 1995New YorkOxford University Press216-302

[21] ZhouY, PedrycaW, QianXu. Application of extension neural network to safety status pattern recognition of coal mines [J]. Journal of Central South University of Technology, 2011, 18(1): 633-641