RankSNN: A Ranking Method to Integrated Evaluation

Abstract

Evaluations are frequent, significant cognitive activities in human society. People make comprehensive judgments and rank the evaluated objects based on multiple factors, including relevant information and alternative data. Existing integrated evaluation methods assign weights based on expert opinions or data dispersion and provide evaluated rankings of objects through simple mapping. These methods are easy to implement, but the models produce biased results when the differences of all features in the evaluated objects are not significant enough. In order to reduce this error, we borrow the ideas from ranking methods and train the model based on ordering relations. We re-formulate the evaluation problem, construct a ranking model, and then design a new method named Ranking by Siamese Neural Network (RankSNN) adaptive to the model. This method uses ordering relations of sample pairs as labels to train the model, and measure the differences between samples by Siamese structure to get rankings. In this way, a trained scoring function can accurately correspond to the ordering relations among the sample pairs by calculating evaluation values and obtaining the best evaluation ranking results. In order to verify the validity and rationality of RankSNN, we conducted experiments on the public dataset of learning to rank and compared our results with the baseline ranking method. Then, we use RankSNN in integrated evaluation problems related to China’s urban business environment for the purpose of conducting empirical analyses, and the results of our method are superior to existing methods.

Keywords

Integrated evaluation / Sample pair / Ordering relations / Scoring function / Siamese Neural Network / 62P25 / 97M10

References

[1] Ao X, He AResearch on the comprehensive evaluation method of college students’ achievements based on multi-view RSR. J. Phys. Conf. Ser. IOP Publishing, 2020, 1616(1012072

[2] Bromley J, Guyon I, LeCun Y, et al.Signature verification using a" siamese" time delay neural network. Adv. Neural. Inf. Process. Syst., 1993, 6: 737-744

[3] Burges, C., Shaked, T., Renshaw, E., et al.: Learning to rank using gradient descent. In: Proceedings of the 22nd International Conference on Machine Learning. pp. 89-96 (2005)

[4] Chopra, S., l, Hadsell, R., LeCun, Y.: Learning a similarity metric discriminatively, with application to face verification. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). vol. 1, pp. 539-546 (2005)

[5] Doing Business.: Doing business 2020. The World Bank. (2019). Accessed 24 October 2020 https://www.doingbusiness.org/en/reports/global-reports/doing-business-2020

[6] Du D, Pang Q, Wu YModern integrated evaluation methods and case selection, 20153BeijingTsinghua University Press113

[7] Järvelin, K., Kekäläinen, J.: IR evaluation methods for retrieving highly relevant documents. ACM SIGIR Forum. New York, NY, USA: ACM, 51(2): 243-250(2017)

[8] Joachims, T.: Optimizing search engines using clickthrough data. In: Proceedings of the eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. pp. 133-142 (2002)

[9] Kumari B, Pandey ACGeo-informatics based multi-criteria decision analysis (MCDA) through analytic hierarchy process (AHP) for forest fire risk mapping in Palamau Tiger Reserve, Jharkhand state. India. J. Earth Syst. Sci., 2020, 1291): 1-16

[10] Li Y, Qiao L, Wang Q, Karácsonyi DTowards the evaluation of rural livability in China: theoretical framework and empirical case study. Habitat Int., 2020, 105102241

[11] Liang L, Yang F, Cook WD, et al.DEA models for supply chain efficiency evaluation. Ann. Oper. Res., 2006, 145(1): 35-49.

[12] Liu TLearning to Rank for Information Retrieval, 2011, Berlin: Springer-Verlag, 3-25.

[13] Qin, T., Liu, T.: Introducing LETOR 4.0 datasets. arXiv preprint arXiv:1306.2597 (2013)

[14] Ran L, Tan X, Xu Y, et al.The application of subjective and objective method in the evaluation of healthy cities: a case study in Central China. Sustain. Cities Soc., 2021, 65102581

[15] Samuel OW, Asogbon GM, Sangaiah AK, et al.An integrated decision support system based on ANN and Fuzzy\_AHP for heart failure risk prediction. Expert Syst. Appl., 2017, 68: 163-172.

[16] Shan, Y., Hoens, T. R., Jiao, J., et al.: Deep crossing: web-scale modeling without manually crafted combinatorial features. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. pp. 255-262 (2016)

[17] Shih H, Shyur H, Lee EAn extension of TOPSIS for group decision making. Math. Comput. Modell., 2007, 45(7): 801-813.

[18] Wang XFood supply chain safety risk evaluation based on AHP fuzzy integrated evaluation method. Int. J. Secur. Appl., 2016, 10: 233-244

[19] Wei Y, Huang C, Li J, Xie LAn evaluation model for urban carrying capacity: a case study of China’s mega-cities. Habitat Int., 2016, 53: 87-96.

[20] Zhang X, Wang C, Li E, Xu CAssessment model of ecoenvironmental vulnerability based on improved entropy weight method. Sci. World J., 2014

Funding

The Major Project of Science & Technology of Anhui Province(Nos. 202203a05020050)

Anhui Center for Applied Mathematics

The Strategic Priority Research Program of Chinese Academy of Sciences(No. XDC08010100)

The NSF of China(No. 92270205)

Rights & permissions

School of Mathematical Sciences, University of Science and Technology of China and Springer-Verlag GmbH Germany, part of Springer Nature