Group Lasso based redundancy-controlled feature selection for fuzzy neural network

Abstract

If there are a lot of inputs, the readability of the “If-then” fuzzy rule is reduced, and the complexity of the fuzzy neural network structure will be increased. Hence, to optimize the structure of the fuzzy rule based neural network, a group Lasso based redundancy-controlled feature selection (input pruning) method is proposed. For realizing feature selection, the linear/nonlinear redundancy between features is considered, and the Pearson’s correlation coefficient is employed to construct the additive redundancy-controlled regularizer in the error function. In addition, considering the past gradient information, a novel parameter optimization method is presented. Finally, we demonstrate the effectiveness of our method on two benchmark classification datasets.

References

[1] BUGATA P, DROTAR P. Feature selection based on a sparse neural-network layer with normalizing constraints[J]. IEEE transactions on cybernetics, 2021.

[2] VenkateshB, AnuradhaJ. A review of feature selection and its methods[J]. Cybernetics and information technologies, 2019, 19(1):3-26

[3] WudR, ZengZ G, MoH, et al.. Interval type-2 fuzzy sets and systems: overview and outlook[J]. Acta automatica sinica, 2020, 46(8):1539-1556

[4] PomaresH, RojasI, GonzálezJ, et al.. Structure identification in complete rule-based fuzzy systems[J]. IEEE transactions on fuzzy systems, 2002, 10(3):349-359

[5] LiR P, MukaidonoM, TurksenI B. A fuzzy neural network for pattern classification and feature selection[J]. Fuzzy sets and systems, 2002, 130(1):101-108

[6] ChenY C, PalN R, ChungI F. An integrated mechanism for feature selection and fuzzy rule extraction for classification[J]. IEEE transactions on fuzzy systems, 2011, 20(4):683-698

[7] LinF J, ChenS G, HsuC W. Intelligent backstepping control using recurrent feature selection fuzzy neural network for synchronous reluctance motor position servo drive system[J]. IEEE transactions on fuzzy systems, 2018, 27(3):413-427

[8] XUE G, CHANG Q, WANG J, et al. An adaptive neuro-fuzzy system with integrated feature selection and rule extraction for high-dimensional classification problems[EB/OL]. (2022-01-28) [2022-03-30]. arXiv preprint arXiv: 2201.03187, 2022.

[9] LinC T, PalN R, WuS L, et al.. An interval type-2 neural fuzzy system for online system identification and feature elimination[J]. IEEE transactions on neural networks and learning systems, 2014, 26(7):1442-1455

[10] HouS, ChuY, FeiJ. Adaptive type-2 fuzzy neural network inherited terminal sliding mode control for power quality improvement[J]. IEEE transactions on industrial informatics, 2021, 17(11):7564-7574

[11] GaoT, BaiX, ZhangL, et al.. Feature selection for fuzzy neural networks using group Lasso regularization[C], 2021, New York, IEEE: 1-8

[12] ChungI F, ChenY C, PalN R. Feature selection with controlled redundancy in a fuzzy rule based framework[J]. IEEE transactions on fuzzy systems, 2017, 26(2):734-748

[13] YuL, LiuH. Efficient feature selection via analysis of relevance and redundancy[J]. The journal of machine learning research, 2004, 5: 1205-1224

[14] PengH, LongF, DingC. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy[J]. IEEE transactions on pattern analysis and machine intelligence, 2005, 27(8):1226-1238

[15] BattitiR. Using mutual information for selecting features in supervised neural net learning[J]. IEEE transactions on neural networks, 1994, 5(4):537-550

[16] EstévezP A, TesmerM, PerezC A, et al.. Normalized mutual information feature selection[J]. IEEE transactions on neural networks, 2009, 20(2):189-201

[17] ZhouL, WangL, ShenC. Feature selection with redundancy-constrained class separability[J]. IEEE transactions on neural networks, 2010, 21(5):853-858

[18] AksuY, MillerD J, KesidisG, et al.. Margin-maximizing feature elimination methods for linear and nonlinear kernel-based discriminant functions[J]. IEEE transactions on neural networks, 2010, 21(5):701-717

[19] LeeH M, ChenC M, ChenJ M, et al.. An efficient fuzzy classifier with feature selection based on fuzzy entropy[J]. IEEE transactions on systems, man, and cybernetics, part B (cybernetics), 2001, 31(3):426-432

[20] PalN R, MalpaniM. Redundancy-constrained feature selection with radial basis function networks[C], 2012, New York, IEEE: 1-8

[21] WangJ, ZhangH, WangJ, et al.. Feature selection using a neural network with group Lasso regularization and controlled redundancy[J]. IEEE transactions on neural networks and learning systems, 2021, 32(3):1110-1123

[22] GaoT, BaiX, ZhangL, et al.. Redundancy-controlled feature selection for fuzzy neural networks[C], 2021, New York, IEEE: 172-176

[23] School of Information and Computer Science, University of California. UCI machine learning repository[EB/OL]. [2022-03-30]. http://archive.ics.uci.edu/ml.