A dynamic core evolutionary clustering algorithm based on saturated memory

Abstract

Because the number of clustering cores needs to be set before implementing the K-means algorithm, this type of algorithm often fails in applications with increasing data and changing distribution characteristics. This paper proposes an evolutionary algorithm DCC, which can dynamically adjust the number of clustering cores with data change. DCC algorithm uses the Gaussian function as the activation function of each core. Each clustering core can adjust its center vector and coverage based on the response to the input data and its memory state to better fit the sample clusters in the space. The DCC algorithm model can evolve from 0. After each new sample is added, the winning dynamic core can be adjusted or split by competitive learning, so that the number of clustering cores of the algorithm always maintains a better adaptation relationship with the existing data. Furthermore, because its clustering core can split, it can subdivide the densely distributed data clusters. Finally, detailed experimental results show that the evolutionary clustering algorithm DCC based on the dynamic core method has excellent clustering performance and strong robustness.

Keywords

Evolutionary clustering / Dynamic core / Memory saturation / Incremental clustering

References

[1] SaxenaA., PrasadM., GuptaA., BharillN., PatelO.P., TiwariA., ErM.J., DingW., LinC.T.. A review of clustering techniques and developments. Neurocomputing, 2017, 267: 664-681

[2] LiF., QiaoH., ZhangB.. Discriminatively boosted image clustering with fully convolutional auto-encoders. Pattern Recognit., 2017, 83: 161-173

[3] BagirovA.M., UgonJ., WebbD.. Fast modified global k-means algorithm for incremental cluster construction. Pattern Recognit., 2011, 44(4):866-876

[4] YiX., ZhangY.. Equally contributory privacy-preserving k-means clustering over vertically partitioned data. Inf. Sci., 2013, 38(1):97-107

[5] FräntiP., SieranojaS.. K-means properties on six clustering benchmark datasets. Appl. Intell., 2018, 48(12):4743-4759

[6] TsapanosN., TefasA., NikolaidisN., PitasI.. A distributed framework for trimmed kernel k-means clustering. Pattern Recognit., 2015, 48(8):2685-2698

[7] TzortzisG., LikasA.. The minmax k-means clustering algorithm. Pattern Recognit., 2014, 47(7):2505-2516

[8] LinK.-P.. A novel wvolutionary kernel intuitionistic fuzzy c-means clustering algorithm. IEEE Trans. Fuzzy Syst., 2014, 22(5):1074-1087

[9] CelebiM.E., KingraviH.A., VelaP.A.. A comparative study of efficient initialization methods for the k-means clustering algorithm. Expert Syst. Appl., 2013, 40(1):200-210

[10] WuJ., LiuH., XiongH., CaoJ., ChenJ.. K-means based consensus clustering: a unified view. IEEE Trans. Knowl. Data Eng., 2015, 27(1):155-169

[11] SahaJ., MukherjeeJ.. Cnak: cluster number assisted k-means. Pattern Recognit., 2021, 110

[12] ZhangY., TangwongsanK., TirthapuraS.. Fast streaming k-means clustering with coreset caching. IEEE Trans. Knowl. Data Eng., 2022, 34: 2740-2754

[13] BortolotiF.D., de OliveiraE., CiarelliP.M.. Supervised kernel density estimation k-means. Expert Syst. Appl., 2021, 168

[14] MehmoodR., ZhangG., BieR., DawoodH., AhmadH.. Clustering by fast search and find of density peaks via heat diffusion. Neurocomputing, 2016, 208: 210-217

[15] RodriguezA., LaioA.. Clustering by fast search and find of density peaks. Science, 2014, 344(6191):1492-1496

[16] XuX., DingS., ShiZ.. An improved density peaks clustering algorithm with fast finding cluster centers. Knowl.-Based Syst., 2018, 158: 65-74

[17] LiZ., TangY.. Comparative density peaks clustering. Expert Syst. Appl., 2018, 95: 236-247

[18] YangM.-S., LaiC.-Y., LinC.-Y.. A robust EM clustering algorithm for Gaussian mixture models. Pattern Recognit., 2012, 45(11):3950-3961

[19] EsterM., KriegelH.-P., SanderJ., XuX.. A density-based algorithm for discovering clusters in large spatial databases with noise. Proceedings of the Second International Conference on Knowledge Discovery and Data Mining. KDD’96, 1996 Menlo Park AAAI Press 226-231

[20] SchubertE., SanderJ., EsterM., KriegelH.P., XuX.. Dbscan revisited, revisited: why and how you should (still) use dbscan. ACM Trans. Database Syst., 2017, 42(3

[21] Mahesh KumarK., Rama Mohan ReddyA.. A fast dbscan clustering algorithm by accelerating neighbor searching using groups method. Pattern Recognit., 2016, 58: 39-48

[22] LuchiD., Loureiros RodriguesA., Miguel VarejãoF.. Sampling approaches for applying dbscan to large datasets. Pattern Recognit. Lett., 2019, 117: 90-96

[23] KohonenT.. The self-organizing map. Neurocomputing, 1998, 21 1):1-6

[24] KobrenA., MonathN., KrishnamurthyA., McCallumA.. A hierarchical algorithm for extreme clustering. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD’17, 2017 New York Association for Computing Machinery 255-264

[25] NguyenT.T., DangM.T., LuongA.V., LiewA.W.-C., LiangT., McCallJ.. Multi-label classification via incremental clustering on an evolving data stream. Pattern Recognit., 2019, 95: 96-113

[26] ShafeeqA.. Dynamic clustering of data with modified k-means algorithm. International Conference on Information and Computer Networks, 2012

[27] LughoferE.. A dynamic split-and-merge approach for evolving cluster models. Evolv. Syst., 2012, 3: 135-151

[28] BlackM.M., HickeyR.J.. The use of time stamps in handling latency and concept drift in online learning. Evolv. Syst., 2012, 3: 203-220

[29] ZhengL.. Improved K-means clustering algorithm based on dynamic clustering. Int. J. Adv. Res. Big Data Manag. Syst., 2019, 4: 17-26

[30] LiH.-J., BuZ., WangZ., CaoJ.. Dynamical clustering in electronic commerce systems via optimization and leadership expansion. IEEE Trans. Ind. Inform., 2020, 16(8):5327-5334

[31] BernsteinF., ModaresiS., SauréD.. A dynamic clustering approach to data-driven assortment personalization. Manag. Sci., 2019, 65(5):2095-2115

[32] KhanI., LuoZ., HuangJ.Z., ShahzadW.. Variable weighting in fuzzy k-means clustering to determine the number of clusters. IEEE Trans. Knowl. Data Eng., 2020, 32(9):1838-1853

[33] GuoP., ChenC.L.P., LyuM.R.. Cluster number selection for a small set of samples using the Bayesian Ying-Yang model. IEEE Trans. Neural Netw., 2002, 13(3):757-763

[34] YaoY., LiY., JiangB., ChenH.. Multiple kernel k-means clustering by selecting representative kernels. IEEE Trans. Neural Netw. Learn. Syst., 2021, 32: 4983-4996

[35] WangX.-F., HuangD.-S.. A novel density-based clustering framework by using level set method. IEEE Trans. Knowl. Data Eng., 2009, 21(11):1515-1531

[36] HuangD., WangC.-D., LaiJ.-H.. Locally weighted ensemble clustering. IEEE Trans. Cybern., 2018, 48(5):1460-1473

[37] MinE., GuoX., LiuQ., ZhangG., CuiJ., LongJ.. A survey of clustering with deep learning: from the perspective of network architecture. IEEE Access, 2018, 6: 39501-39514

[38] YangL., FanW., BouguilaN.. Clustering analysis via deep generative models with mixture models. IEEE Trans. Neural Netw. Learn. Syst., 2022, 33: 340-350

[39] MonathN., KobrenA., KrishnamurthyA., GlassM.R., McCallumA.. Scalable hierarchical clustering with tree grafting. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD’19, 2019 New York Association for Computing Machinery 1438-1448

[40] XieH., LiP.. A density-based evolutionary clustering algorithm for intelligent development. Eng. Appl. Artif. Intell., 2021, 104

[41] YuZ., LuoP., YouJ., WongH.-S., LeungH., WuS., ZhangJ., HanG.. Incremental semi-supervised clustering ensemble for high dimensional data clustering. IEEE Trans. Knowl. Data Eng., 2016, 28(3):701-714

[42] YuH., LuJ., ZhangG.. Online topology learning by a Gaussian membership-based self-organizing incremental neural network. IEEE Trans. Neural Netw. Learn. Syst., 2020, 31(10):3947-3961

[43] BergD.A., SuY., Jimenez-CyrusD., PatelA., HuangN., MorizetD., LeeS., ShahR., RingelingF.R., JainR., EpsteinJ.A., WuQ.-F., CanzarS., MingG.-L., SongH., BondA.M.. A common embryonic origin of stem cells drives developmental and adult neurogenesis. Cell, 2019, 177(3):654-66815

[44] FahadA., AlshatriN., TariZ., AlamriA., KhalilI., ZomayaA.Y., FoufouS., BourasA.. A survey of clustering algorithms for big data: taxonomy and empirical analysis. IEEE Trans. Emerg. Top. Comput., 2014, 2(3):267-279

[45] ŁukasikS., KowalskiP.A., CharytanowiczM., KulczyckiP.. Clustering using flower pollination algorithm and Calinski-Harabasz index. 2016 IEEE Congress on Evolutionary Computation (CEC), 2016 2724-2728

[46] StrehlA., GhoshJ.. Cluster esembles—a knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res., 2003, 3: 583-617

[47] ChakrabortyS., NagwaniN.K.. Analysis and study of incremental k-means clustering algorithm. Commun. Comput. Inf. Sci., 2011, 169: 338-341

[48] DeyL., ChakrabortyS., NagwaniN.K.. Performance comparison of incremental k-means and incremental DBSCAN algorithms. Comput. Sci., 2013, 27(11):14-18 http://doi.org/10.5120/3346-4611

[49] FritzkeB.. A growing neural gas network learns topologies. Proceedings of the 7th International Conference on Neural Information Processing Systems, 1994 625-632

[50] MarslandS., ShapiroJ., NehmzowU.. A self-organising network that grows when required. Neural Netw., 2002, 15: 1041-1058

[51] FuraoS., HasegawaO.. An incremental network for on-line unsupervised classification and topology learning. Neural Netw., 2016, 19(1):90-106