Social or Individual Learning? An Aggregated Solution for Coordination in Multiagent Systems

Abstract

There are mainly two different ways of learning for animals and humans: trying on yourself through interactions or imitating/copying others through communication/observation. How these two learning strategies differ and what roles they are playing in achieving coordination among individuals are two challenging problems for researchers from various disciplines. In multiagent systems, most existing work simply focuses on individual learning for achieving coordination among agents. The social learning perspective has been largely neglected. Against this background, this article contributes by proposing an integrated solution to decision making between social learning and individual learning in multiagent systems. Two integration modes have been proposed that enable agents to choose in between these two learning strategies, either in a fixed or in an adaptive manner. Experimental evaluations have shown that these two kinds of leaning strategies have different roles in maintaining efficient coordination among agents. These differences can reveal some significant insights into the manipulation and control of agent behaviors in multiagent systems, and also shed light on understanding the social factors in shaping coordinated behaviors in humans and animals.

Keywords

Individual learning / social learning / coordination / multiagent systems

References

[1] Acemoglu D., Ozdaglar A.. Opinion dynamics and learning in social networks. Dynamic Games and Applications, 2011, 1(1): 3-49.

[2] Barabási A., Albert R.. Statistical mechanics of complex networks. Rev. Modern Phys., 2002, 74: 47-97.

[3] Castellano C., Fortunato S., Loreto V.. Statistical physics of social dynamics. Reviews of modern physics, 2009, 81(2): 591.

[4] Hegselmann R., Krause U., et al. Opinion dynamics and bounded confidence models, analysis, and simulation. Journal of artificial societies and social simulation, 2002, 5(3): 1-33.

[5] Hethcote H. W.. The mathematics of infectious diseases. SIAM review, 2000, 42(4): 599-653.

[6] Hofmann L., Chakraborty N., Sycara K.. The evolution of cooperation in self-interested agent societies: a critical study. In Proc. of 10th AAMAS, 2011 685-692.

[7] Laland K. N.. Social learning strategies. Learning & behavior, 2004, 32(1): 4-14.

[8] Mihaylov M., Tuyls K., Nowé A.. A decentralized approach for convention emergence in multi-agent systems. Autonomous Agents and Multi-Agent Systems, 2014, 28(5): 749-778.

[9] Mukherjee P., Sen S., Airiau S.. Norm emergence under constrained interactions in diverse societies. In Proc. of 7th AAMAS, 2008 779-786.

[10] Rendell L., Fogarty L., Hoppitt W. J., Morgan T. J., Webster M. M., Laland K. N.. Cognitive culture: theoretical and empirical insights into social learning strategies. Trends in cognitive sciences, 2011, 15(2): 68-76.

[11] Savarimuthu B., Arulanandam R., Purvis M.. Aspects of active norm learning and the effect of lying on norm emergence in agent societies. Agents in Principle, Agents in Practice, LNAI, 2011, 7047: 36-50.

[12] Sen O., Sen S.. Effects of social network topology and options on norm emergence. Coordination, Organizations, Institutions and Norms in Agent Systems V, LNAI, 2010, 6069: 211-222.

[13] Sen S., Airiau S.. Emergence of norms through social learning. In Proc. of 20th IJCAI, 2007 1507-1512.

[14] Shibusawa R., Sugawara T.. Norm emergence via influential weight propagation in complex networks. In Network Intelligence Conference (ENIC), 2014 European. IEEE, 2014 30-37.

[15] Shoham Y., Tennenholtz M.. On the emergence of social conventions: modeling, analysis, and simulations. Artif. Intel., 1997, 94(1-2): 139-166.

[16] Szabo G., Fáth G.. Evolutionary games on graphs. Phys. Rep., 2007, 446(4-6): 97-216.

[17] Verhagen H.. Simulation of the learning of norms. Social Science Computer Review, 2001, 19(3): 296-306.

[18] Villatoro D., Sabater-Mir J., Sen S.. Social instruments for robust convention emergence. In Proc. of 22nd IJCAI, 2011 420-425.

[19] Villatoro D., Sen S., Sabater-Mir J.. Topology and memory effect on convention emergence. In Proc. of WIIAT’09, 2009 233-240.

[20] Watts D., Strogatz S.. Collective dynamics of smallworld networks. Nature, 1998, 393: 440-442.

[21] Watts D. J., Dodds P. S.. Influentials, networks, and public opinion formation. Journal of consumer research, 2007, 34(4): 441-458.

[22] Young H.. The economics of convention. J. Econ. Pers., 1996, 10(2): 105-122.

[23] Yu C., Lv H., Sen S., Hao J., Ren F., Liu R.. An adaptive learning framework for efficient emergence of social norms: (extended abstract). In International Conference on Autonomous Agents Multiagent Systems, 2016 1307-1308.

[24] Yu C., Liu R., Wang Y.. Social learning in networked agent societies. In^IEEE International Conference on Agents, 2016 57-62.

[25] Yu C., Lv H., Ren F., Bao H., Hao J.. Hierarchical learning for emergence of social norms in networked multiagent systems. In Australasian Joint Conference on Artificial Intelligence, 2015 630-643.

[26] Yu C., Lv H., Sen S., Ren F., Tan G.. Adaptive learning for efficient emergence of social norms in networked multiagent systems. In Pacific Rim International Conference on Trends in Artificial Intelligence, 2016 805-818.

[27] Yu C., Tan G., Lv H., Wang Z., Meng J., Hao J., Ren F.. Modelling adaptive learning behaviours for consensus formation in human societies. Scientific Reports, 2016, 6: 27626.

[28] Yu C., Wang Z., Lv H., Bao H., Li Y.. Collective Learning and Information Diffusion for Efficient Emergence of Social Norms, 2017, Singapore: Springer.

[29] Yu C. Y. C., Lv H., Ren J., Ge H., Sun L.. Neural learning for the emergence of social norms in multiagent systems. In^IEEE International Conference on Agents, 2017 40-45.

[30] Yu C., Zhang M., Ren F.. Collective learning for the emergence of social norms in networked multiagent systems. IEEE Transactions on Cybernetics, 2014, 44(12): 2342-2355.

[31] Yu C., Zhang M., Ren F., Luo X.. Emergence of social norms through collective learning in networked agent societies. In Proc. of AAMAS2013, 2013 475-482.

[32] Yu C., Zhang M., Ren F.. Emotional multiagent reinforcement learning in social dilemmas. In International Conference on Principles and Practice of Multi-Agent Systems, 2013 372-387.

[33] Yu C., Zhang M., Ren F., Tan G.. Emotional multiagent reinforcement learning in spatial social dilemmas. IEEE Transactions on Neural Networks & Learning Systems, 2015, 26(12): 3083-3096.