Decoding micro-social interactions in public space: a computer-vision-based method

Abstract

As cities strive for greater liveability, data-driven methods can enhance our understanding of public space behaviours and social interactions. Recent developments in computer vision technologies have significantly advanced the accuracy of micro-scale human behaviour detection, but there is a lack of methodologies that capture relational, nuanced behaviours within specific spatial and temporal environments. This paper presents the development of a computer vision and machine learning-based methodology to analyse co-presence and micro-social interactions in urban spaces, introducing new metrics for spatial behavioural analysis. The methodology was tested on a 22.5-min video dataset obtained at a university campus, demonstrating its capacity for trajectory analysis and detecting nuanced interpersonal behaviours including encountering, congregating, approaching and avoiding. Human observers validated the computer-generated behaviour classifications, achieving high agreement levels and demonstrating the system's accuracy in detecting diverse pedestrian interactions. The approach successfully offers fine-grained analysis of social behaviours and spatial patterns of co-presence, revealing how urban morphology influences social interaction hotspots. It advances environment-behaviour research by providing scalable, automated tools for detailed, data-driven analysis of public space vitality, with potential applications in urban design, social sciences, and policy-making.

Keywords

Public Space Studies / Pedestrian Location Tracking / Computer Vision Object Detection / Micro-Social Behaviour Analysis / Machine Learning

References

[1] Akpınar, A. (2021). How perceived sensory dimensions of urban green spaces are associated with teenagers’ perceived restoration, stress, and mental health?. Landscape and Urban Planning, 214, 104185. https://doi.org/10.1016/j.landurbplan.2021.104185

[2] Aggarwal JK, Ryoo MS. Human activity analysis: A review. Acm Computing Surveys, 2011, 43(3): 1-43.

[3] Aharon, N., Orfaig, R., & Bobrovsky, B. Z. (2022). BoT-SORT: Robust associations multi-pedestrian tracking. arXiv preprint arXiv:2206.14651. http://arxiv.org/abs/2206.14651

[4] Amin A. Collective culture and urban public space. City, 2008, 12(1): 5-24.

[5] Amin A, Thrift NSeeing like a city, 2017John Wiley & Sons

[6] Anik MAH, Hossain M, Habib MA. Investigation of pedestrian jaywalking behaviour at mid-block locations using artificial neural networks. Safety Science, 2021, 144. 105448

[7] Appleton JThe experience of landscape, 1975Wiley

[8] Badidi E, Moumane K, El Ghazi F. Opportunities, applications, and challenges of edge-AI enabled video analytics in smart cities: A systematic review. IEEE Access, 2023.

[9] Balsa-Barreiro J, Menendez M, Morales AJ. Scale, context, and heterogeneity: The complexity of the social space. Scientific Reports, 2022, 12. 9037

[10] Bibri, S. E. (2020). Advances in the leading paradigms of urbanism and their amalgamation: compact cities, eco–cities, and data–driven smart cities. Springer Nature.

[11] Bobick AF, Davis JW. The recognition of human movement using temporal templates. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(3): 257-267.

[12] Bornioli A, Subiza-Pérez M. Restorative urban environments for healthy cities: A theoretical model for the study of restorative experiences in urban built settings. Landscape Research, 2023, 48(1): 152-163.

[13] Bratman GN, Daily GC, Levy BJ, Gross JJ. The benefits of nature experience: Improved affect and cognition. Landscape and Urban Planning, 2015, 138: 41-50.

[14] Breheny M. Urban compaction: Feasible and acceptable?. Cities, 1997, 14(4): 209-217.

[15] Brunetti A, Buongiorno D, Trotta GF, Bevilacqua V. Computer vision and deep learning techniques for pedestrian detection and tracking: A survey. Neurocomputing, 2018, 300: 17-33.

[16] Can I, Heath T. In-between spaces and social interaction: A morphological analysis of Izmir using space syntax. Journal of Housing and the Built Environment, 2016, 31(1): 31-49.

[17] Capillé, C. C. (2016). Spatial cultures of public libraries: architecture, collective use and political agendas in Medellín’s Library-Parks (Doctoral dissertation, UCL (University College London)).

[18] Carmona, M. (2021). Public places urban spaces : the dimensions of urban design (Third edition.). New York, New York; London: Routledge. https://doi.org/10.4324/9781315158457

[19] Cattell V, Dines N, Gesler W, Curtis S. Mingling, observing, and lingering: Everyday public spaces and their implications for well-being and social relations. Health & Place, 2008, 14(3): 544-561.

[20] Choubey N, Verma A, Chakraborty A. An unsupervised group detection method for understanding group dynamics in crowds. Physica a: Statistical Mechanics and Its Applications, 2024, 655. 130195

[21] Collins RInteraction Ritual Chains, 2014Princeton University Press

[22] Cresswell TPlace: An introduction, 2014John Wiley & Sons

[23] Ewing R, Handy S. Measuring the unmeasurable: Urban design qualities related to walkability. Journal of Urban Design, 2009, 14(1): 65-84.

[24] Fan Z, Loo BP. Street life and pedestrian activities in smart cities: Opportunities and challenges for computational urban science. Computational Urban Science, 2021, 1(1. 26

[25] Fisher KD. Placing social interaction: An integrative approach to analysing past built environments. Journal of Anthropological Archaeology, 2009, 28(4): 439-457.

[26] Friedmann J. Place and place-making in cities: A global perspective. Planning Theory & Practice, 2010, 11(2): 149-165.

[27] Gawande U, Hajari K, Golhar Y. Real-time deep learning approach for pedestrian detection and suspicious activity recognition. Procedia Computer Science, 2023, 218: 2438-2447.

[28] Gehl JLife between buildings: Using public space, 1971Danish Architectural Press

[29] Gehl, J. (2010). Cities for people. Washington, DC: Island press.

[30] Gehl J, Svarre BHow to study public life, 2013Island press.

[31] Ghermandi A, Depietri Y, Sinclair M. In the AI of the beholder: A comparative analysis of computer vision-assisted characterizations of human-nature interactions in urban green spaces. Landscape and Urban Planning, 2022, 217. 104261

[32] Giddens, A. (1984). The constitution of society: Outline of the theory of structuration. Univ. of California Press.

[33] Goffman EBehaviour in public places, 1963Free Press

[34] Gou Z, Xie X, Lu Y, Khoshbakht M. Quality of life (QoL) survey in Hong Kong: Understanding the importance of housing environment and needs of residents from different housing sectors. International Journal of Environmental Research and Public Health, 2018, 15(2. 219

[35] Grannis R. The importance of trivial streets: Residential streets and residential segregation. American Journal of Sociology, 1998, 103(6): 1530-1564.

[36] Hall ETThe hidden dimension, 1966Doubleday

[37] Haritaoglu I, Harwood D, Davis LS. W4: Real-time surveillance of people and their activities. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(8): 809-830.

[38] He, K., Li, H., Zhang, H., Hu, Q., Yu, Y., & Qiu, W. (2025). Revisiting Gehl’s urban design principles with computer vision and webcam data: Associations between public space and public life. Environment and Planning B: Urban Analytics and City Science, 23998083251328771.

[39] Heffernan E, Heffernan T, Pan W. The relationship between the quality of active frontages and public perceptions of public spaces. Urban Design International, 2014, 19: 92-102.

[40] Hidalgo MC, Hernandez B. Place attachment: Conceptual and empirical questions. JOurnal of Environmental Psychology, 2001, 21(3): 273-281.

[41] Hillier BSpace is the machine: A configurational theory of architecture, 1998Cambridge University Press

[42] Hillier B, Vaughan L. The city as one thing. Progress in Planning, 2007, 67(3): 205-230

[43] Hillier B, Penn A, Hanson J, Grajewski T, Xu J. Natural movement: Or, configuration and attraction in urban pedestrian movement. Environment and Planning b: Planning & Design, 1993, 20(1): 29-66.

[44] Hosain MT, Zaman A, Abir MR, Akter S, Mursalin S, Khan SS. Synchronizing object detection: Applications, advancements and existing challenges. IEEE Access, 2024, 12: 54129-54167.

[45] Hunter RF, Garcia L, de Sa TH, Zapata-Diomedi B, Millett C, Woodcock J, Moro E. Effect of COVID-19 response policies on walking behavior in US cities. Nature Communications, 2021, 12(1. 3652

[46] Istrate AL, Chen F, Kadetz P, Chang Y, Williams AR. Developing an analytical framework for liveable streets in Shanghai. Urban Design International, 2021, 26: 3-20.

[47] Jacobs, A. B. (1993). Great streets. University of California Transportation Center.

[48] Jacobs JThe death and life of great American cities, 1961Vintage Books

[49] Jhangiani, R. S., Chiang, I. C. A., Cuttler, C., & Leighton, D. C. (2019). Research methods in psychology. Kwantlen Polytechnic University.

[50] Kaplan R, Kaplan SCognition and environment: Functioning in an uncertain world, 1982Praeger

[51] Kendon, A. (1990). Conducting interaction: Patterns of behaviour in focused encounters. Cambridge; New York: Cambridge University Press.

[52] Lai, C. (2018). Open Space Opinion Survey. Jockey Club Civic Exchange “Reconnecting open space” Programme.

[53] Ley D, Samuels MHumanistic geography: Prospects and problems, 1978C.Helm

[54] Liao B, Li X. Neighborhood environment and affective walking experience: Cluster analysis results of a virtual-environment-based conjoint experiment. International Journal of Environmental Research and Public Health, 2023.

[55] Lindheim R, Syme SL. Environments, people, and health. Annual Review of Public Health, 1983, 4(1): 335-359.

[56] Loo BP, Fan Z. Social interaction in public space: Spatial edges, moveable furniture, and visual landmarks. Environment and Planning b: Urban Analytics and City Science, 2023, 50(9): 2510-2526

[57] Loo BP, Zhang F. Design of public open space: Site features, playing, and physical activity. Health & Place, 2024, 85. 103149

[58] Low SOn the plaza: The politics of public space and culture, 2000University of Texas Press

[59] Low SSpatializing culture: The ethnography of space and place, 2016Routledge.

[60] Maciel FB, Zampieri FL. Co-presence patterns in dispersed residential neighbourhoods of Brazilian medium-sized cities. Environment and Planning b: Urban Analytics and City Science, 2021, 48(7): 1943-1954

[61] Markevych I, Schoierer J, Hartig T, Chudnovsky A, Hystad P, Dzhambov AM, de Vries S, Triguero-Mas M, Brauer M, Nieuwenhuijsen MJ, Lupp G, Richardson EA, Astell-Burt T, Dimitrova D, Feng X, Sadeh M, Standl M, Heinrich J, Fuertes E. Exploring pathways linking greenspace to health: Theoretical and methodological guidance. Environmental Research, 2017, 158: 301-317.

[62] Marquardt N, Greenberg S. Proxemic interactions: From theory to practice. Synthesis Lectures on Human-Centered Informatics, 2015, 8(1): 1-199.

[63] Mehta V. Lively streets: Determining environmental characteristics to support social behaviour. Journal of Planning Education and Research, 2007, 27(2): 165-187.

[64] Miller, H. J. (2017). Time geography and space-time prism. International encyclopedia of geography: People, the earth, environment and technology, 1.

[65] Mondada, L. (2009). Emergent focused interactions in public places: A systematic analysis of the multimodal achievement of a common interactional space. Journal of pragmatics, 41(10), 1977-97.

[66] Mortaheb R, Jankowski P. Smart city re-imagined: City planning and geoai in the age of big data. Journal of Urban Management, 2023, 12(1): 4-15.

[67] Mouratidis K. Is compact city livable? The impact of compact versus sprawled neighbourhoods on neighbourhood satisfaction. Urban Studies, 2018, 55(11): 2408-2430.

[68] Mouratidis K, Yiannakou A. What makes cities livable? Determinants of neighborhood satisfaction and neighborhood happiness in different contexts. Land Use Policy, 2022.

[69] Navarrete-Hernandez, P., Vetro, A., & Concha, P. (2021). Building safer public spaces: Exploring gender difference in the perception of safety in public space through urban design interventions. Landscape and Urban Planning, 214, 104180.

[70] Newman PWG. Sustainability and cities: Extending the metabolism model. Landscape and Urban Planning, 1999, 44(4): 219-226.

[71] Ng, M. K., Yeung, T. C., Kwan, M.-P., Tieben, H., Lau, T. Y. T., Zhu, J., & Xu, Y. (2022). Place qualities, sense of place and subjective well-being: A study of two typical urban neighbourhoods in Hong Kong. Cities and Health. Scopus. https://doi.org/10.1080/23748834.2021.1993038

[72] Niu T, Qing L, Han L, Long Y, Hou J, Li L, Tang W, Teng Q. Small public space vitality analysis and evaluation based on human trajectory modeling using video data. Building and Environment, 2022, 225. 109563

[73] Oliver NM, Rosario B, Pentland AP. A Bayesian computer vision system for modeling human interactions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(8): 831-843.

[74] Oppio A, Bottero M, Arcidiacono A. Assessing urban quality: A proposal for a MCDA evaluation framework. Annals of Operations Research, 2018.

[75] Pal SK, Pramanik A, Maiti J, Mitra P. Deep learning in multi-object detection and tracking: State of the art. Applied Intelligence, 2021, 51: 6400-6429.

[76] Pillet-Shore, D. M. (2008). Coming together: Creating and maintaining social relationships through the openings of face-to-face interactions. University of California, Los Angeles.

[77] Popoola OP, Wang K. Video-based abnormal human behavior recognition—a review. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 2012, 42(6): 865-878.

[78] Qi, J., Mazumdar, S., & Vasconcelos, A. C. (2024). Understanding the relationship between urban public space and social cohesion: A systematic review. International Journal of Community Well-Being, 7(2), 155-212.

[79] Raynor K. Social representations of children in higher density housing: Enviable, inevitable or evil?. Housing Studies, 2018, 33(8): 1207-1226.

[80] Ratnayake, R. (2017). Sense of safety in public spaces: University student safety experiences in an Australian regional city. Rural society, 26(1), 69-84.

[81] Reis, D., Kupec, J., Hong, J., & Daoudi, A. (2023). Real-Time Flying Object Detection with YOLOv8 (arXiv:2305.09972). arXiv. http://arxiv.org/abs/2305.09972

[82] San Juan C, Subiza-Pérez M, Vozmediano L. Restoration and the city: The role of public urban squares. Frontiers in Psychology, 2017.

[83] Schegloff EA. The routine as achievement. Human Studies, 1986, 9(2–3): 111-151.

[84] Schlickman E. Assessing automation: Methodological insights from experimenting with computer vision for public life research. Journal of Landscape Architecture, 2020, 15(3): 48-59.

[85] Schneekloth L, Shibley RPlacemaking: The art and practice of building communities, 1995Wiley

[86] Shechtman, E., & Irani, M. (2005). Space-time behavior based correlation. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), 1, 405–412 vol. 1. New York: IEEE. https://doi.org/10.1109/CVPR.2005.328

[87] Simões Aelbrecht P. ‘Fourth places’: The contemporary public settings for informal social interaction among strangers. Journal of Urban Design, 2016, 21(1): 124-152.

[88] Stamps AEIII. Isovists, enclosure, and permeability theory. Environment and Planning b: Planning & Design, 2005, 32(5): 735-762.

[89] Stauffer C, Grimson WEL. Adaptive background mixture models for real-time tracking. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1999, 2: 246-252.

[90] Stokols D. Translating social ecological theory into guidelines for community health promotion. American Journal of Health Promotion, 1996, 10(4): 282-298.

[91] Talen E. Sense of Community and Neighbourhood Form: An Assessment of the Social Doctrine of New Urbanism. Urban Studies., 1999, 36: 1361-1379.

[92] Tang WS, Lee JW, Hui TW, Yip MK. The “Urban density” question in Hong Kong: From absolute space to social processes. City, Culture and Society, 2019, 17: 46-53.

[93] van Ameijde, J. & Leung, K.S. (2022). UAV-based People Location Tracking and Analysis, for the Data-Driven Assessment of Social Activities in Public Spaces. In: van Ameijde, J. Gardner, N., Hyun, K.H., Luo, D., Sheth, U. (eds.), Post-Carbon - Proceedings of the 27th CAADRIA Conference, Sydney, 9-15 April 2022 https://doi.org/10.52842/conf.caadria.2022.1.293

[94] Wahl HW, Iwarsson S, Oswald F. Aging well and the environment: Toward an integrative model and research agenda for the future. The Gerontologist, 2012, 52(3): 306-316.

[95] Wang, R., Yuan, Y., Liu, Y., Zhang, J., Liu, P., Lu, Y., & Yao, Y. (2019). Using street view data and machine learning to assess how perception of neighborhood safety influences urban residents’ mental health. Health & place, 59, 102186.

[96] Wang Z, Li Y, An J, Dong W, Li H, Ma H, Wang J, Wu J, Jiang T, Wang G. Effects of Restorative Environment and Presence on Anxiety and Depression Based on Interactive Virtual Reality Scenarios. International Journal of Environmental Research and Public Health, 2022, 19(13): 7878.

[97] Whyte WHThe social life of small urban spaces, 1980Conservation Foundation

[98] Yan, W., & Forsyth, D. A. (2005). Learning the behavior of users in a public space through video tracking. In 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05)-Volume 1 (Vol. 1, pp. 370–377). IEEE.

[99] Yang Y, Yuan Y, Pan T, Zang X, Liu G. A framework for occupancy prediction based on image information fusion and machine learning. Building and Environment, 2022, 207. 108524

[100] Yue Y, Zhuang Y, Yeh AGO, Xie JY, Ma CL, Li QQ. Measurements of POI-based mixed use and their relationships with neighbourhood vibrancy. International Journal of Geographical Information Science, 2016, 31(4): 658-675.

[101] Yung HK, Ko HW, Chan HWE. Elderly satisfaction with planning and design of public parks in high density old districts: An ordered logit model. Landscape and Urban Planning, 2017, 165: 39-53.

[102] Zaki MH, Sayed T. Automated analysis of pedestrian group behavior in urban settings. IEEE Transactions on Intelligent Transportation Systems, 2017, 19(6): 1880-1889.

[103] Zapata, O., & Honey-Rosés, J. (2022). The behavioral response to increased pedestrian and staying activity in public space: a field experiment. Environment and Behavior, 54(1), 36-57.

[104] Zhang J, Liu H, Li Y, Qin X, Wang S. Video-driven group behavior simulation based on social comparison theory. Physica a: Statistical Mechanics and Its Applications, 2018, 512: 620-634.

[105] Zhang, J., Qi. R., Zhang, H. (2023) Examining the Impact of Crowding Perception on the Generation of Negative Emotions among Users of Small Urban Micro Public Spaces. Sustainability,15(22), 16104. https://doi.org/10.3390/su152216104

[106] Zukin SThe culture of cities, 1995Blackwell

Funding

Research Grants Council, University Grants Committee(14616723)

Chinese University of Hong Kong(4052224)

Hong Kong Construction Industry Council(4640)

RIGHTS & PERMISSIONS

The Author(s)