Using gaze direction to study human collaboration behaviour

Wright, Robert Dale (2023) Using gaze direction to study human collaboration behaviour. (PhD thesis), Kingston University, .

Abstract

Every day people are involved in group discussions with little understanding of the subtle behaviours and interactions that play out in the conversation. The investigation into conversational behaviour and the interaction of participants in general discussions has been an active research area for many years. Research has focused on the subtle interchanges (verbal and non-verbal) between participants with the result of identifying user profiles of dominance and personality. In most cases, and still today, psychologists and others use manual observations in workplace analysis. In my career as a workplace strategist, manual observation plays a crucial role in understanding office use and organisational behaviour. Only recently has video and sensor technology been investigated to support the capture and analysis of conversational interactions. Nearly all observation methods use invasive combinations of modalities, for example, voice recording, video capture and other sensors to observe participants. These technologies are either directly in the line of sight (e.g. video cameras for recording facial expression and gaze direction) or attached to the participant (eye-tracking glasses, lapel microphones or sensors attached to the head for capturing head rotation). This research explores a passive approach, significantly minimising the impact of technology on the participants' natural behaviour using an overhead-mounted depth camera. Unlike other research, this thesis uses only non-verbal cues (e.g. gaze directions) to analyse the conversation flow in order to build a picture of how the discussion progresses. Since this approach is unable to capture details such as eye movement, facial expressions or changes in the tone of the participant's voice, the challenge is to explore what is achievable from such a sensor configuration and what level of detail it can provide in the study of group conversational behaviour. The research makes several contributions to this topic: a) investigation into an overhead capture method using a depth data sensor, which avoids impact on the participants natural behaviour, b) using computer vision techniques to automatically extracts non-verbal features of head localisation, gaze direction and interpersonal posture measurements, c) the application of machine learning classification methods on the extracted features, patterns of behaviours are detected and associated with each participant and the group, d) quantitative analysis of participant behaviours such as speaking contribution, focus of attention of speakers and listeners, and the postures used by each person. The data supports an 4 understanding of the conversational flow and the development of individual profiles. Output is a single visual of the complete conversation with all non-verbal interactions presented and group profiles of contribution levels. The research demonstrates what can be achieved using a single overhead sensor method to capture specific behaviours of an individual and their collective impact on a group discussion. It employs computer vision analysis and supervised classification methods to deliver an automated and repeatable approach to studying conversational behaviour.

Actions (Repository Editors)

Item Control Page