Real time hand movement trajectory tracking for enhancing dementia screening in ageing deaf signers of British Sign Language

Liang, Xing, Kapetanios, Epaminondas, Woll, Bencie and Angelopoulou, Anastassia (2019) Real time hand movement trajectory tracking for enhancing dementia screening in ageing deaf signers of British Sign Language. In: Cross Domain Conference for Machine Learning and Knowledge Extraction; 26-28 Aug 2019, Canterbury, U.K..

Official URL: https://dx.doi.org/10.1007/978-3-030-29726-8_24

Abstract

Real time hand movement trajectory tracking based on machine learning approaches may assist the early identification of dementia in ageing Deaf individuals who are users of British Sign Language (BSL), since there are few clinicians with appropriate communication skills, and a shortage of sign language interpreters. Unlike other computer vision systems used in dementia stage assessment such as RGB-D video with the aid of depth camera, activities of daily living (ADL) monitored by information and communication technologies (ICT) facilities, or X-Ray, computed tomography (CT), and magnetic resonance imaging (MRI) images fed to machine learning algorithms, the system developed here focuses on analysing the sign language space envelope (sign trajectories/depth/speed) and facial expression of deaf individuals, using normal 2D videos. In this work, we are interested in providing a more accurate segmentation of objects of interest in relation to the background, so that accurate real-time hand trajectories (path of the trajectory and speed) can be achieved. The paper presents and evaluates two types of hand movement trajectory models. In the first model, the hand sign trajectory is tracked by implementing skin colour segmentation. In the second model, the hand sign trajectory is tracked using Part Affinity Fields based on the OpenPose Skeleton Model [1, 2]. Comparisons of results between the two different models demonstrate that the second model provides enhanced improvements in terms of tracking accuracy and robustness of tracking. The pattern differences in facial and trajectory motion data achieved from the presented models will be beneficial not only for screening of deaf individuals for dementia, but also for assessment of other acquired neurological impairments associated with motor changes, for example, stroke and Parkinson’s disease.

Official URL:	https://dx.doi.org/10.1007/978-3-030-29726-8_24
Item Type:	Conference or Workshop Item (Paper)
Event Title:	Cross Domain Conference for Machine Learning and Knowledge Extraction
Organising Body:	International Federation for Information Processing
Additional Information:	Published in: Holzinger, Andreas , Kieseberg, Pete , Tjoa, A Min and Weippl, Edgar, (eds.) (2019) International Cross-Domain Conference for Machine Learning and Knowledge Extraction CD-MAKE 2019: Machine Learning and Knowledge Extraction. Cham, Switzerland : Springer International Publishing. pp. 377–394. (Lecture Notes in Computer Science, no. 11713) ISSN (print) 0302-9743. This work was supported by the Dunhill Medical Trust Grant [Grant Number: RPGF1802\37 UK]
Uncontrolled Keywords:	Segmentation; Hand tracking; OpenPose; Sign language; Dementia; Time-series data analytics
Research Area:	Research Areas > Computer science and informatics
Faculty, School or Research Centre:	Faculty of Science, Engineering and Computing Faculty of Science, Engineering and Computing > School of Computer Science and Mathematics
Date Deposited:	23 Jun 2022 13:59
Last Modified:	23 Jun 2022 13:59
DOI:	https://doi.org/10.1007/978-3-030-29726-8_24
URI:	https://eprints.kingston.ac.uk/id/eprint/51908

Actions (Repository Editors)

Item Control Page