Alzheimer's disease detection via human activity recognition and temporal segmentation in untrimmed videos

Alzheimer's disease (AD) is a well-known type of dementia, characterized by neurodegeneration that interferes with daily activities. It affects the cells in the hippocampus region of the brain, leading to an irreversible brain disorder. People developing AD often struggle to remember small details and gradually lose the ability to perform complex daily living tasks. AD is a progressive disease that shows no symptoms in its early stages, making diagnosis challenging through periodic medical exams. Proposing a test that could be fully performed remotely at home by a family member could have a significant impact on diagnosing AD in its early stages. In this study, we propose a novel approach for detecting Alzheimer's disease by analyzing daily living activities from untrimmed videos (such as visual data from the Dem@care dataset). The proposed approach leverages deep learning techniques to determine whether an individual is affected by Alzheimer's or not, through a three-step pipeline: The first step (a) employs a Human action recognition approach, where an I3D model as Spatio-Temporal Convolutional Neural Network (ST-CNN) is trained and applied to differentiate between various daily activities based on RGB video data. The second step (b) is presented as Temporal action segmentation approach, using a sliding window technique to extract temporal information by generating a sequence of predicted actions from the untrimmed videos, this is achieved by applying the trained I3D model over the entire video. The third step (c) involves final classification, where a Long Short-Term Memory (LSTM) network is used to analyze the sequence of predicted actions and classify individuals as either cognitively healthy or affected by dementia.Our approach achieves an accuracy of 92.98 % on the Dem@care dataset, which, to the best of our knowledge, is the first result of this kind for such a methodology. Furthermore, we developed and tested an application for its deployment in real-world settings.