EnNoisis

Automatic Recognition of Daily Activities, within an MCI/Alzheimer monitoring and support context

In the context of the Greek, nationally funded, research project EnNOISIS, which aims at advancing progression monitoring and treatment of MCI and Alzheimer’s disease, we have developed a framework enabling the automatic recognition of in-house activities of daily living, such as cooking, eating, etc., of MCI/Alzheimer patients, through processing of depth images provided by Kinect sensors. Activity detection is based on the 2D trajectories that are virtually generated on the house floor plan, as the user moves around the house during performing the target ADLs. This video demonstrates how daily activities of cooking, eating and dishwashing are detected through our system, using only the user location trajectories.

 

Daily activity detection demonstration video

 

Physical exercise interventions for MCI, guided through VR-trainer and monitored through markerless motion tracking

In the context of the EnNOISIS project, we have developed a VR-based physical exercise companion tool, with the aim to provide doctors with motion-tracking based feedback, over the patient’s capacity to perform a set of physical exercises typically used for obtaining insight over MCI progression (e.g. arms lifting, arms stretching, torso bending and twisting). The tool incorporates a virtual trainer demonstrating exercises that should be performed by the user. While performing the exercises, the user is monitored through a Kinect camera, utilizing only depth images and markeressly estimated joint locations. Through a set of metrics we have developed, based on viewpoint-invariant angular skeletal information, the similarity between the current patient’s attempt and either (a) the optimal movement recorded by a trainer or (b) the movement of the same user recorded in the past is assessed. Eventually, the tool provides evaluation scores over the patient’s capacity to perform the supported exercises, as well as insight regarding the patient’s capacity progression over time.

 

Video demonstration of the developed tool

 

Cognitive reinforcement and monitoring tools, targeting MCI and Alzheimer’s disease

In the context of the EnNOISIS project, we have developed a series of tools, for assisting healthcare professionals in applying interventions for cognitive reinforcement and monitoring of MCI and Alzheimer patients. Our developed tools include (1) a virtual supermarket, where the user is engaged in a shopping experience, demanding the discovery and purchase of specific goods, (2) a virtual garden, where the user is engaged in gardening activities, as well as a (3) a memory screening test suite, involving tests that assess optical and lexical memory, capacity of image comparison, reaction time to stimuli, orientation in space and time, etc. These tools are accessible over the Internet, at the following links:

  1. Virtual Supermarket (requires Unity Web Player)
  2. Virtual Garden (requires Unity Web Player)
  3. Memory Screening Test