Ambulatory measurement of facial expressions in health and disease
Research type
Research Study
Full title
Ambulatory measurement of facial expressions in health and disease
IRAS ID
236495
Contact name
Charles Nduka
Contact email
Sponsor organisation
Queen Victoria NHS Foundation Trust
Duration of Study in the UK
2 years, 0 months, 1 days
Research summary
Summary of Research
Spontaneous facial expressions are part of daily interactions, but can be affected by a number of health conditions. The aim of this project is to develop a sensor enabled glasses, that can detect facial expressions of the wearer to provide pervasive monitoring of treatment effects outside the clinic. Potential beneficiaries of this technology include service users with conditions that affect facial expressions such as those living with facial palsy, Parkinson's disease and depression. FRAME is being developed as a NIHR-funded project in partnership between the host, Queen Victoria Hospital NHS Foundation Trust, and Emteq Ltd, a technology company co-founded by the study PI, Charles Nduka.In order to assess facial expressions in specific conditions, we need to understand the patterns of data created by non-clinical volunteers as well as service users. The pilot study consists of 2 parts. First, we will investigate facial expression of service users living with these conditions and of healthy participants in response to standardised video clips designed to provoke emotional responses (Samson, Kreibig, Soderstrom, Wade, & Gross, 2016). Whilst participants are watching these videos, we will assess facial muscle activity using (i) electromyography (EMG), (ii) the non-invasive sensor technology, FRAME, embedded in a pair of glasses and (iii) video recording. This will enable us to establish a baseline and highlight markers which can help enable the technology to distinguish between emotional facial expression responses. We will also ask participants to complete a series of self-assessments.
The second part of the study will investigate the recruitment usability, and retention rates of participants wearing FRAME over an extended period of time. This study will enable us to evaluate how well we can monitor facial expressions “in the wild” by having service users use the glasses at home. Participants will be asked to wear the FRAME glasses, during weekdays for up to 4 weeks at home. In addition to these measures, participants will be asked to complete short condition-specific questionnaire 3 times a day.
The eventual outcome of this pilot project will be a technology that will enable objective, remote measurement of facial expression responses. For facial palsy service users, this will enable biofeedback when expressions are asymmetric beyond a set threshold. For service users with Parkinson’s disease this may help in treatment monitoring, identifying clinical subtypes, and with managing and maximising the effectiveness of ongoing treatment.
Summary of Results
This project was designed to help patients suffering with the consequences of facial paralysis with their muscle rehabilitation. Facial Remote Activity Monitoring Eyewear (F.R.A.M.E.) aimed to develop a pair of glasses that discreetly provides real-time feedback to the wearer about their facial muscle function, helping them practice their rehabilitative exercises regularly and correctly, thereby speeding recovery of normal, symmetric facial expressions.Studies show that following facial paralysis rehabilitative exercises speeds recovery and improves the patient’s outcome. However, patients usually have limited awareness of the abnormal movements (called synkinesis) their faces display. Therefore, without muscle feedback about these movements, their facial function may worsen, developing permanently abnormal movements.
Research gathered in the initial work packages (via patients and therapists focus groups) highlighted the fact that most patients with synkinesis are not under the care of a rehabilitation specialist and will therefore not be ready to start active facial movements. Their high facial muscle tone means that significant work will be needed to stretch tightened facial muscles and lower resting muscle tone. It was recognised that prior to use of the rehabilitation technology for active rehabilitation, patients will need to lower their resting facial muscle tone through the use of contact sensors on the cheeks and around the eyes
To ensure a thorough understanding of user needs, a facial palsy research priority setting exercise was undertaken in collaboration with the Centre for Appearance Research (University of the West of England) to ensure that the potential needs of stakeholders were being considered. The results were published in the Journal of Plastic Reconstructive and Aesthetic Surgery https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fu2790089.ct.sendgrid.net%2Fls%2Fclick%3Fupn%3DXv3JSvJ-2B3M71ppf7N9agbZdV5vKXPbKGPQxM6rZTJIai2maWY7qhbeqdKWU98R4ocU4uRZI1Qbk6SLgctQTdnbeO1LowpRIY5n1uHiXrNjAk96eRh5Bdsqpfv89PiVK31pCyjDPqt2Wy0IzepUPdhpwR0Ty0M3d3H5jyv-2BlgyxDqskoTdkuGLTCS9zoUs1a342QEOQ71nj6l5lCC9Nj4C5SHKhExN-2B1Va4KAkDRbgBc-3DCv_W_E1aO2-2BZlVOSJJV-2FajQqskegTd6IRomHYTi-2Fbt8SH3YLqNpzOpgfpxlJakItKrbYyqd4uxtF8LyI832w02TRnV8yXICh28t945qU2o3DTB0altLgFhuGxJVthIWZnEqOpROFshvHOvhOx1vR0mRYdDHOMP0c03EV0v3gHBFeTu1Kg93dAyyVHCGIm4EKqxhV93-2FYGL-2FKHw6yLBHcXgaYbzA-3D-3D&data=05%7C01%7Capprovals%40hra.nhs.uk%7C91a83c30b5e14a3f18d008da8fec277d%7C8e1f0acad87d4f20939e36243d574267%7C0%7C0%7C637980541681102566%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=JTHkzjhoiaR6qs6bUcWavvisNURWUwkFZhta85X9tcE%3D&reserved=0
In terms of glasses technology development, facial muscle sensor locations were defined to capture 8 key expressions. This required the development of 3 iterations to achieve a sufficient degree of accuracy.
• Data collected from the first 38 subjects were subjected to newly developed machine learning methods and incorporated into an accurate algorithm.
• The device was optimised for stability and accuracy and expression engine softwarePPI Involvement & dissemination
A feedback sessions held at QVH, engaged with 10 Facial Palsy patients and asking them to give their reactions to the design of the prototype EMG goggles, and for them to wear the goggles for a more extended period and give feedback on the comfort. The patients were all asked to complete a questionnaire, and the outcome was a 100% positive reaction to the design. All would be willing to wear the goggles at home while practicing relaxation exercises, and found the device comfortable to wear. Some felt that the device felt fragile, and this has been fed back into the prototype design work package.Between January and March 2019, glasses data was collected from patients who used them paired with a smartphone. There were 18 Facial palsy patients and 13 healthy volunteers. Recruitment was slightly lower than planned due to high incidence of sickness and chest infections. Participants performed a protocol of daily use at home, conducted usability feedback and watched emotive videos to assess real-world facial expression responses
PPI was conducted via Facial Palsy UK, and a presentation was made by the lead investigator to update the community on the project.
The economic viability of FRAME technology was assessed in a separate work package. An economic analysis was conducted based on expected or average costs and outcomes, while review evidence on the effectiveness of facial rehabilitation therapy demonstrates substantial variation in recovery patterns across patients. Such heterogeneity in effectiveness should be accounted for in a final economic evaluation. The results of the review are awaiting publication in the journal Clinical Rehabilitation, entitled: Physical therapy for facial nerve paralysis (Bell’s palsy): An updated and extended systematic review of the evidence for facial exercise therapy [Amir Khan; Ala Szczepura; Shea Palmer; Chris Bark; Catriona Neville; David Thomson; Helen Martin; Charles Nduka].
A cohort based study or a RCT is recommended for a full economic evaluation of the FRAME technology. Since the study completed, further work has been undertaken to set up a larger clinical evaluation.
REC name
West Midlands - Edgbaston Research Ethics Committee
REC reference
18/WM/0205
Date of REC Opinion
14 Sep 2018
REC opinion
Further Information Favourable Opinion