CiViL
Research type
Research Study
Full title
Common-sense and Visually-enhanced Natural Language Generation (CiViL) for Human-Robot Interaction
IRAS ID
325635
Contact name
Dimitra Gkatzia
Contact email
Sponsor organisation
Edinburgh Napier University
Duration of Study in the UK
3 years, 0 months, 0 days
Research summary
Video Demo: https://youtu.be/Jkubvfwuqec
In the UK more than 3.25 million people were in contact mental health services during 2021/22 accounting for around 5.8% of the population. With the increasing demand for such services, the ability of the NHS to meet the needs of service users is stretched by understaffing, increasing costs and a lack of readily available facilities and equipment, resulting in additional pressure on staff and services. In respect of patient needs, the ability to undertake basic domestic tasks such as cooking to sustain a healthy and balanced diet is one of the most important health factors in our daily life. Nevertheless, some people with specific mental health needs can struggle with such tasks, which can often result in low mood, a lack of motivation and physical and mental exhaustion leading to eventual decline. 4 Typically, skills training in this area can be highly demanding and time consuming within a healthcare setting due to limited staff availability and resources for on-site and in-home personal assistance.
In consideration of the above, our research moves away from classic instruction giving tasks and incorporates question-answering for clarification requests, and common-sense abilities, such as swapping ingredients and requesting information on how to use or locate specific utensils within a kitchen environment. This results in altering the goal of the communication from cooking a recipe to requesting information on how to use a tool or explain a method, and then resuming to the main overall goal. In a series of human evaluations using a 102 random sample, we quickly observed that changing the dialogue goal from completing the recipe to providing information about relevant tasks resulted in failure of task completion, i.e., such questions are normally out of scope. 8 This outcome is typical in virtual home assistants such as like Alexa, Siri or Cortana, as their pre-determined dialogue frameworks cannot effectively manage natural communication phenomena that is not pre-scripted.
We therefore addressed this issue by re-framing failure as a temporary dialogue goal change, which allowed the users to engage in question answering that was not necessarily relevant to a specific recipe and then force the system to resume the original goal after completing an essential sub-task. With our humanoid robotic chef ‘Euclid’, we aim to provide around-the-clock skills training in food preparation with emphasis on healthy living in a hands-on learning experience for rehabilitation and assistive learning settings.
In our evaluation aim to measure how effective our robot is at helping people to understand instructions, correctly handle tools and objects, locate missing objects in each environment and find appropriate alternative ingredients for missing items without human assistance. We believe this flexibility to be significant for mental health applications, particularly in areas of mental health such as rehabilitation and skills training where goal changes are frequent. Finally, our robot is novel in that it is a bespoke, low-cost, low-powered, human-like, open-source, and platform-independent system that is designed to be configured in a number of different ways to adapt to different tasks. We feel that unlocking the above challenges in the cooking domain may open doors to other areas of skills training and rehabilitation in healthcare settings such as personal hygiene, establishing routines and maintenance tasks.
REC name
London - Camberwell St Giles Research Ethics Committee
REC reference
23/PR/0670
Date of REC Opinion
30 Jun 2023
REC opinion
Favourable Opinion