Military Service Identification Tool
Research type
Research Study
Full title
Validating the Military Service Identification Tool in its ability to correctly identify civilians and those who have served in the military.
IRAS ID
278987
Contact name
Reza Razavi
Contact email
Sponsor organisation
King's College London
Duration of Study in the UK
1 years, 0 months, 1 days
Research summary
Research Summary
Estimates of the United Kingdom’s (UK) military veteran population, defined by the British Government as those who have served in the military for at least one day [1], is approximately 2.5 million, equivalent to around 5% of household residents aged 16 years or over in the UK [2]. UK military veterans receive healthcare provision from the National Health Service (NHS), with care recorded in local, regional and national Electronic Healthcare Records (EHRs) [3]. EHRs – structured and unstructured (i.e. free text) – can be used to evaluate disease prevalence, surveillance, to perform epidemiological analyses and investigate quality of care and to improve clinical decision-making [4], [5].
There is no national marker in UK EHRs to identify veterans, nor is there a requirement for healthcare professionals to record it, making it difficult to evaluate the unique healthcare needs of those who have served in the UK Armed Forces [6]. This study, funded by Forces in Mind Trust, seeks to validate the Military Service Identification Tool, an open-source computer program that searches through free-text clinical notes to make a prediction on a person’s military status. It is in the public interest to know the health of our Armed Forces. The Tool has been validated using manually annotated datasets, but we now need to validate an individual’s military status by contacting them via post or telephone and asking, “Have you ever served in the Armed Forces”. The research team will work closely with the Clinical Record Interactive Search (CRIS) Patient Advisor Group and local healthcare professionals.
Summary of Results
This study used electronic healthcare records within the Clinical Record Interactive Search (CRIS) system and the South London and Maudsley (SLaM) Biomedical Research Centre (BRC) Case Register to validate a computer-based tool, the Military Identification Service Tool, that sought to identify veterans and civilians who had accessed secondary mental health care services through the SLaM National Health Service (NHS) Foundation Trust.
To validate the veteran/civilian status characterised by the Military Identification Service Tool, we emailed and text patients with a survey link (with an information sheet and consent form included) that allowed them to self-disclose their veteran/civilian status.
Of those eligible to take part in the study (i.e. having Consent for Contact, were not a current inpatient, did not have dementia or psychosis, were alive and aged over 18 years old), 902 individuals (n=723 civilians, n=179 veterans) were contacted. A total of n=149 unique respondents engaged with the survey and 146 responses were used for analysis. Of the n=146 sample, 112 were categorised as MSIT civilians and 34 were categorised as veterans.
The present study found that 84.2% of the sample were accurately categorised by MSIT. The overall sensitivity of the tool (that is, the ability to detect true civilians) was found to be 0.82, and the specificity was 0.91 (that is, the ability to detect true veterans). An examination of the exceptional cases where MSIT misclassified veteran or civilians showed that MSIT tended to categorise civilians as veterans on the basis of keywords like the Salvation Army (n=23 compared to n=1 veteran inaccurately categorised as a civilian). This information helped the researchers to apply minor revisions to the keywords and to better train the tool.
REC name
East of Scotland Research Ethics Service REC 2
REC reference
20/ES/0060
Date of REC Opinion
13 Jul 2020
REC opinion
Further Information Favourable Opinion