Report of a publications review jointly commissioned by the Health Research Authority and the University of Bristol

Report of a publications review jointly commissioned by the Health Research Authority and the University of Bristol 

EXECUTIVE SUMMARY 

Report 

This report describes the deliberations and conclusions of a panel of experts commissioned by the Health Research Authority and the University of Bristol to review eleven publications on chronic fatigue syndrome and myalgic encephalomyelitis, all of which cited the same National Health Service Research Ethics Committee reference 07/Q2006/48 in the ethics statement. 

Process 

Terms of reference setting out the purpose and principles of the review were agreed in advance between the Health Research Authority, the University of Bristol and the members of the panel. 

The panel met in person at the University of Bristol on 5 June 2019, having previously been provided with copies of all eleven publications along with other relevant correspondence and documents. The panel requested some additional information before finalising the review. 

The panel's findings and recommendations were unanimous. 

This report was prepared on behalf of the panel and was approved in draft by all members. 

Key findings 

(a) The Research Ethics Committee documentation was incomplete and was in some respects lacking in clarity and open to interpretation;

(b) No favourable ethical opinion was obtained in respect of any of the papers reviewed;

(c) In all eleven papers, the Research Ethics Committee reference, as cited in the ethics statement, was either inappropriate or inadequate to cover the activity described;

(d) However, no additional ethical review was required and the position could in all the cases considered be rectified by rewording and clarification of the ethics statement;

(e) In two papers, additional minor textual changes in the main body of the publication would improve clarity.

Recommendations 

The panel did not recommend the withdrawal of any publication. The panel made specific recommendations for corrections to the ethics statements in all papers. There were suggested minor amendments to the text of two papers, to be notified to the relevant journal editors for each of the publications reviewed. The panel also recommended that funding bodies be notified, where appropriate. 

Next steps 

This report and the panel's findings and recommendations will be considered by the Chief Executive of the Health Research Authority and the Pro Vice-Chancellor for Research at the University of Bristol, who will decide on any further action to be taken. 

Main report

Introduction 

1. This report contains a summary of the process, deliberations and conclusions of a confidential review by a panel of experts appointed by the Health Research Authority (HRA) and the University of Bristol, of eleven publications on chronic fatigue syndrome and myalgic encephalomyelitis (CFS/ME), all of which cited the same National Health Service Research Ethics Committee (REC) reference 07/Q2006/48 in the ethics statement. 
   
   
2. The publications review was commissioned following a number of complaints received by the HRA and by journal editors, questioning the repeated use of the same REC reference in eleven papers published in peer-reviewed journals over a period of several years, for which the main author and Principal Investigator (Pl) was a member of academic staff at the University of Bristol. The REC reference was cited in each case as confirming that the collection and analysis of the data was part of service evaluation and did not require ethical review.
   
   

   Selection and constitution of the panel

   
   
3. The panel was made up of two members of academic staff at the University of Bristol ('the University') and an independent Chair nominated by the HRA who was a clinical academic from another Russell Group university. All three members of the panel were of senior professorial status, from different professional disciplines, and were selected by agreement between the HRA and the University, for their specific expertise and experience in relation to the ethics of research into health related matters and for their ability to carry out the task which they had been assigned in a fair, impartial and objective manner. No member of the panel had any prior connection with research into CFS/ME or with the complaints that had been received. 
   
   
4. In view of the controversy surrounding research in this area and the potential for personal comment in social and other media, the panel requested that their anonymity be preserved. All members of the panel had agreed to undertake the review outside their normal professional duties. 
   

   The review process
   
   

5. Terms of reference setting out the purposes and principles of the review were agreed in advance between the HRA, the University and the members of the panel, and are attached to this report as Appendix A. 
   
   
6. A bundle of documents as listed in Appendix B was prepared and provided to the panel several weeks before the date of the review in order to allow sufficient time for members to read and review all the material. The panel met in person in Bristol on 5 June 2019. 
   
   
7. The panel reviewed and discussed each publication in turn, as set out below, and unanimously agreed its initial findings and conclusions at the meeting. The panel requested some additional information from the University and the HRA before finalising its review. This report was prepared in draft for comment and approval by the panel after the meeting. 
   

   The panel's approach 
   
   

8. The panel began the review by discussing a general definition of research and their agreed understanding of current best practice in relation to the collection of clinical data. The panel agreed that even where data was being collected in the routine course of clinical care and would be anonymised, it would be best practice to inform patients that this data might in future be used for research purposes and to allow patients to opt out. However this expectation of best practice today must be seen in the light of changing practice in research generally over the years since the ethics application in question was made in 2007 and the changing nature of the specific guidance issued by research ethics committees, the HRA and other regulatory bodies during the time period under consideration. 
   
   
9.  In identifying the essential characteristics that define research, the panel had regard to the HRA leaflet Defining Research (2013) and to the decision tool published on the HRA website: http://www.hra-decisiontools.org.uk/research/. The panel noted that key indicators to distinguish research from service evaluation or audit included attempts to test a hypothesis and to derive new knowledge that could be generalised. 
   
   
10. Before reviewing each of the publications in turn, the panel scrutinised the application for ethical approval that resulted in the REC reference cited in the ethics statements. 
    

    North Somerset & South Bristol Research Ethics Committee reference 07/Q2006/48 
    
    

11. the REC documentation before the panel consisted of the following: 
    
    

    (a) A letter dated 29 January 2007 ('the January letter') from the REC Chair to the Pl in response to a communication from the Pl dated 9 January 2007
    

    (b) The full application for ethical review completed by the Pl and dated 1 April 2007 for a study entitled 'What happens to children with CFS/ME? The study of a longitudinal cohort of children who access a paediatric CFS/ME service'

    (c) Minutes of the meeting of the REC on 30 April 2007

    (d) A letter dated 1 May 2007 ('the May letter') from the REC Chair to the Pl.

12. In view of their significant relevance to the panel's findings and recommendations, documents (a) to (d) above are reproduced in Appendix C. The panel did not have copies of the supporting material provided to the REC by the Pl (information sheets, consent forms, etc) but did not consider this material to be necessary to its review. 
    
    

    The panel was not provided with the original communication dated 9 January 2007 from the Pl to the REC Chair and despite further enquiries to the Pl, the HRA and the REC, a copy of this communication could not be found. 

    (a) The January letter
    
    

13. The panel understood from the January letter that the Pl had sent in a written enquiry that was discussed at the REC meeting on 26 January 2007. (1. The HRA has since confirmed that there is no record of this discussion in the archived Minutes of the REC meeting on 26 January 2007.) The January letter contained responses to three questions that had been put by the Pl. The panel deduced from those responses that the Pl had explained that she was intending to collect patient data as part of clinical assessment, as well as data for two extra time points that were not clinically necessary; that this data would be anonymised for the purpose of service evaluation 'as required by the national cooperative' (which the panel took to refer to the intention to create a National Outcomes Database (NOD) for CFS/ME services nationwide); and that she had asked what information should be provided to patients and what ethical permissions would be required. 
    
    
14. The January letter referred under question (1) to 'data being collected as part of clinical assessment' and 'data for the extra time points' and advised that if the Pl envisaged using 'this data' for a research project in the future, it would be best to inform the families. The panel found that it was not clear whether the term 'this data' referred to all the data being collected, or only the data for the extra time points, and that although the latter was more likely, either interpretation was possible. 
    
    
15. Under question (2), the January letter stated that it would not be necessary to apply for ethical permission to use 'the data that you are collecting as part of clinical assessment and as part of service evaluation'. The panel took this to refer to the data first mentioned under question (1). 
    
    
16. Under question (3), the January letter stated that ethical permission to use the data generated from the extra time points would be required. 
    
    
17. The final paragraph of the January letter stated that if in future the Pl wished to use data that was being currently collected and subsequently anonymised as part of a research project, 'this would be agreeable'. The panel found that it was not clear whether the 'data' referred to in this paragraph included all the clinical data being collected and anonymised for the national collaborative, or only the paediatric data in respect of which the ethics application was being made, and that either interpretation was reasonable. Subject to this question of interpretation, the panel took the phrase 'this would be agreeable' to mean that the REC did not require a further ethics submission from the Pl in the circumstances described in this paragraph. 
    

    (b) The application for ethical review
    
    

18. As required by the January letter, the Pl submitted an ethics application for the use of the data generated from the two extra time points. The panel reviewed the application form in detail, noting the following points by reference to the section numbers in the form: 
    
    

    Section 1: In response to the question 'Is your project an audit or service evaluation?' the Pl selected the answer No. The panel found this surprising, as although the application only related to the two extra time points, other sections of the form (e.g. l0A-1, 38 and 45-1) indicated that this data was also being collected for the purpose of service evaluation. 

    Section 2: The research category selected was 'Research administering questionnaires/interviews for quantitative analysis or using mixed quantitative/qualitative methodology'. 

    Section A4: The primary purposes of the research were categorised as 'Establishing a database/data storage facility' and the evaluation of a new service for children with CFS/ME (under 'Other'). The panel believed the first purpose to refer to the establishment of the NOD. 

    Section AG: The research is described as involving 'routine clinical monitoring using questionnaires that are already being used, at two extra time points'. 

    Section A7: This section sets out the research question and objective: 'to measure outcomes in children with CFS/ME in order to allow us to evaluate our service' [emphasis added] and 'to develop a method for being able to assess whether outcome improves as we improve the service'. 

    Sections A8 to 10: These sections give further information about the purpose, design and methodology of the project, including an explanation of the reasons for selecting the two extra time points for administering questionnaires that were not clinically necessary. The panel found that all of this information was consistent with an intention to carry out an evaluation of the service. 

    Sections A26 and 27: These sections confirm that consent will be obtained for the two extra time points. 

    Sections A35-2 and 35-3: These sections confirm that NHS sites will be used and that the NHS indemnity scheme will apply. 

    Sections A37 and 38: These sections set out how the results of the data collection will be published, including in patient journals 'if useful', stating that this is primarily service evaluation. 

    Sections A40 to 44: These sections cover the anonymisation and confidentiality of the data collected, including the intention to store the data on a secure database on a university computer. The panel understood this to refer to the establishment of the NOD, subsequently hosted by the University of Bristol. 

    Section A45-1: This section repeats that the data collection is not primarily for research, but for service evaluation. 

    Section ASS: This section states that no external funding has been secured, as it is considered that the data collection will improve the service provided by the clinical team. The panel noted in their discussions that external funding is not normally available for service evaluation. 

    Section A68: This section, headed 'What are the main ethical issues with the research?, includes a request from the Pl for advice from the REC on whether, rather than seeking consent for collecting the extra data for service evaluation, it might be better for patients to be able to opt out of the anonymised data collection. The section concludes by questioning whether this was what the REC had in mind in the January letter. 

    (c) The REC meeting on 30 April 2007
    
    

19. The panel noted that the minutes of this meeting record that the Pl attended to discuss the application and that in discussion, the REC felt that an opt out system would be acceptable for data collected at the extra time points. The REC decided at this meeting that the study was service evaluation and therefore did not require review by an NHS REC. 
    

    (d) The May letter
    
    

20. This letter formally communicated the outcome of the REC meeting to the Pl. The panel noted in particular the following points:
    
    

    (i) The reference to the Research Governance Framework setting out the responsibilities and standards applying to work managed within the formal research context;

    (ii) The decision that the project was service evaluation and therefore did not require ethical review by an NHS REC or approval from the NHS Research & Development Office;

    (iii) The opinion of the REC that an opt out system for the collection of data at extra time points would be satisfactory;

    (iv) A statement that the May letter should not be interpreted as giving ethical approval to the project, but that it might be provided to a journal or other body as evidence that ethical approval was not required under NHS research governance arrangements.
    

    The panel's conclusions in respect of the REC review 

21. The panel found that, while there were some inconsistencies in the application, the Pl had done her best both to comply with and to clarify the requirements set out in the January letter. The panel found no evidence, on the basis of the materials supplied to the panel, of any intention to mislead the REC or to bypass ethical considerations. 
    
    
22. The panel agreed with the conclusion of the REC that the study described in the application was service evaluation, finding that the only indications of research were the answer to Section 1 and possibly the references in sections A37 and 38 to publication of the results. The panel noted that, despite the requirement in the January letter to obtain ethical approval, the REC had later accepted that the collection of data for the two extra time points was also for the purpose of service evaluation; that consent was not required for the collection of this data; and that the ability to opt out would be sufficient. 
    
    
23. The panel was however concerned that the May letter did not assist with the interpretation of the statements in the January letter highlighted in paragraphs 14 to 17 above, and that it remained unclear whether the Pl was expected to seek ethical review for the future collection and analysis of data collected in the course of routine clinical practice and anonymised for the NOD, or for the subsequent use of this data for research purposes. The May letter did not specifically address or comment on the process of collecting and anonymising data described in detail in sections A40 to 44 of the ethics application. 
    
    
24. In the absence of any further clarification or evidence to the contrary, the panel found that it was not unreasonable to interpret the response to question (2) and the final paragraph in the January letter as expressing the opinion of the REC that: 
    
    

    (a) anonymised data collected for the purpose of benchmarking services, as required by the national collaborative (by establishing the NOD), could be used in future for research purposes without further ethical review; and

    (b) that this opinion was not confined to the specific data on the CFS/ME service for children covered by the application under reference 07/Q2006/48.

25. The panel went on to consider whether any additional clarification could be obtained by consulting the Research Governance Framework referred to in the May letter.
    

    The Research Governance Framework
    
    

26. At the time of the ethics application in April 2007, the applicable guidance was contained in Governance Arrangements for Research Ethics Committees published in July 2001 (GAfREC 2001).
    
    
27. Sections 1.3, 1.5 and 1.7 of GAfREC 2001 provide that all research falling within certain categories is reviewed independently to ensure it meets the required ethical standards; that before undertaking or hosting any research a favourable opinion must be obtained from an appropriate REC; and that it is the responsibility of the Pl to apply for REC approval. The categories of research requiring advice from the appropriate NHS REC include any research proposal involving patients and users of the NHS. 
    
    
28. The panel found that the Pl had complied with the GAfREC 2001 requirements by submitting her initial written enquiry and application for ethical review, and that any failure to seek further approval for the use in subsequent research studies of data that had been collected as part of routine clinical care and anonymised was understandable, in the light of the opinion expressed in the final paragraph of the January letter, which the panel had concluded was open to this interpretation.
    
    
29. The guidance in GAfREC 2001 was later superseded by Governance Arrangements for Research Ethics Committees: A harmonised edition published in September 2011 (GAfREC 2011). Under GAfREC 2011, certain types of research no longer require REC review. 
    
    
30. Paragraph 6 of the document entitled 'Changes to the remit of Research Ethics Committees (September 2011)' published alongside GAfREC 2001 states that REC review is not required for research limited to the use of previously collected, non-identifiable patient information. This exception also applies to research undertaken by staff within a care team using information previously collected in the course of care for their own patients, provided that data is anonymised or pseudonymised in conducting that research. 
    
    
31. The panel took the view that this provision in GAfREC 2011 effectively codified existing practice, as exemplified in the opinion expressed in the final paragraph of the January letter.
    
    
32. The panel therefore found that, both before and after the publication of GAfREC 2011, it was reasonable for the Pl to conclude that the use of anonymised data collected in the course of routine clinical care (whether or not for inclusion in the NOD) for the purposes of quantitative research did not require separate ethical approval. The panel did not consider that this position was confined to data collected for service evaluation, noting however that the NOD was primarily set up for the benchmarking of CFS/ME services, which was a form of service evaluation. 
    
    
33. The panel also concluded, based on the January letter and GAfREC 2011, that no consent was legally required for the collation and anonymisation of data collected in the course of routine clinical care, although it would have been good practice to inform patients of this intention and of the possibility that the anonymised data might be used in future for research purposes. If at the time of collection of the data there was a current intention to use it for specific research purposes, then consent would normally be required. The panel felt that the final paragraph of the January letter should have been clearer on this point. 
    

    Review of publications 
    
    

34. The panel reviewed each of the eleven publications in turn, in each case considering the following questions:
    
    

    (i) Did the publication describe an activity falling within the definition of research?

    (ii) If so, was the ethics statement appropriate and adequate to cover the research activity described?

    (iii) If not, should additional ethical review have been sought?

    (iv) If additional ethical review was not required, were any corrections to the ethics statements or to other sections of the publication desirable?

    (v) In the event that corrections were recommended, what notifications should be considered?

35. In the ensuing paragraphs of this report, each publication is identified by reference to the numbered list of papers attached to the terms of reference, which includes the full titles and links to the paper and publication details in each case. 
    

    Paper 1 (published 9 November 2008)
    
    

36. The panel noted that the data used for Paper 1 was collected during the period 2004 to 2007 and consisted of self-completed assessment forms on children attending a regional specialist service. The panel found that this was the paediatric service described in the ethics application submitted by the Pl in April 2007 and decided under REC reference 07/Q2006/48. 
    
    
37. The following ethics statement was included in Paper 1:
    

    The North Somerset & South Bristol Research Ethics Committee decided that the collection and analysis of this data was service evaluation and as such did not require ethical review by an NHS research ethics committee or approval from the NHS R&D Office (REC reference number 07/Q2006/48). 

38. The panel did not consider the project described in Paper 1 to be service evaluation. The panel found that the project fell within the definition of research, noting that one aim of the study was to improve care for children with CFS/ME by recognising that school attendance is associated with physical function rather than anxiety. This represented the testing of a hypothesis with the aim of creating new generalisable knowledge. The panel therefore found that the ethics statement was not appropriate or adequate as written. 
    
    
39. However, the panel found that the study involved retrospective analysis of data previously collected in the course of routine clinical care and that its use for later research was covered by the final paragraph of the January letter. The panel noted that the self-assessment questionnaires referred to in Paper 1 were included among those listed in the ethics application decided under REC reference 07/Q2006/48. These questionnaires were also among those used in compiling the NOD. 
    
    
40. The panel therefore concluded that no additional ethical review was required, but that the ethics statement in Paper 1 should be corrected to read as follows (amendments in bold type): 
    
    The North Somerset & South Bristol Research Ethics Committee decided that the original collection and analysis of data from the regional specialist paediatric service was service evaluation and as such did not require ethical review by an NHS research ethics committee or approval from the NHS R&D Office (REC reference number 07 /Q2006/48). In a letter dated 29 January 2007, the Chair of the Research Ethics Committee had previously confirmed (a) that it would not be necessary to apply for ethical permission to use the data being collected as part of service evaluation for the national CFS/ME collaborative and (b) that if in future this data were to be used as part of a research project, this would be agreeable. 
    
    
41. The panel noted that the study was funded by the Lin bury Trust. The panel recommended that the Trust be informed of any corrections to Paper 1 that are notified to the journal editor. 
    

    Paper 2 (published 19 May 2009) 
    
    

42. The panel noted that the data used for Paper 2 was collected during the period 2004 to 2007 and was therefore included in the data collated for the national CFS/ME collaborative. The study looked in detail at anxiety symptoms among children with CFS/ME, compared against a control group of children in the Netherlands, using retrospective analysis of assessment forms completed by children attending a regional specialist service. The panel found that this was the paediatric service described in the ethics application submitted by the Pl in April 2007 and decided under REC reference 07/Q2006/48. The self-assessment questionnaires referred to in Paper 2 were included among those listed in the ethics application decided under REC reference 07/Q2006/48. 
    
    
43. The following ethics statement was included in Paper 2
    The local Research Ethics Committee decided that the collection and analysis of this data were service evaluation and as such did not require ethical review by an NHS research ethics committee or approval from the NHS R&D Office (07/Q2006/48). 
    
    
44. The panel did not consider the project described in Paper 2 to be service evaluation. The panel found that the project fell within the definition of research, with the aim of making generalisable findings. The panel therefore found that the ethics statement was not appropriate or adequate as written. 
    
    
45. As in the case of Paper 1, the panel found that the use for future research purposes of patient data previously obtained in the course of routine clinical care was permitted under the terms of the final paragraph of the January letter. The panel observed that this appeared to represent normal practice prior to the publication of GAfREC 2011, which removed the need for ethical review in these circumstances. 
    
    
46. The panel therefore concluded that no additional ethical review was required, but that the ethics statement in Paper 2 should be corrected to read as follows (amendments in bold type): 
    
    The North Somerset & Bristol Research Ethics Committee decided that the original collection and analysis of this data were service evaluation and as such did not require ethical review by an NHS research ethics committee or approval from the NHS R&D Office (07 /Q2006/48). In a letter dated 29 January 2007, the Chair of the Research Ethics Committee had previously confirmed (a) that it would not be necessary to apply for ethical permission to use the data being collected as part of service evaluation for the national CFS/ME collaborative and (b) that if in future this data were to be used as part of a research project, this would be agreeable. 
    
    
47. The panel noted that part of the salary of the Pl at the time of the study was funded by the Linbury Trust. The panel recommended that the Trust be informed of any corrections to Paper 2 that are notified to the journal editor. 
    

    Paper 3 (published 15 September 2011) 
    
    

48. The panel noted that the data used for Paper 3 was patient-level data collected from adult patients in five NHS CFS/ME services during the period 2006 to 2010 and collated in the NOD. The study involved retrospective analysis of this previously-collected data to assess the impact of CFS/ME on employment and productivity in the UK. Paper 3 explains that the NOD collects assessment and follow up data on patients attending NHS specialist CFS/ME services and that it was set up to enable benchmarking and evaluation of these services. 
    
    
49. The following ethics statement was included in Paper 3:
    
    The North Somerset & South Bristol Research Ethics Committee decided that the collection and analysis of CFS/MS patient data were part of service evaluation and as such did not require ethical review by an NHS Research Ethics Committee or approval by NHS Research & Development Offices (REC reference number 07/Q2006/48). 
    
    
50. The panel concurred with the information about the NOD contained in Paper 3, but did not consider the study described in Paper 3 to be service evaluation. The panel found that the project fell within the definition of research, being an investigation with the aim of creating new knowledge that could be generalised. The panel also noted that the patients were adults from a number of different NHS services around the country and so the use of their clinical assessment data could not be covered by the cited REC reference, which was specific to the collection of paediatric data in a single regional specialist service. The panel therefore found that the ethics statement in Paper 3 was inaccurate and potentially misleading, although the panel did not believe that there was any deliberate intention to mislead. 
    
    
51. For the reasons stated in paragraph 45 above, the panel did not however consider that additional ethical review was required for the retrospective analysis of data collated in the NOD. The panel requested further information from the HRA about the rules for setting up and using the data collated for NODs, but no written guidance appeared to be available. 
    
    
52. The panel recommended that the ethics statement in Paper 3 be deleted and replaced by the following: 
    
    The data used in this study was originally collected and collated in the National Outcomes Database for the purposes of evaluation of CFS/ME services. The collection of a subset of CFS/ME patient data as part of the.national CFS/ME collaborative was confirmed to be service evaluation by the North Somerset & Bristol Research Ethics Committee under REC reference 07 /Q2006/48, and in a letter dated 29 January 2007 the Chair of the Research Ethics Committee had previously confirmed (a) that it would not be necessary to apply for ethical permission to use the data being collected as part of service evaluation for the national CFS/ME collaborative and (b) that if in future this data were to be used as part of a research project, this would be agreeable. 
    
    
53. The panel noted that the study described in Paper 3 was partly funded by Action for ME and that the Pl and another researcher were funded by the National Institute for Health Research (NIHR). The panel recommended that both funding bodies be informed of any corrections to Paper 3 that are notified to the journal editor. 
    

    Paper 4 (published 11 November 2011) 
    
    

54. Paper 4 described a combination of qualitative and quantitative research. The panel noted that the data used for the quantitative element was collected during the period 2006 to 2009, and that the method involved the analysis of assessment forms previously completed by children attending a regional specialist service. The panel found that this was the same paediatric service described in the ethics application submitted by the Pl in April 2007 and decided under REC reference 07/Q2006/48. The self-assessment questionnaires referred to in Paper 4 were included among those listed in the ethics application decided under REC reference 07/Q2006/48. The study also involved qualitative research in the form of patient interviews conducted by the research team. 
    
    
55. The following ethics statement was included in Paper 4: 
    
    Ethical permission for the qualitative part of the study was granted by the North Somerset and South Bristol Research Ethics Committee (REC Reference number 09/H0106/81). The study was also approved by the Research and Development department of the RNHRD. The North Somerset & South Bristol Research Ethics Committee decided that the collection and analysis of this data was service evaluation and as such did not require ethical review by an NHS research ethics committee or approval from the NHS R&D Office (REC reference number 07/Q2006/48). 
    
    
56. The panel did not consider that the quantitative element of the study described in Paper 4 was conducted for the purposes of service evaluation only. The panel found that parts of the quantitative element fell within the definition of research, being an investigation with the aim of identifying factors that might be associated with the time taken to access specialist services and generating recommendations of a generalisable nature. The panel therefore found that the ethics statement was not adequate as written. However, the panel noted that a separate ethical approval of the qualitative element had been obtained and the study as a whole had been approved by the Research and Development Office, therefore only minor amendments were required to the ethics statement in Paper 4. 
    
    
57. The panel therefore concluded that no additional ethical review was necessary, but that the ethics statement in Paper 4 should be corrected to read as follows (amendments in bold type): 
    
    

    Ethical permission for the qualitative part of the study was granted by the North Somerset and South Bristol Research Ethics Committee (REC Reference number 09/H0106/81. The study was also approved by the Research and Development department of the RNHRD. The North Somerset & South Bristol Research Ethics Committee decided that the collection and analysis of this data for service evaluation [words deleted] did not require ethical review by an NHS research ethics committee or approval from the NHS R&D Office (REC reference number 07/Q2006/48). 

58. The panel noted that there was no statement that the study described in Paper 4 was funded, so that only the journal editor need be informed of any corrections made to Paper 4. 
    

    Paper 5 (published 12 December 2011) 
    
    

59. Paper 5 had prompted the initial complaints about inappropriate citing of REC reference number 07/Q2006/48 and complaints had been received about this publication both by the journal editor and by the HRA. The panel noted that Paper 5 reported on a pilot clinical service set up between September 2007 and February 2008 with the school attendance service in Bath to try and improve school attendance. Children with low school attendance were assessed in school clinics and those identified to have CFS/ME were referred to a specialist service. The study compared children identified through school clinics with those referred through primary healthcare services, in terms of both the degree to which they were affected at the time of diagnosis and of the outcome of treatment. 
    
    
60. In a letter to the HRA of 4 July 2018, the University Research Governance Manager explained that the Pl had at the time sought guidance from the Coordinator of the Research Ethics Committee, who had confirmed that the study was service evaluation and did not require ethical review. This was also stated by the Pl in her response to peer review comments of the paper. The panel asked whether there was any documentary evidence of this exchange. Further enquiries of the Pl, the HRA and the REC failed to produce any evidence, but in the absence of evidence to the contrary, the panel accepted the Pl's statement that guidance had been sought from the REC and that advice had been given that no ethical review was necessary. The panel acknowledged that this exchange had taken place over twelve years ago, when electronic records were not as commonly used, and that the absence of records was not surprising after this length of time. However, as a result, the panel concluded that it was unable to confirm how the study had been described by the Pl to the REC Coordinator. 
    
    
61. The following ethics statement was included in Paper 5:
    The clinical service in this study was provided as an outreach from the Bath specialist CFS/ME service. The North Somerset & South Bristol Research Ethics Committee decided that the collection and analysis of data from children and young people seen by the specialist CFS/ME service were part of service evaluation and as such did not require ethical review by an NHS research ethics committee or approval from the NHS R&D Office (REC reference number 07/Q2006/48).
    
    The panel noted that, in an email to the University of 20 April 2018, the Chief Executive of the HRA had assumed that the Pl had quoted REC reference 07 /Q2006/48 in error and that the study described in Paper 5 had been approved under a different REC reference (07/Q2006/103). Having reviewed the correspondence, the panel found that this assumption was not correct and that the study that had been approved under REC reference (07 /Q2006/103) was never carried out. However the panel acknowledged that the two studies appeared strikingly similar in some respects and that the HRA's assumption was not unreasonable. 
    
    
62. The panel found that, although there were clearly elements of service evaluation in the project described by Paper 5, there were also elements suggesting research. The study included an investigation with the aim of identifying factors that might be associated with the time taken to access specialist services, setting out a research hypothesis, and generating recommendations of a generalisable nature. The paper describes differences in characteristic of the populations and outcomes in the two services and observations that emerge from the data from service evaluation. The panel considered that there was in this case a fine line between service evaluation and research, and that it was arguable that the analysis of data with a view to observing whether a school based service was a feasible way to identify CFS/ME sufferers earlier was a form of service evaluation. The panel found that the specialist service referred to in the ethics statement in Paper 5 was the same service in respect of which the January letter and the REC decision under reference 07/Q2006/48 had been issued and where one of the stated aims of the collection of data was to improve paediatric CFS/ME services. 
    
    
63. The panel concluded that the use for research purposes of patient data obtained in the course of routine clinical care was permitted under the terms of the final paragraph of the January letter and (where applicable) under GAfREC 2011. The panel also found that the assessment of children in school clinics described in Paper 5 was for the purpose of diagnosis, with a view to providing clinical care to children earlier than might otherwise have been the case, by identifying potential CFS/ME sufferers through school attendance records. While the panel did not consider that ethical review was necessary for clinical assessment in the outreach service, it would have been good practice for the Pl to inform the children and their parents at the time of the purpose of the in­school assessments and that the outcomes might be published in future. However, on balance, the panel considered that the project described in Paper 5 constituted service delivery and evaluation, for the purpose of improving paediatric CFS/ME services, rather than research. In the absence of any evidence to the contrary, the panel also accepted that the Pl had properly sought guidance in advance from the REC Coordinator as to whether separate ethical approval was required for this study and had been advised that it was not. The panel found that it was reasonable for the Pl to have acted on this advice and that only minor amendments to the ethics statement were required for the purpose of clarification. 
    
    
64. The panel therefore concluded that no additional ethical review was necessary, but that the ethics statement in Paper 5 should be corrected to read as follows (amendments in bold type): 
    
    The clinical service in this study was provided as an outreach from the Bath specialist CFS/ME service. The North Somerset & South Bristol Research Ethics Committee decided that the collection and analysis of data from children and young people seen by the specialist CFS/ME service as part of service evaluation [words deleted] did not require ethical review by an NHS research ethics committee or approval from the NHS R&D Office (REC reference number 07/Q2006/48). 
    
    
65. The panel was concerned that the term 'surveillance' in the title of Paper 5 might be misconstrued by a lay reader as meaning that patients were being covertly monitored by the researchers, noting that references to surveillance had for the most part been removed by the Pl in her written response to pre-publication comments during the peer review process. On the basis that this response had been found acceptable by the journal editor and to the referees, the panel found that the use of the term 'surveillance' was not inappropriate but might have been more clearly and less controversially expressed as 'follow up'. The panel recommended that a change of wording in the title be considered. 
    
    
66. The panel noted that the study was funded by the Linbury Trust. The panel recommended hat the Trust be informed of any corrections to Paper 5 that are notified to the journal editor.
    

    Paper 6 (published 28 March 2013
    
    

67. The panel noted that the study described in Paper 6, as in the case of Paper 3, involved the retrospective analysis of adult clinical data in the NOD collated from several CFS/ME services.
    
    
68. The ethics statement and funding arrangements for Paper 6 were the same as those for Paper 3 and the panel made the same findings and recommendations. 
    

    Paper 7 (published 9 May 2013)
    
    

69. The panel noted that the project described in Paper 7 sought to identify the prevalence of depression in children with CFS/ME and to investigate the relationship between depression and other clinical symptoms in sufferers. The study involved the retrospective analysis of patient assessment data collected by the regional specialist service in the course of providing clinical care between 2005 and 2011, resulting in a finding that depression is a common comorbidity with CFS/ME and a recommendation that patients be screened and treated for depression. 
    
    
70. The following ethics statement was included in Paper 7:
    
    The local Research and Ethics Committee decided that the collection and analysis of this data were service evaluation and as such did not require ethical review by a National Health Service Research Ethics Committee or approval from the National Health Service Research & Development Office (07/Q2006/48). 
    
    
71. The panel found that this study was for the purpose of service evaluation, with the aim of improving services for children with CFS/ME. The panel observed that in Section A7 of the application for ethical review in April 2007, one of the stated purposes of the project described was 'to measure outcomes in children with CFS/ME in order to allow us to evaluate our service' and 'to develop a method for being able to assess whether outcome improves as we improve the service' (see paragraph 18 of this report). This project had been deemed to be service evaluation by the REC under the cited reference 07/Q2006/48 and the panel found that it was reasonable for the Pl to assume that the study in Paper 7 was also service evaluation. However the reference to 'this data' in the ethics statement was potentially misleading and should be corrected, as it was not possible for data collected subsequently to the REC decision to be covered by that decision. 
    
    
72. The panel therefore concluded that no additional ethical review was necessary, but that the ethics statement in Paper 7 should be corrected to read as follows (amendments in bold type): 
    
    

    The North Somerset & South Bristol Research Ethics Committee decided that the collection and analysis of clinical data collected by the regional specialist service for service evaluation [words deleted] did not require ethical review by a National Health Service Research Ethics Committee or approval from the National Health Service Research & Development Office (07/Q2006/48). 

73. The panel noted that the study was funded for service evaluation by the Lin bury Trust and supported by additional funding by Action for ME. The panel recommended that both funding bodies be informed of any corrections to Paper 7 that are notified to the journal editor. 
    

    Paper 8 (published 28 October 2015) 
    
    

74. The panel noted that Paper 8 involved the retrospective analysis of clinical data on adult and child CFS/ME patients collected for the NOD between 2004 and 2014 and the comparison of this data with a clinical cohort in the Netherlands, with the aim of discovering whether there were differences between children and adult sufferers. The paper includes secondary analysis of clinical trial results from the registered FITNET trial, but results are not related to the outcome of the trial. 
    
    
75. The ethics statement contained in Paper 8 was the same as that included in Paper 3. Separate ethical review and approval were obtained in the Netherlands for the Dutch cohort. 
    
    
76. The panel did not consider that the study in England described in Paper 8 was service evaluation as it was designed to test a research hypothesis and produce generalisable findings, but for the reasons stated in respect of Papers 3 and 6, the panel did not consider that further ethical review was required for the retrospective analysis of NOD data for research purposes, as this was covered by last paragraph of the January letter or (where appropriate) by GAfREC 2011. 
    
    
77. The panel recommended that the ethics statement in Paper 8 be deleted and replaced by the following: 
    
    The patient data used in this study were collected as part of routine clinical practice and anonymised for the National Outcomes Database. Under the Governance Arrangements for Research Ethics Committees (September 2011), ethical review is not required for research limited to the use of previously collected, non-identifiable patient information. 
    
    
78. The panel observed that there were several references in Paper 8 to patients being 'recruited' for the study, which implied that the research team had actively sought out the participants. This did not accurately reflect the fact that the patient data analysed for the study was already available in the NOD. The panel recommended that references to 'recruiting' were amended throughout Paper 8 to avoid any possible misunderstanding. 
    
    
79. The panel noted that the study was funded for service evaluation by the Linbury Trust and supported by additional funding from Action for ME. The panel recommended that the funding bodies be informed of any corrections to Paper 8 that are notified to the journal editor. 
    

    Paper 9 (published February 2016) 
    
    

80. The panel noted that, as for Paper 8, this study involved the retrospective analysis of clinical data collected for the NOD (in this case between 2010 and 2013) and the comparison of this data with a clinical cohort in the Netherlands. The study was confined to adult patients and recorded the prevalence of certain symptoms, with a view to identifying symptom-based phenotypes for chronic fatigue syndrome (CFS) sufferers in both cohorts. 
    
    
81. The following ethics statement was included in Paper 9:
    
    The North Somerset & South Bristol Research Ethics Committee decided that the collection and analysis of these CFS patient data did not require ethical review by an NHS Research Ethics Committee or approval by NHS Research & Development Offices (REC reference number 07/Q2006/48). The medical-ethical committee of the Radboud University Nijmegen Medical Centre ruled that the collection and analysis of Dutch CFS patient data did not require ethical review. Dutch CFS patient data were collected as part of routine clinical practice. 
    
    
82. The panel noted that the ethics statement in Paper 9 contained no reference to service evaluation. The panel found that the study in England, which was intended to generate new knowledge about the phenotypes of CFS patients, was research and not service evaluation, and that the decision under REC reference 07/Q2006/48 was not applicable to this project and should not have been cited in the ethics statement. The panel did not however consider that additional ethical review was required for the analysis of patient data collected in the course of routine clinical practice and anonymised for the NOD, as this was permitted under GAfREC 2011. 
    
    
83. The panel recommended that the ethics statement in respect of the UK patient data be deleted and replaced by the following: 
    
    The UK CFS patient data were collected as part of routine clinical practice and anonymised for the National Outcomes Database. Under the Governance Arrangements for Research Ethics Committees (September 2011), ethical review is not required for research limited to the use of previously collected, non-identifiable patient information. 
    
    
84. The panel noted that the Pl was funded during this project by a NIHR Senior Research Fellowship but that this funding was not specific to the study. The panel recommended that any corrections made to the ethics statement in Paper 9 be notified to the journal editor only. 
    

    Paper 10 (published 21 September 2016) 
    
    

85. The panel noted that the project described in Paper 9 involved comparing patient data from UK CFS/ME services collected for the NOD between 2004 and 2014 with data collected in the Avon Longitudinal Study of Parents and Children (ALSPAC), with a view to ascertaining whether there was a higher prevalence of obesity among adolescents who had attended specialist CFS/ME services. The study concluded that there was a higher prevalence of obesity in adolescents who were affected severely enough to be referred to a specialist CFS/ME service. 
    
    
86. The following ethics statement was included in Paper 10:
    

    Ethical approval for the ALSPAC study was obtained from the ALSPAC Ethics and Law Committee (IRB00003312} and the Local Research Ethics Committee. The North Somerset & South Bristol Research Ethics Committee decided that the collection and analysis of CFS/ME specialist service patient data were part of service evaluation and as such did not require ethical review by an NHS research ethics committee or approval from the NHS R&D Office (REC reference number (07/Q2006/48). 

87. The panel found that the study described in Paper 10, which was intended to generate new knowledge and new directions of enquiry about obesity in adolescent CFS/ME patients, was research and not service evaluation. The panel noted in particular, the reference to 'this research' in the funding statement, which confirmed that the study had been awarded a research grant. Therefore the decision under REC reference 07 /Q2006/48 was not applicable to this project and should not have been cited in the ethics statement. The panel did not however consider that additional ethical review was required for the analysis of patient data collected in the course of routine clinical practice and anonymised for the NOD, as this was permitted under GAfREC 2011. 
    
    
88. The panel recommended that the second sentence of the ethics statement in Paper 10 be deleted and replaced by the following: 
    

    The CFS/ME patient data used in this study were collected as part of routine clinical practice and anonymised for the National Outcomes Database. Under the Governance Arrangements for Research Ethics Committees (September 2011), ethical review is not required for research limited to the use of previously collected, non-identifiable patient information. 

89. The panel noted that the Pl was funded during this project by a NIHR Senior Research Fellowship but that this funding was not specific to the study. The study was specifically funded by a Medical Research Council grant and supported through the NIHR Biomedical Research Centre and Unit Funding Scheme. The panel recommended that these bodies be informed of any changes to Paper 10 that are notified to the journal editor. 
    

    Paper 11 (published 19 October 2018) 
    
    

90. The panel noted that the project described in Paper 11 used both quantitative and qualitative methods. The quantitative element involved the retrospective analysis of routinely collected clinical data from a specialist paediatric CFS/ME service in South West England; this was the same service referred to in other papers and in the application for ethical review. The qualitative element involved the analysis of data from a larger qualitative study and individual patient interviews. The aim of the study was to define the minimally clinically important difference of a physical function subscale for paediatric CFS/ME to enable the design and analysis of future clinical trials. 
    
    
91. The ethics statement contained in Paper 11 confirmed that full ethical approval had been obtained for the qualitative element of the study. In respect of the quantitative element, the following statement was provided: 
    
    Quantitative data: The quantitative data was collected as part of service evaluation. The North Somerset & South Bristol Research and Ethics Committee determined that collection and analysis of these CFS/ME patient data did not require ethical review by a National Health Service Research Ethics Committee or approval from the National Health Service Research & Development Office (07/Q2006/48). 
    
    
92. While accepting that the quantitative data had been collected for the purposes of service evaluation, the panel found that the study was research and not service evaluation, so that the decision under REC reference 07/Q2006/48 was not applicable to this project and should not have been cited in the ethics statement. The panel did not however consider that additional ethical review was required for the analysis of patient data collected in the course of routine clinical practice and anonymised for the NOD, as this was permitted under GAfREC 2011. 
    
    
93. The panel recommended that the ethics statement in Paper 11 for the quantitative element be deleted and replaced by the following: 
    Quantitative data: The quantitative data used in this study were collected as part of routine clinical practice and anonymised for the National Outcomes Database. Under the Governance Arrangements for Research Ethics Committees (September 2011}, ethical review is not required for research limited to the use of previously collected, non-identifiable patient information. 
    
    
94. The panel noted that the Pl was funded during this project by an NIHR Senior Research Fellowship but that this funding was not specific to the study. The panel recommended that any corrections made to the ethics statement in Paper 11 be notified to the journal editor only. 
    

    Conclusions and general recommendations 
    
    

95. After a detailed review of the eleven publications, the panel concluded that the ethics statements in every case should be amended but that no additional ethical review was required. The panel's findings and recommendations are set out above and are specific to each of the publications considered. 
    
    
96. In addition, the panel recommended that, if clinical data is still being collected for the NOD, new patients should be informed that the data would be anonymised and collated centrally and might be used in the future for research purposes. The panel noted from correspondence in the bundle of documents that at least one patient whose data had been included in the NOD had raised an objection on the grounds that she had not been informed that such use may be made of it in future. The panel regretted the apparent lack of published guidance for the setting up of a NOD and for the subsequent use of the data collected. 
    
    
97. The panel also recommended that the Pl should be advised not to cite REC reference 07/Q2006/48 in any future publication as this decision was both unclear and out of date and could not be reasonably applied to current or future research. The panel pointed out that, given the dates of publication, some of the research for which this reference had been cited did not in any event require separate ethical review under GAfREC 2011. The panel requested that a copy of this report be provided to the Pl and that she take note of the panel's various suggestions as to best practice for the future, in particular that, even where data collection is for clinical purposes, if it is known at the time of collection that the data will be used for research, then ethical review and patient consent should be sought. 
    
    
98. The findings and recommendations of the panel were unanimous. This report has been approved by all members of the panel for signature on their behalf by the Director of Legal Services. The report and the panel's findings and recommendations will be considered by the Chief Executive of the Health Research Authority and the Pro Vice-Chancellor for Research at the University of Bristol, who will decide on any further action to be taken and determine together how and to whom the outcome of this publications review should be made known. 

Appendix A: Terms of reference

TERMS OF REFERENCE 

Publications review 

The following terms of reference set out the purpose and principles of a publications review, to be carried out by an expert panel jointly commissioned by the Health Research Authority (HRA) and the University of Bristol. 

Purpose 

1.  To read and review eleven publications on chronic fatigue syndrome/myalgic encephalomyelitis (CFS/ME) citing Research Ethics Committee reference 07/Q2006/48, as listed below. 
   
   
2. To determine in each case: 
   

   (a) Whether the collection and analysis of the data referred to in each publication was covered by an NHS Research Ethics Committee ethical review ('ethical review') and favourable opinion, including whether there was any departure from a reviewed project that was not covered by additional ethical review

   (b) Whether the collection and analysis of the data referred to in each publication was part of service evaluation (either under HRA guidance or in line with National Outcomes Database practice), and therefore did not require ethical review

   (c) If not, whether ethical review for the collection and analysis of the data (i) was required at the time of publication and (ii) would be required under current Governance Arrangements for Research Ethics Committees (GAfREC) if published today

   (d) Whether further action should be considered in relation to any publication based on the outcome of this review.

3. At the panel's discretion, to raise questions in writing or to seek assistance or advice from any person or persons that the panel considers to be relevant to its enquiries. 
   
   
4. To make recommendations in respect of any action to be taken where the panel considers that ethical review should have been but was not carried out, including for example, notification to journal editors and funding bodies, correction or withdrawal of publications. 
   
   
5. To submit a written report of the panel's findings and any recommendations to the Chief Executive of the Health Research Authority and to the Pro Vice-Chancellor (Research and Enterprise) of the University of Bristol. 
   

   Principles

6. The panel will have access to all the information that it needs to conduct a fair and objective review of the issues under consideration, including access to current and historic published guidance from the HRA, GAfREC, the Concordat to Support Research Integrity and other relevant sources. 
   
   
7. The panel and any person or persons invited to assist the panel with its enquiries will treat all matters related to the review as strictly confidential. 
   
   
8. It is not within the panel's remit to determine whether there has been misconduct by any person. 
   
   
9. The panel will aim to achieve unanimity but where this is not possible, may make decisions by a majority. 

March 2019

Appendix B: Index of documents

Appendix C: Research Ethics Committee (REC) documents

(a) Letter dated 29 January 2007 from the REC Chair to the Pl;

(b) Application for ethical review dated 1 April 2007 for a study entitled 'What happens to children with CFS/ME? The study of a longitudinal cohort of children who access a paediatric CFS/ME service';

(c) Minutes of the meeting of the REC on 30 April 2007;

(d) Letter dated 1 May 2007 from the REC Chair to the Pl.