Mental Health Scrutiny and Assurance Evidence Review

Methods

A mixed methods approach was adopted, with three stages: a rapid review of the literature, survey of representatives from organisations involved and interested in the scrutiny of metal health services, finally a series of semi-structured interviews with purposively selected respondents.

Literature review

A rapid literature review and narrative synthesis of evidence regarding the scrutiny and assurance of mental health services in Scotland was undertaken. This mapped current arrangements for external scrutiny and assurance, and local governance and assurance practices. The emerging findings from the review informed the qualitative study by shaping the topic guides for stakeholder interviews.

The search strategy incorporated literature from both published peer-reviewed academic research articles, and the grey literature, including formal reports published by public sector organisations and government. Further detail on method is outlined in Appendix 1.

Survey

A brief survey captured the national perspective of current scrutiny of mental health service provision within Scotland.

A structured pro forma (questionnaire) was developed and used to facilitate the collation of responses into a single, purpose-built electronic database prior to return to the contracting team for data analysis and synthesis. The questionnaire was brief to maximise completion rates, while soliciting the range of necessary information. It comprised of two parts, Part A and Part B.

Questions in Part A included the following subject areas:

• The accuracy of findings and information about current local governance and external scrutiny arrangements (as appropriate depending on the respondent)
• Views on the strengths of current local governance and external scrutiny arrangements (as appropriate depending on the respondent)
• Views on the weaknesses and gaps in current local governance and external scrutiny arrangements (as appropriate depending on the respondent)
• Views on how to strengthen current local governance and external scrutiny arrangements (as appropriate depending on the respondent).

Questions in Part B requested information on:

• The implementation process Health Boards/IJBs adopt following recommendations from scrutiny bodies.

Forms closed with an opportunity to share any other information or feedback about scrutiny of services, which may have been relevant for this exercise.

Participants

It was considered important to gather the perspective of scrutiny bodies, Health Boards/IJBs, but also wider views of organisations with an interest in the impact of mental health scrutiny, for example, NHS Education for Scotland (NES) and Voices of Experience (VOX). Participants were selected based on their knowledge and experience of the scrutiny processes and ability to respond to questions posed of senior level management; given their level of accountability and associated responsibilities.

A broad representation of views was gathered from twenty-six responses to the survey, see Table 1. Replies emerged from three special health boards (The State Hospital, NHS 24 and NHS Education for Scotland), nine Health Boards/IJBs (Ayrshire and Arran, Borders, Forth Valley, Greater Glasgow and Clyde, Highland, Lanarkshire, Lothian, Orkney and Shetland), VOX and three scrutiny bodies (MWC, CI and SPSO).

Part A (See Appendix 1) was completed by the Chief Executive/Lead for Health Boards/IJBs and representatives from MWC, SPSO, SIHCG, NES, CI and VOX. Part B (See Appendix 2) was completed by a Director of Nursing and/or Medical Director from Health Boards. Questions solicited a mix of numerical and free text responses.

Scrutiny bodies all commented that they found the survey questions quite restrictive and felt more able to provide information through interview. For this reason, only one completed the scale data; analysis is confined to narrative data for this small group. Members of the SIHCG respectfully declined to submit a response to the survey because many of the group members were invited to participate in interviews as part of the wider research process; their preference was to contribute via interview, in order to provide a more comprehensive response. HIS declined to respond to survey for the same reason, but a number of different members actively engaged in subsequent interviews.

Interviews

A topic guide was developed, it was separately informed by the literature review and early assessment of findings and informal discussion with a member of the represented groups. The contributing group member who assisted with the development of the interview guide was not subsequently invited to interview. Topic guides facilitated an interview of 45-60 minutes.

To address the representation of the views of a range of stakeholders and interested parties, 20 semi-structured qualitative interviews were conducted with a range of professional stakeholders, from a variety of Health Boards/IJBs, see Table 1. This was a purposive sample where participants were selected because of their knowledge on the subject area and ability to answer the questions. The interviews were designed to explore beyond areas of concern highlighted by previous reports and to complement the quantitative elements. The sample size of thirty participants provided adequate information power (Malterud, Siersma and Guassora, 2016) accepted for the purpose of this research method.

A written invitation to interview was extended to all potential participants. Appropriate representation was sought across 50% of the Scottish Health Boards ensuring contributions from North, South, East and West geographical areas, urban and rural locations, taking account of variation in size location and representative of IJBs. A sample of 50% of the population eligible to participate is likely to result in an accurate representation of opinion. Interviews were undertaken via Microsoft Teams. With assent from interviewees, interviews were recorded then shared with the Researcher who facilitated interview transcription.

Ethical considerations

Consent from staff was implied if potential participants returned questionnaires issued to them or agreed to be interviewed. IRAS ethics committee was approached for ethical permission and advised ethical approval was not necessary since all communication was with members of staff and not patients. All data collected within the study was managed in accordance with Data Protection Act (UK Parliament, 1998) and Caldicott Principles (The Caldicott report, 1999).

Analytical Approach

Descriptive statistics were used to report findings from quantitative data, due to the sample size results did not reach statistical significance. Additional narrative/free text supported the scale data. Data is reported according to Health Board/IJBs, presented alongside salient characteristics of the respective organisations. The Strang report indicated that scrutiny was highly variable across individual Health Boards, so the standardised pro forma aimed to aid calculation of key variables, facilitate comparisons across Health Boards/IJBs, and identify good practice and gaps in the current provision. Qualitative data was analysed by three members of the project team, coded in Nvivo using thematic analysis.