« Previous | Contents | Next »
Listen
EXECUTIVE SUMMARY
Introduction
Background
Central to Delivering for Mental Health ( DfMH) (2006) is the promotion of a well-being and recovery-based mental health service model, Commitment One of DfMH states:
'We will develop a tool to assess the degree to which organisations and programmes meet our expectations in respect of equality, social inclusion, recovery and rights. The tool will be piloted in 2007 and be in general use by 2010.'
This evaluation is concerned with the Scottish Recovery Indicator ( SRI). The SRI enables the assessment of whether and how organisations, and those who work in them, are ensuring that individuals who use their services and their carers' needs are met in terms of their rights to equality, social inclusion and support to recover. The SRI is intended to assist the successful implementation of Commitment One. The SRI is primarily a developmental tool and its key elements are:
- Meeting basic needs
- Personalisation and choice
- Strengths-based approach
- Comprehensive service
- Service-user involvement/participation
- Involving support networks and promoting social inclusion and community integration
- Service user in control and active participant even when subject to compulsion
- Recovery focus
The SRI data is collected from a range of sources, including: assessments and care plans, service information, policies and procedures and interviews with service providers and service users. Under the DfMH Leadership Programme Leading Change, four health board areas included recovery and social-inclusion focused elements within their Leading Change projects and along with two other interested areas agreed to pilot the SRI of whom one did so.
Evaluation aim and objectives
The overarching aim of the evaluation is to assess if the SRI results in real impacts in terms of the implementation of DfMH. The evaluation objectives are to:
- Assess the relevance and appropriateness of the tool to a variety of settings
- Identify what preparation organisations and individuals need before using the tool
- Identify how best the tool should be used ( e.g. who to involve in gathering and providing information)
- Identify whether the tool may be able to measure changes in services
- Identify the potential for the use of SRI as a means of promoting change
- Assess whether service users and carers feel added benefit from being involved in the piloting of SRI
Methods
The evaluation explored and reviewed the developmental process of the SRI from the perspectives of all of the groups of individuals involved. The methods involved 4 stages.
Stage 1: Preparation
Stage one included local pilot site group discussions with local pilot site leads and other pilot participants to introduce the evaluation and gather contextual information and the development of a detailed database map of the planned and actual use of the SRI during the pilot.
Stage 2: Local case studies
Stage 2 involved documentary analysis of a maximum of 3 completed SRIs per pilot area, interviews with SRI administrators, service-user and service-provider group participants, a review of any action planning documentation and focus groups with those involved in action planning/implementing change.
Stage 3: Analysis
The analysis brought together the mapping and interview data to allow a full analysis of the use of the SRI within the range of settings provided by the pilot sites in order to provide evidence-based, realistic and thorough guidance for the effective use of SRI within mental health services. Qualitative data was analysed using a staged content process of identifying themes. Quantitative data from SRIs was cross-tabulated to compare scoring between the elements of the SRI and across pilot sites and settings.
Results
Potential of the tool as facilitator for change
The SRI appears to have good potential to influence change. The evaluation demonstrated how the SRI results can challenge service cultures and point to changes that can be made to promote a stronger recovery orientation.
A key strength of the SRI was found to be in its level of detail, which made it possible to pinpoint both areas of good practice and areas for improvement and which provided a structure for identifying what should change and how. However, evaluation participants were confused by the scoring of the SRI and felt that there was limited value in the summary scores.
Administering the SRI was time-consuming but participants accepted that the resource input was necessary and worthwhile especially when the SRI was part of a development process. Although considered to be very relevant by most evaluation participants, the SRI requires a thorough review to improve the clarity, quality and comprehensiveness of the content and to prepare guidelines for administration.
The SRI seemed to enhance local efforts to improve services by linking in well to current service development plans and recovery initiatives within local areas, and further developing local participants' insights into the recovery, equalities and inclusion agenda.
Participants were able to evidence changes already put in place and these were very much focused on activities that would promote social inclusion, equality and recovery, ranging from changing to strengths-based care documentation and procedures, to addressing the issue of ensuring equal employment opportunities for people with mental health problems
The evaluation demonstrated the value of the SRI process as much as its output. Participation in the SRI pilot brought with it a number of added benefits. The SRI created a sense amongst the service users that the SRI places those who use services at the centre of the change process and provides them with motivation to move forward in their own recovery. The tool helped professional and service-user team-working and embedded recovery concepts and language into everyday practice and communication, whilst providing a values-based framework within which to assess how the service system facilitates recovery by encompassing policy, services and practice.
Most pilot sites expressed an intention to re-administer the SRI within a year with the belief that they will be able to measure and demonstrate improvements in their practice.
Completeness and relevance of the SRI
The SRI was considered by most participants to be sensitive to the recovery, equality and social inclusion needs of the individuals and groups who use the range of services to which the tool was applied. However, evaluation participants, particularly those in the pilot area that focused on the inequalities agenda, were not completely convinced of the SRI's contribution towards identifying the extent to which services were identifying and addressing inequalities. There may be a need for further work to develop the SRI's potential on this.
The pilot demonstrated that the SRI can and should be used within and across all services (statutory and non-statutory) for mental health, although some work is required to make the tool less health and medically focused. The pilot has also revealed that the SRI is flexible enough to be used in a number of different ways, from a benchmarking tool or an impact assessment tool for new services, to a reference for everyday supervision. How the tool is used in the long term and how often, may be better dictated by local development needs rather than prescribed nationally.
Recommendations for the development and roll-out of SRI
The evaluation resulted in a number of detailed recommendations for those responsible for the development and roll-out of the SRI, including:
- Ensuring an inclusive approach to developing and rolling out the SRI, including support and learning networks and information on how SRI fits with other initiatives that promote recovery orientation
- Employing strategies to achieve and sustain buy-in to the SRI process, e.g. strong strategic commitment, creating clear lines of accountability and/or formal approaches to the use of SRI, producing national-level supporting documentation, building SRI in to local strategies on recovery and operational procedures, good project management and a non-judgemental and supportive ethos
- Improving SRI preparation days to increase the time available to cover the extensive ground, be responsive to the needs of different individuals, provide more on the background and evidence behind the recovery concept and on the practicalities of the administration process, including the teaching of sampling and data collection skills
- Supporting SRI implementation by local recovery awareness sessions prior to the administration
- Improving the content of SRI using the evaluation results to revise the content of specific questions and each whole part of the SRI. Sections 4 and 5 in particular needs to be simplified and reviewed to weed out any duplication. A crucial aspect of the revision of the SRI content should be adaptations to enable wider service setting applicability of SRI. The language should be less healthcare-focused and more generic to enable those in non-healthcare services to use the SRI.
- Involving service users in a number of ways including as interview participants, administrators, scorers and action planners. Consideration should be given to paying service users for their involvement in the SRI development process, especially as administrators, scorers and action planners.
- Giving consideration to using different people to administer different parts of the SRI and to work in administration teams
- Providing further guidance on the administration process, particularly sampling, recruitment and data sourcing
- Service-user interviews should be supported with prior provision of information, conducted in groups, less than one hour long, have a familiar interviewer and venue, support practical needs and allow carers or representatives to attend
- Service-provider interviews should be supported by provision of information on the purpose and content of SRI and implications of participation prior to the interview, recruited-for through open invitation to participate, held with multi-disciplinary teams with management representation
- Providing improved guidance on scoring methods
- Undertaking scoring as a group effort soon after administration with additional notes being made in the SRI during the administration and consulted during SRI scoring, allowing further investigation or follow-up of issues as required
- Providing forums for those leading SRI locally to network and share experience, not only in relation to SRI administration but more importantly in relation to service and system change (feedback and action planning templates could be produced nationally)
Those involved in the roll-out of the SRI need to address the key challenges and barriers to use of SRI by:
- Committing further attention to acknowledging the contribution to the recovery agenda of psychiatry and involving psychiatry in the future roll-out of SRI
- Challenging the persisting perception from some nursing professionals on acute wards working with those under MHA and those with dementia, who consider that aspects of the recovery agenda such as occupation or shelter are not relevant to their service
- Giving consideration to engaging foreign language and hearing interpreters to assist administration of the SRI
« Previous | Contents | Next »