Draft National Quality Standards for Substance Misuse Services

Draft National Quality Standards for Substance Misuse Services - Report on Pilot Projects

3 Scottish Drugs Forum Pilot Project Report

BACKGROUND

Following the Scottish Executive's Review of Drug Treatment and Rehabilitation Services an Action Plan was produced in October 2004 recommending the development of national standards with an emphasis on outcomes for service users, together with systems for the monitoring and evaluation of these outcomes.

To date no overarching nationally agreed framework that details the quality of provision and delivery required for substance misuse services exists. As part of the consultation on the Scottish Executive's proposed Draft National Quality Standards For Substance Misuse Services the Scottish Drugs Forum ( SDF) were invited to facilitate a pilot study to test the ability to measure the proposed Quality Standards with Service Users.

In view of the fact that within Glasgow there was already a well established Service User Involvement Group (Glasgow Involvement Group, GIG) comprised of trained volunteers who had experience of peer research work it was agreed to carry out the pilot in the Glasgow area.

The peers undertaking this pilot study were members of GIG who had been trained by SDF to carry out survey work within a variety of settings throughout Glasgow and elsewhere. Prior to this study User Involvement ( UI) group members have carried out 800 interviews in one to one or group settings over a period of five years.

METHODOLOGY

The fieldwork undertaken utilised the peer research method. This innovative methodology marks a departure from 'top down' approaches to ensure that those with little or no influence acquire the capacity to develop opinions and effect change.

Data collection

The fieldwork took place over a five week period between April and May 2006 at five services across the Glasgow area with the interviews being conducted within the service premises. Structured interviews were carried out with 40 service users using a questionnaire developed by GIG in conjunction with SDF's Regional Manager (see appendix 1).

Confidentiality

All respondents were provided with a study information sheet. Written consent was agreed prior to the interview. No information is used in this report that could identify any individual or service.

Processing

On completion of the fieldwork the GIG volunteers processed the questionnaire data using SPSS. To ensure that the data had been accurately inputted a sample of data entry was checked by the SDF Research Officer. The data was analysed by the SDF Research Officer using SPSS v14.

Study Process

Participants

Managers of five services across Glasgow agreed to be part of the pilot study. This included both residential and community services. It is important to highlight that the purpose of the study was to test the measurability of the Draft National Quality Standards and their set of underpinning statements not to audit the five participating services themselves. In total 63% of participants in the study came from community based services with the remaining 37% from residential services.

Questionnaire

The questionnaire mirrored the eleven Quality Standards Statements. In analysing the content of the questionnaire the GIG researchers had some concerns about aspects of the language used specifically whether it was clear enough in terms of outcomes. However given that the questionnaire used the terms from the National Draft Quality Standards it was agreed that to change it would lead to a pilot study which was not evaluating the Draft National Quality Standards. The group of peer researchers felt that as page 1 of the consultation document stated, "In common with the National Care Standards, the Quality Standards have also been written from the point of view of the service user", that the pilot study should test this.

Data collection

There was a degree of concern from staff at the pilot sites that service users were expected to become involved in another evaluation, nonetheless they showed willingness to encourage their service users to participate. It was evident in some services that frontline staff had no awareness of the Draft National Quality Standards and it was left to volunteer peer researchers to inform and supply staff with copies of the Draft National Quality Standards document.

Over a period of five weeks volunteer peer researchers aimed to conduct 10 interviews in each service. In total 40 Service User interviews were completed across the five services. This was ten short of the target number. This discrepancy occurred for a variety of reasons including two residential services not working to their full capacity and one community project which selected service users to be interviewed, allocated them an appointment time which had a 50% default.

A difficulty in the process for the peer researchers was a lack of clarity amongst frontline staff as to why the pilot was taking place even though projects all volunteered to be involved and had been given Capacity Building Support from the SDF Capacity Building Officer prior to peer researchers accessing projects.

Results

For each quality standard statement quantitative results are provided with the data from the five services collapsed for ease of analysis and to protect confidentiality. Qualitative results in the form of direct quotes from service users will also be provided.

Quantitative Analysis

Standard Statement 1 "You will be provided with all the information you need to help you decide about using the service".

The majority of respondents (92%/n37) stated they were given all information required to assist their decision to attend a service.
30% (n12) though were not given details re 'making a complaint'

Standard Statement 2 "You will be able to access safe, quality surroundings when engaging with the service".

Over 82% (n33) agreed with standard statement 2. Service users said the facilities at the service were of a good standard, staff treated them with respect, bullying is not tolerated and the services actively supported their religious and cultural beliefs.

Standard Statement 3 "You will be involved in a full assessment which makes sure that decisions about your care and support are based on your needs".

Over 85% (n34) agreed on points 1 -3 of the standard statement.
In relation to point 4 of the standard statement 70% (n28) of respondents felt they were given information on how the assessment would be shared with other agencies but only 57% (n23) were aware that assessment information might be shared without the clients' permission.

Standard Statement 4 "You will receive a written personal plan that clearly sets out the service that will be provided to meet your needs".

67% (n27) had or were about to receive a personal care plan but 87% (n35) saying they were involved in developing a care plan. This discrepancy requires further clarity but it could indicate in itself that service users may not be fully aware of what a care plan is or that service users may have helped to develop a care plan but did not receive a formal written copy of said plan. It was out with the scope of this questionnaire to explore this issue further.
77% (n31) of respondents said their care plan was reviewed as their situations changed.
80% (n32) of respondents were clear on what services they would receive and who would deliver them.

Standard Statement 5 "The service will work with you to achieve jointly agreed actions in your personal plan".

Over 82% provided positive responses to standard statement 5 (n33). Indeed 97% (n39) said they worked jointly with the service to address their needs.

Standard Statement 6 "You will be able to discuss and plan your long-term support with service staff, involving your family, other organisations, services or representatives as appropriate".

82% (n33) of respondents said positive relationships were being maintained whilst in services.
85% (n34) said they had support to achieve long-term goals.
73% said there were plans to re-establish positive contacts in lead up to leaving service; 10% said no; 17% of respondents said they didn't know.

Standards Statement 7 "You will receive quality support and care".

80% (n32) said other services were used when the service could not meet their needs and 87% (n35) felt service workers were providing quality support and care.
25% (n10) said they were not asked by services about the quality of support and care that was offered although 75% (n30) said they were regularly asked about how the service can be improved.

Standard Statement 8 "The service will work with a wide range of partners, including other services, so that your needs are met".

90% (n36) said the services used the best approaches to meet their needs.
77% (n31) thought services met their specific needs. Of those who did not when asked ' were other services accessed…?' 67% said yes.
67% (n27) said other relevant agencies were consulted.

Standard Statement 9 "The service you receive has been designed with you, your family and the needs of the local community in mind".

Over 82% (n33) felt they were treated fairly & equally and that all services involved in a personal plan were working together but 38% (n15) said the needs of their families and those they lived with were not recognized and only 50% (n20) said support was sought for their families etc.

Standard Statement 10 "Your views will be sought in order to constantly monitor the type, delivery and development of services".

75% (n30) of the sample agreed they were given a list of their right & responsibilities.
75% (n30) said their views on the service were asked at least once per year and 77% (n31) said they had been offered training & support to establish a service user group.

Standard Statement 11 "You will have all the information you need about arrangements for ending your contact with the service".

(This information may be unknown, please complete where possible regarding the current service.)

40% (n16) of respondents said they had information about ending contact with a service.
40% (n16) said the needs of their children were considered when ending contact with a service.
45% (n18) said they had been told what agencies would be informed upon their leaving.
65% (n26) said the service talks to them about choices and risks.
72% (n29) are advised on how to cope with risky situations.
35% (n14) said they knew their views on the service would be asked when leaving a service.
37% (n15) said they knew services send progress reports to other agencies.
57% (n23) said the service asked them to keep in contact.
82% (n33) said they would be able to contact the service after leaving.

Qualitative analysis

Six people said the questionnaire was straightforward and could be understood however three people said others may have difficulty understanding the standards. A typical statement regarding understanding was ' I understand what the statements mean but some people would need the statement broken down into a language you could understand.'
Four people found the questionnaire ' Awkward', ' repetitive' and ' contradictory'.
A key observational finding to emerge from the peer researchers was the fact that while the results were overwhelmingly positive in terms of being able to measure the quality standards, the peer researchers felt that respondents were not always fully understanding of the quality standards outlined. Specifically the quality standards are not precise enough which may lead to differences in interpretation.

Summary of results

The aim of the study was to ascertain whether the Draft National Quality Standards are measurable in their current form. Based on interviews with forty service users the data provided suggests that the standards are quantifiable. However, two caveats to this interpretation have to be highlighted:

First, further clarity is required as to whether the outcomes are specific enough for service users to fully understand what they can expect from a service and whether the language or terms used in the standards is clear enough for all service users.
Second and relating to the previous point, the forty service users who participated in the study are not necessarily representative of all service users so the reliability of the results requires further investigation.

Recommendations

Considering both the process and findings from the pilot study, the following recommendations can be made:

Quality Standards must be service user friendly. Therefore any format that they are delivered in should be accessible and presented in such a way that they can be easily understood by all service users and avoid repetition and ambiguity.
Frontline staff need to be fully aware of developments around Quality Standards at national and local level.

Services should ensure service users are informed of complaints procedures and processes and that these are delivered in a non-discriminatory way.
Services must ensure that service users are aware information relating to them may be shared with other providers.
Service Users should be provided with a formal written care plan which they have been fully involved in developing. Any changes arising during their treatment should be agreed with the service user and a revised care plan should be issued.
Services need to be proactive in seeking service user views on service provision/delivery. This could be sought through a comments or suggestion box as is the case in some services. Additionally a more proactive approach could be adopted at the point of treatment.
Increase/improve family support through on site provision or signposting to local or national support groups.

Appendix 1
Quality Standards for Substance Misuse Services in Scotland Glasgow Pilot