< Previous | Contents | Next >

EFFECTIVE AND COST-EFFECTIVE MEASURES TO REDUCE ALCOHOL MISUSE IN SCOTLAND: A LITERATURE REVIEW

CHAPTER TWELVE EVALUATIING INTERVENTIONS IN SCOTLAND

Introduction

12.1 The purpose of this chapter is to describe some of the interventions being carried out in Scotland and the extent to which they have or have not been evaluated. These examples will be used to draw lessons about improvements in the evaluation framework and the process of knowledge management. The topic coverage is representative rather than comprehensive.

Evaluations in Scotland

12.2 As a parallel exercise to the main literature review, health promotion specialists and local alcohol advisory committees were contacted in an effort to identify interventions taking place in Scotland that had been evaluated. Eight organisations provided positive responses, some including more than one initiative. The types of intervention reported were:

• Designated Driver Scheme
• Befriending Services
• GP Alcohol Counselling Services
• Provision of Complementary Therapy
• Rough Sleepers Initiative/Counselling for Young Women
• Home detoxification services
• Young Scot Card
• Community School initiative

12.3 Other interventions in Scotland that were known about included server intervention programmes, nurse based minimal intervention initiatives, designated places and the teenwise alcohol project (TAP). A further limited search was undertaken to try to identify other research reports, published or unpublished, relating to such interventions. Some examples of the evaluations undertaken follow and these cover most of the categories of intervention reported in previous chapters, although it is notable that no examples of policy evaluations were identified. Treatment studies, such as drug trials, have not been included as these are well covered by the effectiveness reviews and there are not thought to be distinctive messages from Scottish based studies.

Enforcement

Designated Driver Scheme

12.4 Dumfries and Galloway introduced a designated driver scheme, starting with a pilot in 1992 (Cawte 1995). The primary aim was to evaluate the viability of the scheme, in which a designated driver with 2 or more passengers would receive free soft drinks in participating licensed premises. The pilot scheme involved 19 licensees and over 160 drivers took part. For the main scheme, 156 licensed premises stratified by category and selected at random were approached. 64 premises agreed to participate. The evaluation concentrated on reactions from licensees, awareness of the scheme amongst customers and attitudes of customers towards drinking and drink driving. Information was collected through questionnaires and interviews. In addition, licensees were asked to record the number of free drinks provided each month and unannounced visits were made to observe the use of the scheme in participating premises. A number of recommendations were made including the need for incentives for licensees, who bore the cost of the free drinks, and for the promotion of such schemes in the future.

Teenwise Alcohol Projects

12.5 The Teenwise Alcohol Projects (TAP) were a series of police led, community based initiatives to tackle under-age drinking (Anderson and Sawyer 1999). The evaluation was concerned with the implementation and impact of the campaign. Views were sought from young people, their parents, the police and staff in the licensed trade and the focus of the research included the nature of the problem as well as responses to the problem. Questionnaires and interviews were used with young people to obtain information on drinking behaviour. Whilst there was general support for interventions on under-age drinking from parents and licensees, young people thought that they should be left alone if they were not causing trouble. There was perceived to be a reduction in public drinking by young people but it was unclear whether this could be attributed to the TAP intervention. Both local bye-laws and national legislation came into effect at the same time. The evaluation was not designed to identify whether under-age drinking had reduced or had been displaced to other locations.

Prevention

Alcohol education initiatives

12.6 A review of alcohol education initiatives in Scotland was published in 1996 (Bagnall and Fossey 1996). The report included discussion of:

• school-based initiatives, using materials developed by the Scottish Health Education Group and the Health Education Board Scotland, with some exploratory work on delivery methods;
• workplace initiatives by the Scottish Council on Alcohol and the development of alcohol policies for the workplace, training schemes and an awards scheme sponsored by drinks industry;
• the Grampian Server Training Initiative started in 1992 (Server Intervention Prevention Strategy SIPS) targeting bar staff and providing training on licensing issues and good service practice;
• community interventions such as designated driver schemes and Drinkwise campaigns; and
• primary care interventions, such as training for GPs to provide skills to intervene opportunistically (Drinking Reasonably and Moderately with Self-control DRAMS).

12.7 The review was not specifically intended to consider effectiveness evidence, although evaluation is discussed. The authors draw attention to published systematic reviews in the area, which are dominated by US literature because few UK studies meet the inclusion criteria on design and evaluation. Although the need to incorporate evaluation into the design and implementation of alcohol education initiatives was increasingly recognised, this was generally a secondary concern. The resulting evaluations were often short term, as was the funding of the initiative itself. The need for more rigorous evaluation studies to be funded, with longer follow-up periods and at least quasi-experimental designs, was emphasised.

Brief interventions

Health visitor scheme

12.8 This brief intervention was undertaken as part of a programme of innovative alcohol services with evaluation funded by the Chief Scientist Office. Health visitors were trained and then supported in screening women in their existing caseload, who had agreed to participate in the study, and in delivering a minimal intervention to those drinking more than 14 units per week (Scott 2000). Screening was by means of a 7 day drinking diary and 13% of the sample of 430 were found to be drinking above the recommended level. Outcomes were measured by 7 day drinking diaries at 6 months following the intervention. At this point, 92% of the sample were found to have reduced their weekly alcohol consumption. Those still drinking above the recommended level could receive up to two further interventions.

12.9 As evidence already existed about the effectiveness of brief interventions, the issue raised in this study was the success of health visitors in undertaking the intervention. The intervention undoubtedly had an effect with the women recruited and the intervention process is well described. The high success rates, which are greater than those generally reported in the literature, may be partly explained by the probable exclusion of women who were uninterested or unwilling to participate. Also, part of the effect may be achieved by simply asking about drinking behaviour without any intervention. This would be consistent with findings from controlled studies.

Counselling services

12.10 Two areas in Scotland report having evaluated counselling services provided by voluntary organisations working in primary care settings. The service in Forth Valley began in 1997 and was evaluated in its third year of operation (Martinus et al 2001). 349 people had been referred to the service and 81% had kept their initial appointment. Reported drinking fell from a weekly average of 112 units to 19 units by the fourth week of counselling.

12.11 The average number of sessions per patient was 10 (Martinus et al 2000a) and the estimated cost per patient was £420. This number of sessions indicates that the intervention would be classed as more than a brief intervention. Although drinking levels were recorded, alcohol dependency was not assessed. Therefore, it is not possible to judge whether the study subjects were unsuitable for a less intensive intervention.

12.12 A similar service was set up in 1995 in Fife and was evaluated in 1997 (Fife Alcohol Advisory Service). Over 11 months, a total of 257 patients were offered counselling in one of 9 health centres and the take up was 81%. The average alcohol consumption of patients starting counselling was 142 units per week. The service was in the process of developing its records and information from a 1^st progress review was available for 62 patients; their reported alcohol consumption fell from 129 units per week at assessment to 19 units per week.

12.13 The objectives of the Fife service were stated in terms such as providing better access to alcohol services, widening the range of services and improving the working together of the voluntary sector and primary care. The evaluation that took place was directed at these issues (Centre for Health and Social Research). Interviews with primary health care staff and counsellors found considerable support for the counselling service but suggested that further efforts were required in terms of service integration.

Complementary therapy

12.14 The provision of complementary therapies (CT) as an adjunct to counselling was introduced in Clydebank in 1997. Participants had chosen to receive CT and may have been better motivated than the non CT control group. However, comparisons were also possible between those who received CT and those placed on a waiting list because of excess demand (McMahon 1998). The numbers available for analysis were small (25 received CT, 18 on the waiting list WCT, and 14 declined NCT). The CT group did achieve the greatest reduction in weekly alcohol consumption at 6 weeks but the difference was not statistically significant when compared with the NCT group, probably because of the small numbers. Interestingly, the WCT group did less well than either the CT or NCT groups. The results for the WCT did improve when reassessed after receiving CT but not by very much. This suggests that the timing of providing CT may be important.

12.15 In Forth Valley, the Complementary Therapies Project provided access to a range of complementary therapies for people already receiving treatment for substance misuse from either the Central Scotland Council on Alcohol or the Community Alcohol and Drugs Service (Martinus et al 2000b). The numbers available for evaluation were very small and results are not specific for individual therapies. Only qualitative effects on drug and alcohol use are reported.

Detoxification

Home detoxification

12.16 A home-based detoxification service was introduced in Ayrshire and Arran in 1995 to offer an alternative approach for suitable patients (Stark et al 1999a). The service would not be offered if home circumstances were unsuitable or if there were medical contraindications. Referrals in the first year were 747, of which 559 were for alcohol. Of the alcohol cases, 57% completed detoxification and follow-up. Substance use at discharge showed 13% with controlled use of alcohol and 57% abstinent.

12.17 There was no control group but a no treatment group would not be feasible in studies of detoxification. The outcomes could have been compared with inpatient treatment but patient characteristics would differ. The accepted spontaneous remission rate in the literature is one third and this service achieved rates which were double this for patients who accepted treatment. No longer term follow-up was reported.

12.18 Other areas have also reported providing home detoxification services. The first such service was established in 1991 to relieve demands on inpatient facilities (Bennie 1998). A comparison was carried out between the home detoxification service and a minimal intervention treatment strategy, with random allocation of referrals. At 6 months both groups showed some improvement with respect to drinking behaviour, alcohol related problems and use of other services. Patients in the home detoxification group remained abstinent twice as long after treatment as those in the minimal intervention group. Another study has shown both home detoxification and day hospital treatment to be viable alternatives to inpatient treatment Allan et al 2000). Home detoxification is often suggested as a more suitable service for meeting the needs of women but no difference in the gender mix between inpatient and home based services has been found (Madden et al 2000).

Critical appraisal

12.19 What emerges very clearly from the studies described above is that the simple term ‘evaluation’ has a host of meanings. The aims of the evaluations carried out were very disparate and they were not primarily concerned with the effectiveness of the services in most cases. Those reports that did address the issue of effectiveness were necessarily limited by small numbers and the adequacy of controls. Nevertheless, these studies mainly had a clear objective and an appropriate research design. They could provide useful information for service planning where effectiveness has been confirmed by peer-reviewed research evidence. In some cases, the content of the service being provided may need to be compared with that of the intervention evaluated in the literature.

12.20 Some of the evaluations had less clearly focussed objectives and could be improved by a clearer statement of the underlying hypothesis. Other common problems include the lack of comparison groups, the need for longer follow up periods and independent confirmation of drinking status. These issues are recognised by some of the authors and research designs are frequently constrained by what is possible. The same criticisms can be levelled at much of the wider literature. Well-conducted qualitative research has a role to play in exploring underlying issues and developing innovative services and this also requires a robust design.

Evaluation guidelines

12.21 Essential elements of evaluation do vary with the purpose of the evaluation. Here attention is concentrated on two types. Evaluation to find out if an intervention works and evaluation of the local implementation of an intervention that has been shown to be effective in other studies. Some basic principles are addressed but more detailed evaluation guidelines have been developed by other bodies (Health Education Board for Scotland (Evaluation Toolkit) and Scottish Executive Effective Interventions Unit).

12.22 Effectiveness evaluations require:

• a clear definition of the aims and objectives of the intervention;
• an appropriate outcome and how it is to be measured;
• a control or comparison group is required with a robust research design to eliminate bias in the results. Randomisation is the gold standard but reliable results can be obtained from well-conducted before and after or case-control studies; and
• sufficient numbers for statistically valid results and a longer term follow up.

The last point is possibly the most difficult to achieve and it would be better for effectiveness evaluations to be concentrated on a smaller number of studies of adequate size and duration. The need to include independent measures of outcome, where possible, also has implications for the resources required for evaluations. Sample sizes need to be calculated with respect to the difference between groups that is to be detected. Considering issues of cost-effectiveness can increase the value of evaluations or at least providing details of the resources employed in the intervention.

12.23 When a service is implementing an intervention shown to be effective in the literature, the main purpose of the evaluation is to ensure that the results achieved elsewhere are transferred. Such evaluations should;

• document any differences in the local implementation of the intervention or in the intervention population;
• measure or monitor outcomes against the benchmark provided by the literature; and
• provide information on the resources employed in the intervention.

The benchmarking of outcomes may need to allow for fact that effectiveness rates are often based on ‘volunteer’ populations and may not be directly applicable to general populations.

Accessibility of findings

12.24 Research findings should be accessible if they are to be of value in shaping the development of services and avoiding duplication of effort. The examples discussed in this chapter have been obtained from a variety of sources, including personal contact with authors. They cover research conducted by or for government departments, health services, voluntary organisations and academic institutions. There is no comprehensive source of such research reports, particularly those that are unpublished, and there will be further examples not uncovered by the searches undertaken for this report. This situation could be improved by the development of a searchable Scottish database of research findings. The value of such a database would be enhanced if it were possible to assess the quality and generalisability of results.

Conclusions

12.25 There has been considerable activity in Scotland in implementing initiatives to reduce alcohol misuse and its consequences. Not all of these have been subject to rigorous evaluation and where evaluation has been carried out the results are not always readily accessible. The most useful findings reported here relate to brief interventions by health visitors and home detoxification services. Counselling services are clearly effective but research is required to establish whether the current pattern of provision is more effective than a briefer intervention. The role of complementary therapy requires larger studies to be carried out. A culture of evaluation that is focussed on outcomes needs to be fostered at all levels from policy making to service delivery.

< Previous | Contents | Next >