< Previous | Contents | Next >

EFFECTIVE AND COST-EFFECTIVE MEASURES TO REDUCE ALCOHOL MISUSE IN SCOTLAND: A LITERATURE REVIEW

CHAPTER TWO REVIEW OF EFFECTIVENESS REVIEWS: METHODS

Introduction

2.1 There is an extensive literature, of variable quality, on the effectiveness of interventions to reduce alcohol misuse. A number of reviews of the effectiveness of interventions have been carried out. These include some good quality meta-analyses of evidence from well-conducted trials of treatment interventions and some more general reviews from authoritative sources. Given the volume of literature, it would not have been possible to review all of the individual studies within the alcohol misuse area in the time available for the study. It was therefore decided to base this part of the study on a review of existing reviews of effectiveness.

Methods

Search Strategy

2.2 The search of electronic databases included MEDLINE, EMBASE, HealthSTAR, ISI Databases (Science & Social Citation Indexes), PsychINFO (on line version of PsychLIT), International Bibliography of the Social Sciences (IBSS), Cumulative Index of Nursing & Allied Health Literature (CINAHL), International Pharmaceutical Abstracts, Health Management Information Consortium (HMIC), EconLIT, SIGLE (System for Grey Literature in Europe) and The Cochrane Library.

2.3 Details of the search strategies used are given in Annex 1. The search for reviews in relation to alcohol misuse used the high sensitivity, low precision strategy developed by the NHS CRD for the identification of reviews and meta-analysis (Boynton et al, 1998). This involves identifying a wide range of references that can be checked against inclusion criteria, rather than using a strategy that is more precise in the references included but risks excluding relevant material. Studies were included if they had carried out a review and this included specific interventions.

2.4 In addition, some studies were reviewed which dealt with issues such as provider roles or settings. Even if these did not report specific comparative interventions, these were retained and reported separately. Finally, individual studies of interventions implemented in Scotland, published or unpublished, were identified through direct contact with local services. These are reported as service examples rather than as evaluations. The time-scale of the study did not permit further hand searching or follow up of references from the retrieved literature.

Quality Assessment

2.5 The gold standard for conducting systematic reviews of effectiveness in the clinical sphere has been set by the Cochrane Collaboration. Reviews that appear in The Cochrane Library have been conducted to rigorous standards and will identify the quality of the studies reviewed. The review process adopted by The Cochrane Library involves, wherever possible, checking details of the original studies with the authors, rather than relying solely on the information reported in the published source. This would not have been feasible in the time scale available for the current review. The strict standards applied by the Cochrane Collaboration, however, would have excluded a large number of reviews that may contain relevant results.

2.6 As indicated above, the search strategy was designed to identify all reviews of interventions to reduce alcohol misuse. The quality of the review process was assessed in terms of whether the authors reported that they had:

• carried out a systematic search for literature;
• applied clear inclusion and exclusion criteria; and
• assessed the quality of the studies included.

Effectiveness Measures

2.7 An important omission in the majority of studies is the lack of any generic measures of health outcome. Given the range of interventions covered by this report, this means that it will not be possible to report uniform measures of effectiveness across all areas. Indeed, different measures of effectiveness are used within the literature on single interventions. The outcomes of interventions may be reported in terms of their impact on alcohol consumption or their impact on harms associated with alcohol consumption. Changes in alcohol consumption may be evaluated by measuring the level of consumption or the proportion of patients reducing consumption. Examples of some of the appropriate measures that may be used for each type of intervention are indicated in chart 2.1.

2.8 The review process must also take into account the definitions of alcohol misuse used. The targets for reducing alcohol misuse identified by the Scottish Executive are concerned with ‘heavy drinking’ by a large proportion of the adult population. The literature on alcohol misuse is generally concerned with ‘problem drinking’ and the reduction of harms associated with this. Problem drinking is variously defined; for example, as alcohol dependence, alcohol abuse (serious disturbances of normal functioning without satisfying the criteria for dependence) and hazardous use of alcohol (binge or chronic heavy drinking) (Dinh-Zarr et al, 2000). It will be important therefore to identify population subgroups not only in demographic terms but also in terms of their level of alcohol misuse.

Chart 2.1 Overview of measures of effectiveness

Statistical methods used within reviews

2.9 Some of the reviews included in this report have simply described the results that have been found in the literature. In some cases, this is entirely appropriate, as the studies are not sufficiently similar for pooling of results to take place. In other cases, the reviews only provide qualitative information about the studies included. Where studies have reported statistical summaries of the findings, reviews use different methods, some of which are easier to interpret than others, in terms of the impact of the intervention on alcohol misuse. Meta-analysis, involving the use of statistical methods to pool results from studies, is applied in some of the reviews, although it should be noted that in older studies the term meta-analysis may be used more generally to mean a summarising of findings.

2.10 A number of primary studies report results in terms of effect size. For the outcome variable of interest, this is measured as the difference between the intervention and control group means, divided by the pooled standard deviation. It measures the extent to which the distribution of the outcome variable has been changed by the intervention. This is a valid method for determining whether the intervention has had a statistically significant impact. However, it is not always possible to provide a meaningful interpretation of the effect size without reference to the original study data.

2.11 Where reviews have carried out a quantitative analysis of such studies, the pooled results are reported in terms of the weighted mean effect size; each effect size is weighted by the inverse of its variance. This process gives greater weight to larger samples with more precise results.

2.12 Study results can be more easily understood when the results have been reported in terms of the change in the outcome variable of interest; for example, the reduction in units of alcohol consumed or the increase in abstinence rates. Another method of reporting results is the odds ratio, which is the likelihood of observing an outcome for the intervention group compared with the comparison group. An odds ratio of 1 reflects no difference between the groups. An odds ratio of 2 indicates the outcome is twice as likely.

2.13 The statistical significance of the findings refers to the possibility that differences in the intervention and comparison groups are observed by chance. A result is referred to as being statistically significant when the probability of the result occurring by chance falls below some threshold, usually 5%. Alternatively, this information can be presented in terms of a confidence interval (CI), usually 95%. This gives a range around the estimated value within which the true value is expected to lie. There is only a 5% chance that the true value lies outside a 95% CI.

< Previous | Contents | Next >