« Previous | Contents | Next »
Listen
CHAPTER TWO: METHODS
Review approach
2.1 The review team followed the 'gold standard' protocol for the systematic review method set out by the Cochrane Collaboration and the NHS Centre for Reviews & Dissemination. The core principles of this methodology, which set it aside from the more traditional approach to carrying out a review, are:
- A comprehensive and replicable search strategy
- Quality control of included material
- Objective synthesis of the evidence
Pitfalls of the systematic review approach, which are increasingly highlighted in the literature, are:
- Long time scale
- Narrow focus
- Lack of cost-effectiveness
- Wasteful approach to data retrieval
- Lack of clinical relevance
2.2 Aside from time-scale, which is an ubiquitous complaint regarding all research, the above concerns are, in fact, not an inherent feature of the systematic review method. They derive primarily from the approaches taken to data retrieval and analysis. With regard to data retrieval, 'live' on-line search and retrieval strategies commonly result in high cost and, perhaps more importantly, the necessity of discarding substantive quantities of potentially relevant material. The narrow focus and lack of clinical relevance commonly cited results from the decision to set tight search parameters in advance of initial citation retrieval and to focus, for similar reasons, on retrieving and extracting data from only the 'highest quality' studies (currently synonymous with randomised controlled trials in the context of intervention research). The latter requirement relies on the implicit and erroneous assumptions that 'poorer quality' evidence is no evidence at all and that poorly executed high quality designs are nevertheless able to provide superior evidence to that provided by well executed but less ideal methods.
2.3 In carrying out a similarly wide-ranging systematic review of risk assessment and intervention in the context of other-directed violent behaviour (Leitner et al 2006), we developed an alternative approach to data retrieval which we believe better serves the pragmatic needs of clinical research and which has advantages for future research in retaining rather than discarding material suited to addressing novel questions which may arise following the initial outcomes of a review. Simply put, our approach is to set very broad initial search parameters, download all initially retrieved citations to a bibliographic software package and develop syntax to carry out post hoc explorations of the resulting extensive database.
2.4 The above approach allows outcomes to be explored 'iteratively' following the standard empirical approaches to theory testing used in primary research. The syntax models we use are based on the successive fractions approach of Hartley et al (1993) which test the impact of permutations of main terms (such as 'suicide') and restriction terms (such as 'intervention') in determining the volume and specificity of retrieved material. An additional benefit of using this approach in the current context is that it allows any 'clustering' of the literature around core themes to be data driven. In combination with an approach to data analysis which embraces a broad range of distinct study designs and which evaluates outcomes using studies as well as participants as a unit of analysis, we feel that this approach provides the necessary flexibility to address immediate clinical concerns in complex areas such as suicide. It also provides the option of revisiting the broader database of initially retrieved citations should additional queries arise following preliminary research.
Review protocol
2.5 Since the aim of the review was to provide as broad as possible an overview of the relevant literature in this field, the range of databases searched was chosen to reflect a diverse range of approaches to the issue of intervention and to access, in so far as was possible within the restricted time period available, both formal and 'grey' 4 sources of literature. The databases chosen on this basis were as follows:
Medical Literature: Medline, National Research Register, NICE, Controlled Clinical Trials Register
Nursing, Allied Health & Complementary Medicine:CINAHL, AMED
Social Sciences & Psychology: PsychInfo, ASSIA (applied social sciences), Social Sciences Citation Index, APA PsychArticles
Specialist Reviews Literature: Cochrane (Medical) (including DARE and Cochrane Reviews and Cochrane Methodological Reviews), C2-Spectr (Criminological/forensic)
Health Economics & Health Technology Assessment:NCCHTA, NHSEED, ECONLIT
'Grey' Literature:PROQUEST, FADE
2.6 Within each of the above databases, searches were unrestricted by date, except in respect to the limits set by the database itself. The earliest citation retrieved was from 1956 (via Social Sciences Citation Index). Searches were updated and finalised in June 2006. Material appearing in electronic databases after this point is therefore outside the scope of the review. Given the limited time and resources available for the review, it was necessary to restrict searches to the English language literature. Previous experience with similar reviews suggests that this restriction is unlikely to have had a significant impact on outcomes. Only around 1% of available material in this research field is likely to be accessible only in languages other than English. In line with specifications in the tender, the search was not restricted by age of study participants, or by the type of intervention considered, or by study design or by the population or setting for which an intervention had been developed or in which an intervention was evaluated. Following initial trials of possible search strategies and subsequent discussions with the Research Advisory Group, the following limited range of inclusion/exclusion criteria were set to 'fine tune' the otherwise very broad remit of the review outlined above:
- Only 'empirical' studies to be included, broadly defined to include any quantitative or qualitative approach aiming to evaluate the impact of an intervention on self-harm or suicide
- Outcomes to include all completed suicide and suicidal behaviour, including self-harm and suicidal ideation
- Focus on 'intentional' behaviour only (e.g. exclusion of non-intentional self-harm in people with learning disabilities or conditions such as Lesch Nyhan's disease)
2.7 In categorising studies with respect to the type of suicidal behaviour addressed (completed suicide, attempted suicide, self-harm, suicidal ideation) we were, of necessity, wholly dependent on the descriptions given by study authors. Since the descriptions provided were in general quite poor, it has not been possible to draw any fine-grained distinctions within and between categories. For example, we are unable to differentiate here between attempted suicide/self-harm with or without identified suicidal intent. The four main categories themselves reflect the forms of behaviour specifically included within the remit of the review, but also accurately reflect the most common labels applied by study authors to the behaviours being evaluated. It should be noted that there is likely to be some overlap in the behaviours assigned, respectively, to the categories of 'attempted suicide' and 'self-harm'. One author's definition of attempted suicide may well be another author's definition of 'self-harm' and we have no way of unpicking this further.
2.8 Following the novel approach outlined earlier, initial search terms used for on-line searching in the above databases adopted the generic format outlined below:
Suicid* OR selfharm* OR self-harm* OR (self AND harm*) OR selfinjur* OR self-injur* OR (self AND injur*) OR selfpoison* OR self-poison* OR (self AND poison*) OR selfmutilat* OR self-mutilat* OR (self AND mutilat*) OR selflacerat* OR self-lacerat* OR (self AND lacerat*) OR selfcut* OR self-cut* OR (self AND cut*) OR parasuicid* OR para-suicid* OR ((deliberat* OR intent*) AND overdos*)
2.9 The asterisk in the above search string indicates a 'wildcard', which allows for the retrieval of all terms including the preceding phrase (e.g. for 'suicid*' this would retrieve also articles referencing suicide, suicidal, suicidality etc.). The search string as set out is written in a generic format. Different databases use distinct approaches to literal and Boolean searching and substitute diverse wildcards and connection terms. The search string was adapted to the format of each database as necessary. The rationale for restricting the main search terms for 'overdose' by the terms 'deliberat*' and 'intent*' was that, in running trials of the search strategy, it become clear that whilst the other self-harm related terms were in and of themselves comparatively specific, the term 'overdose' was, in search terms, an extremely over-sensitive one, accessing a broad range of irrelevant material including accidental overdosing of patients in the medical context. A Medline trial of the search string including overdose as an unrestricted term, for example, produced a total of 289,799 citations in contrast to 70,371 with the restriction terms added.
2.10 Once citations from all databases had been downloaded into the bibliographic software (Reference Manager) and de-duplicated to remove replications of any given citation which had been identified by more than one database, a restriction term string was developed to identify material relating specifically to interventions:
{Interven*} OR {prevent*} OR {control*} OR {manage*} OR {treat*} OR {reduc*} OR {stop*} OR {restrain*} OR {trial*}
2.11 Additional restriction terms initially run on a trial basis and subsequently rejected included car* (care, caring etc.) and help* (helping etc.). As with 'overdose', these increased the sensitivity but substantially reduced the specificity of the search in identifying relevant material. Although, ideally, a full abstract or full-text search would have been undertaken, given pragmatic constraints, the final search was tied to words appearing either in the title of a citation or in specified keywords. Annex D provides a table summarising outcomes from the above search strategy, setting out the number of citations retrieved via each database (prior to de-duplication) for the full Boolean search (for those databases supporting Boolean search strings) and for each individual set of search terms (self-harm, selfharm, self AND harm etc) taken separately. This is of value not only in tracking the current search through to its sources, but also in evaluating the breadth of coverage of material relating to suicide and self-harm by the different types of source previously outlined.
Review process
2.12 The following diagram gives a visual overview of the stages and outcomes in the review process:
Figure 2.1
2.13 Following de-duplication, the number of individual citations available for searching with the set of restriction terms developed was 26,085. The intervention restriction terms reduced this number by around two thirds. Exploratory random selection searches within the excluded material suggest that a high proportion of the excluded material relates to purely discursive papers. However, it was equally apparent that there is an imbalance in the literature favouring the analysis of risk over intervention.
2.14 The abstracts of the 8,606 citations identified using the intervention restriction terms were each read by one reviewer with the aim of excluding any which very clearly did not meet our review criteria. Abstracts which were ambiguous or which failed to provide sufficient information were initially read by two reviewers and if a decision regarding exclusion could still not be made the full-text material was ordered. At the end of this process 646 citations were identified as potentially meeting all of the review criteria and were obtained in full-text format. Each of these full-text articles were read by two reviewers, with a third reviewer reading any for which an initial decision to include or exclude proved problematic. This resulted in 235 reports of studies meeting the review criteria. An additional 8 reports were identified as of possible relevance, but these could not be retrieved during the timescale of the review. Subsequently we have been provided with copies of 6 of these missing papers 5 and references and brief summaries of the papers are provided at Annex A. Evidence taken from this additional material does not alter any conclusions reached in the review.
2.15 Within the 235 reports identified, we include 'linked' material. That is, separate reports of a study which provide additional rather than identical material - for example, additional years of follow-up for an ongoing trial, or a meta-analysis of data from two or more trials. For current purposes these are counted as separate studies. Duplicate papers, providing no new information, have been discarded, with the named paper for the review referring to the paper providing most comprehensive details of the study methods etc. In total, 198 of the citations subject to full-text retrieval reported on primary empirical studies. A small number of these reported on either two distinct primary studies or on a primary study and a meta-analysis or systematic review. Dividing these out provided a final total of 200 primary empirical studies and 37 systematic reviews falling within the remit of the current review.
2.16 Since the review process was designed to identify studies reporting on suicide, attempted suicide, self-harm or suicidal ideation as explicit outcomes, studies which may include pertinent information but which have not themselves identified these issues as a specific outcome (for example studies reporting on suicide as an unintended adverse consequence or studies focussed on other main outcomes but reporting incidental outcomes for suicidal behaviour) are unlikely to have been included. All material was identified by electronic searches as, for pragmatic reasons, it was not possible to carry out hand searches of key journals or to check the reference lists of all retrieved articles. This may have led to additional material being missed.
Definitions and terminology
2.17 The remit of the review was to identify and report on studies evaluating interventions for suicide, attempted suicide, self-harm and suicidal ideation. Defining these terms is not straightforward and, increasingly, both researchers and clinicians are recognising the need for a standardised nomenclature in this field (cf. Andriessen 2006, Silverman 2006). Definitions of the terms which have been cited fairly widely in the literature are as follows:
Suicide The termination of an individual's life resulting directly or indirectly from a positive or negative act of the victim himself which he knows will produce this fatal result (Durkheim 1857)
Attempted suicide A potentially self injurious action with a non-fatal outcome for which there is evidence, either explicit or implicit, that the individual intended to kill himself or herself (Moscicki 1997)
(Deliberate) Self-Harm An acute non-fatal act of self harm carried out deliberately in the form of an acute episode of behaviour by an individual with variable motivation (Gelder et al 2001)
Suicidal Ideation The existence of current wishes and plans to commit suicide (Steer et al 1993)
2.18 All of the above definitions refer directly or indirectly to the conscious motivations of an individual and it is this reference to the motivation behind an act which makes the definition of suicidal behaviour and ideation problematic. By way of example, at the point at which a person engages in an act which may later be defined by themselves or by others as attempted suicide, they are likely to be in an emotionally charged state, may be under the influence of alcohol or drugs and may have little perspective regarding their own specific motivations or intentions. Following the act, external rationalisations, concerns regarding the perceptions of others and a confused memory of the events leading up to the act or of the act itself may cloud any retrospective interpretation of what happened. Consequently even the person themselves may not be able to provide a clear account of their motivation in carrying out a particular act.
2.19 Defining suicidal behaviour via the perceptions of other key observers of the act or its aftermath (e.g. Harris et al's 2005 definition of self-poisoning, which includes "severe alcohol intoxication where clinical staff consider such cases to be acts of self-harm") is equally unreliable. Studies evaluating variations in clinician-assigned diagnostic codes (e.g. Rhodes et al 2002), for example, have demonstrated that the likelihood that a person who has self-harmed will be diagnosed as having self-harmed depends on a wide variety of factors, including the age of the person, their length of stay in hospital, the prior existence of a mental health diagnosis, the clinical speciality of the diagnosing clinician and whether or not the clinician was asked to identify the diagnosis as 'deliberate self-harm' or simply 'self-harm' in hospital records.
2.20 Attempts to provide operational definitions via reference to 'objective' features of an act such as its lethality or via the development of psychometric measures of intentionality such as the Suicide Intent Scale (Beck 1974) have also failed to resolve the problem of definition, since there appears to be at best only a weak association between intentionality, lethality and the nature of the act itself (cf. Nielsen et al 1993). Finally, attempts to define suicidal behaviours without reference to motivation (e.g. the definition of self-harm as "…intentional self-injury or self-poisoning, irrespective of motivation" given by Hawton et al 2003a and the definition of 'parasuicide' as an act "in which the (person) simulates or mimics suicide, in that he is the immediate agent of an act which is actually or potentially physically harmful to himself" by Kreitman et al, 1969) unfortunately only shift the problem of definition elsewhere by substituting terms which themselves are open to interpretation (e.g. 'intentional' 'simulate').
2.21 The authors of the primary studies included in this review very rarely provided any definition of the behaviours evaluated beyond ascribing the broad labels of 'suicide', 'attempted suicide' etc. It is, however, apparent from the above discussion that the behaviours against which interventions are judged are unlikely to be wholly equivalent across studies even where the labels assigned are the same. Similar problems of definition occur also in respect of the outcome measures used. For example, in the case of completed suicide a study author may establish rates of suicide based on local police statistics, coroner's reports, national statistics or the reports of next of kin. These and other available sources do not operate to the same guidelines, accept the same weight or type of evidence or report their conclusions in the same format. Again, therefore, one study, even of completed suicide, can only be regarded as approximately similar to another. Similarly, one scale-based measure of suicidal ideation may take into account factors not considered by another (e.g. presence or absence of 'plans' to carry out a suicide attempt) and again the behaviours included in the evaluation of an intervention, although broadly similar, cannot be encompassed within precisely the same definition.
2.22 Since we have no access to any account of the behaviours evaluated in the literature other than through the study author's descriptions, we use the labels given by individual study authors to define the behaviours at issue throughout the report. We recognise that the lack of tightly specified concrete definitions of the behaviours addressed by individual studies may be a source of frustration for practitioners attempting to apply the findings to their own clients. Unfortunately, this reflects the reality of the literature available and, more broadly, the complex nature of the behaviours themselves.
2.23 Finally, in respect of a further key definition within the text, we use the terms 'intervention' or 'intervention practice' to cover intervention by the full range of professional and volunteer bodies engaged in suicide prevention activity. In reality, however, the vast majority of the research evidence relates to clinical practice, generally with the implied expectation that the evaluated interventions will be delivered by the medical, mental health and clinical psychology professions. Very few studies specified a particular provider outside of these professional groups. This, of course, does not rule out the effectiveness of such interventions delivered via other groups and, in the small number of cases where studies addressed educational or other public health and related initiatives, the primary provider is in any case likely to be a specialist outside of these professions.
Further contextual details of the included studies and how these may be seen to impact on the interpretation of study outcomes are given in Annex F.
« Previous | Contents | Next »