The Scottish Government

« Previous | Contents | Next »

Listen

Risk Assessment and Management of Serious Violent and Sexual Offenders: A review of current issues

CHAPTER THREE: RISK ASSESSMENT

KEY DEFINITIONS

3.1 'Risk' has traditionally been a neutral term meaning the chance of gain or the chance of loss (Parton, 1996). Increasingly, however, risk has become associated with notions of hazard, danger or harm (Douglas, 1992). The terms of reference for the MacLean Committee for example were to 'consider serious violent and sexual offenders who may present a continuing danger to the public' (2000: 7). However, the term 'dangerousness' was replaced by the term 'risk' on the grounds that the latter incorporates a wider consideration of contextual and circumstantial factors as well as dispositional personality traits. A risk assessment can be therefore be characterised as a:

… probability calculation that a harmful behaviour or event will occur, and involves an assessment about the frequency of the behaviour/event, its likely impact and who it will affect. (Kemshall 1996a: v).

3.2 This definition captures the two key ingredient of any risk assessment:

• A calculation of frequency or likelihood, usually expressed as a probability calculation.
• A calculation of likely impact and where possible identification of likely or potential victims.

3.3 Scott (1977) has expressed this as an assessment of:

• the behaviour of concern;
• the potential damage or harm likely from that behaviour; and,
• the probability that it will occur and under what circumstances.

3.4 The level and impact of harm has been central to recent preventative measures (see section 2), and harm reduction is a key principle of such legislative measures. Harm is understood as psychological as well as physical, sexual and violent harm.

3.5 It is also worth noting that:

• Risk assessment is a constant process involving risk management, review of its effectiveness and re-assessment.
• Re-appraisal in cases of escalation or reduction of risks, risks change over time and in differing contexts.
• Risk is uncertain -only relative probabilities can be estimated.
• Risk operates along a continuum-thresholds of risk can be difficult to establish and are liable to change over time. The difference between an offender at the top of the medium range and an offender at the bottom of the top range is
• minimal. Gradual and flexible responses, particularly to escalating risk are desirable (MacLean 2000: 9).

3.6 Whilst definitions of high-risk are becoming more rigorous, the range of offence types encompassed by this target group is wide. Sex offending is difficult to define and sex offenders are not a homogeneous category (Grubin, 1998; SWSIS 1997). Similar problems apply to violent offenders (Powis 2002). A Commitment to Protect for example concerned itself with those sexual offences that involve exploitation and/or assault. The following categories are presented as a useful starting point:

• familial child sex abusers;
• non-familial abusers;
• paedophiles;
• rapists; and,
• indecent exposers.

3.7 This is supported by Grubin (1998: 14) who presents a categorisation based upon the offender's:

• choice of victim;
• criminal background;
• sexual arousal patterns;
• social functioning; and,
• risk of re-offending.

3.8 Violent offenders are an equally diverse group, encompassing those who:

• are involved in domestic violence;
• harm others in the commission of other offences (e.g. by using firearms);
• harm vulnerable persons (such as children or the elderly);
• use threat or force which is likely to result in injury to people (e.g. offences of robbery) (Megargee, 1976:12); and,
• commit violence as a result of mental disorder (Swanson and Holzer, 1990).

3.9 This diversity indicates that risk assessment tools need to be chosen on the basis of their applicability to the offender group and offence type under consideration.

PROBLEMS IN RISK ASSESSMENT

3.10 The key issue in risk assessment is accuracy, and the avoidance of either over- prediction or under-prediction. In any risk assessment there are four possible outcomes. These are displayed in Figure 1.

Adapted from Moore, 1996

3.11 Risk predictions can be right by predicting correctly that a harmful behaviour will occur (Box A), or by predicting that a harmful behaviour will not occur and it does not (Box D). However, errors are also likely and they carry significant costs for both workers and their agencies. Box B highlights those cases in which a risk of harm is not identified but does occur, and Box C identifies those cases in which harm is predicted but does not occur.

3.12 In Box B cases, the consequences can be very high. Victims may be harmed or killed, and workers and their agencies can be brought into disrepute. In Box C cases, the criminal justice system can over-intervene with significant impact upon civil liberties and waste of precious resources. Box B cases tend to result in defensive practice, caution and over-prediction amongst practitioners and their agencies as a response to costly failures. Box C cases tend to raise significant ethical dilemmas for practitioners, and resistance from those concerned with the erosion of civil liberties.

3.13 Whilst Box B and Box C errors can be reduced, it is usually at the expense of increasing the other type of error (Moore, 1996), and not by increasing true positive or true negative predictions. Tolerance of false positives and false negatives can be a matter of moral and political acceptability. Within child abuse prediction for example, tolerance of false negative predictions (that is, no harm will come to a child) can be low (e.g. the Beckford Enquiry, London Borough of Brent, 1985). However, this does not prevent public outcry in cases of false positive prediction and over-intervention, for example Cleveland (Butler-Schloss 1988), and in the Scottish context, Orkney (Clyde 1992).

ISSUES IN RISK ASSESSMENT: ACTUARIAL VERSUS CLINICAL METHODS

3.14 There are two basic approaches to risk assessment and prediction for offenders:

• the actuarial; and,
• the clinical.

The following section provides a summary of each approach.

Actuarial assessment

3.15 Actuarial risk assessment is based upon statistical calculations of probability. Well used in the insurance industry (Green, 1997), actuarial methods for offender risk prediction utilise the basic methodology pioneered by Burgess (1936) for parole violation. From the study of a large number of cases, certain factors that statistically relate to risk, are selected. These are then retrospectively validated by application to cases with a low expectancy of risk and to those with a high expectancy of risk. Risk factors are then retrospectively validated in terms of statistical probabilities. Such factors are often referred to as static risk factors as they are deemed largely unchangeable and rooted in historical and demographic factors.

3.16 Whilst the method has greater accuracy than clinical assessment (Milner and Campbell, 1985, Quinsey et al 1998), the approach does have its difficulties. These fall under three main headings:

• Statistical fallacy:
• Low base rates; and,
• Limitations within meta-analysis

Statistical fallacy

3.17 Heyman (1997: 8) has argued that probability reasoning reduces the uncertainty of risk by:'...attributing aggregate properties of a category to individuals within that category...'. This systemic flaw is more commonly known as the 'statistical fallacy' (Dingwall, 1989). Therefore, whilst it has greater predictive utility, the actuarial method compares similarities of an individual's profile to the aggregated knowledge of past events.

3.18 Grubin and Wingate (1996: 353) have noted that a number of particular limitations apply to actuarial tools for sex offence recidivism prediction. In reviewing a number of prediction studies they identified that empirical evidence from one population does not necessarily translate to another, and that most prediction scores cluster at around the 40 per cent mark. Grubin and Wingate remarked that even if this represented 'a significant improvement over chance, is not particularly helpful to those who must make decisions about release'. Such predictions merely state that 40 cases in 100 are a potential risk; the method cannot identify with absolute reliability the likely risk in any individual case. They state that in Quinsey et al's study (1995a) only 3 per cent of the sample (that is, just six men) had 'clinically meaningful' scores of around 85 per cent, that is, a risk score indicating an 85 per cent probability of future risky behaviour. This highlights the problem of transferring actuarial data about groups to prediction about individuals. In other words, the extent to which information from a group population can be generalised to the individual under assessment is problematic. In addition, classification profiles can change as what is known changes over time, and thus risk classifications can change as aggregates do. Hence, insurance companies revise their premiums over time.

Low base rates

3.19 Low base rates can also present significant difficulties for accuracy. The base rate is the known frequency of a behaviour occurring within the population as a whole, and provides the basis for an actuarial prediction of behaviour in similar cases. For behaviours with low base rates such as child abuse or sexual offending, prediction in ignorance of the relevant base rate can lead to error. This is because such predictions can be based upon data about infrequently occurring past behaviours limited to small groups of the population (for example, violent behaviour amongst women). Predicting infrequently occurring behaviours amongst the population as a whole can therefore lead to error. In effect, the correlation coefficient is adversely limited by low base rates.

3.20 More recent statistical developments in actuarial research have been able to compensate for this effect through the application of a technique know as 'Relative Operating Characteristic' or ROC (Mossman 1994, Rice and Harris 1995). In short, this technique enables actuarial evaluations of violence prediction free from base rate limitations and clinical 'biases for or against Type I or Type II prediction errors' ⁵ (Mossman, 1994:783). In a re-evaluation of 58 data sets from 44 published studies using the ROC technique Mossman demonstrated that mental health practitioners' predictions of violence were substantially more accurate than chance, that short term predictions were no more accurate than long-term ones, and that past behaviour was the best predictor of future behaviour (p.783).

Limitations of meta-analysis

3.21 This difficulty is exacerbated when meta-analysis is the preferred methodology for establishing actuarial predictors. Meta-analysis is a statistically based technique that analyses the outcomes of a large volume of primary research studies. These outcomes are then aggregated in order to establish which factors and outcomes have the most statistical significance for risk prediction (McGuire, 1997). In risk meta-analysis has been used to establish those factors which have the most predictive utility.

3.22 However, a number of difficulties exist with this approach. As Grubin and Wingate (1996: 356) state, 'meta-analysis is not particularly good at demonstrating multi-variant effects, which require methodologies of a more complex type...'. In other words, they are not good at identifying a range of possible effects and their interaction. In offender risk prediction more generally, meta-analysis also has its limits. Copas (1995: 12) has suggested that whilst useful as a 'descriptive mode', its use in drawing inferences from the data as a whole is limited. Complex outcome measures are often simplistically categorised for ease of comparative analysis (Copas, 1995; Mair, 1997), and the selection of the original studies and the statistical methods employed are open to subjective bias (Losel, 1995; McIvor, 1997).

3.23 Actuarial risk variables can also have limited clinical use in the field as they rarely explain behaviour (Grubin and Wingate, 1996). In essence, this is the distinction between merely predicting that a risk is likely, and explaining and understanding risky behaviours (Pollock et al 1989). The latter is essential for practitioners responsible for establishing treatment plans and implementing risk management interventions. This is supported by Weist (1981) who has suggested that a detailed analysis of the interaction between personality and situational factors is essential to establishing treatability and in aiding the worker to select the most suitable treatment programme. Such analysis focuses worker attention on those situational and clinical factors that can be changed or prevented by targeted interventions.

3.24 In spite of these criticisms, however, actuarial assessment can be used to:

• establish those risk predictors which have a proven track record;
• establish the relevant base rates for clinical assessment;
• increase the accuracy of risk assessments; and,
• increase levels of consistency and reliability.

Clinical assessment

3.25 The clinical method is essentially a diagnostic assessment derived in part from the medical and mental health fields (Monahan, 1981). It is based upon detailed interviewing and observation by the clinician in order to collect information on the social, environmental, behavioural and personality factors that have resulted in harmful behaviour(s) in the past. Hollin and Howells (1989) describe the process as an individualised assessment, usually concerned with providing a diagnosis, establishing treatability, and where release to the community or legal reports are required, with predictions of dangerousness (Pollock et al, 1989). However such predictions have been particularly plagued by unreliability (Monahan 1981, Quinsey et al 1998).

3.26 Imprecise definitions of dangerousness (Brooks 1984), coupled with lack of knowledge on relevant base rate behaviours (Gottfredson and Gottfredson, 1993) and flaws in 'subjective inference' have contributed to the limitations in clinical assessment (Kahneman and Tversky 1973). In particular, clinicians have a limited ability to judge accurately probability, with judgement biased towards the frequency (rather than the probability) of individual events (Kahneman and Tversky 1973). It is argued that the processing of information on both probability and likely impact is affected by a number of cognitive heuristics or 'rules of thumb', such as the 'availability' heuristic in which the risks being assessed are matched to the information most easily available and recalled (Combs and Slovic 1979). Clinicians have traditionally preferred to give weight to case based rather than statistical information (Carroll 1977, Nisbett et al 1976, Shah 1978). False risk predictions can also arise from 'creeping determinism' (Fischoff, 1975) which suggests a causal connection between factors in a case where none in fact exists. In the production of a coherent narrative, it has been argued that causal connections can be literally imputed (Einhorn 1986, Pollock et al 1989).

3.27 Pollock et al (1989) contend that three decades of 'vigorous research' have yet to produce the 'scientific knowledge needed to predict violent behaviour' (p.96). However, more recent commentators (Limandri and Sheridan, 1995) have argued that if combined with the appropriate actuarial data, clinically-based interviewing can have an important role in establishing the significant personality and situational factors which can trigger or exacerbate risky behaviour (Megargee, 1976; Prins, 1988). This assists with explanations of behaviour and the planning of treatment interventions, presented by some commentators as a more preferable role for clinical assessment (Pollock et al 1989, Weist 1981), and as such is likely to be more helpful than prediction per se.

3.28 Structured clinical interviewing around empirically grounded risk factors or 'criminogenic needs' are already in use in the general assessment of recidivism in probation work (for example the Level of Service Inventory-Revised (Andrews and Bonta, 1995), and have been incorporated into the piloted national prison-probation risk tool OASys ⁶ in England and Wales. The current figures for reconviction prediction are 69.2 per cent representing a 37.4 per cent improvement over chance (Clark 2002). Whilst the tool does have a separate risk of harm section, this has been less extensively evaluated, and is the least actuarial in construction (in part reflecting traditional base rate problems in this area). The tool's major contribution is in the area of criminogenic needs assessment and the targeting of offenders for accredited programmes of intervention in both prison and probation settings.

3.29 Structured clinical assessment has also been used in cognitive self-change programmes for violent offenders (Bush, 1995) and in offending behaviour programmes rooted in the 'What Works' research of Andrews (1995), McGuire and Priestley (1985; 1995) and Ross and Fabiano (1985). In violence assessment the 1990s saw the development of detailed lists and 'aides memoire' to guide the assessment of clinicians. For example Webster et al's (1994) ASSESS-LIST which was offered as a 'guide' to 'comprehensive inquiry' rather than as a thoroughly evaluated predictive tool (p.46). This was subsequently superseded by the HCR-20 version 2, although this again is described as an 'aide memoire' (Webster et al 1997:5).

3.30 Clinical assessment of sex offenders against children, particularly in terms of their suitability for cognitive-behavioural group treatment programmes, has been assisted in the 1980s and 1990s by theoretical and empirical work in a number of areas. These include work on:

• predisposing preconditions (Finkelhor 1984);
• the 'cycle of assault' which emphasises the physiological, psychological, behavioural and situational factors which contribute to sexual offending (Ryan et al 1987, Wolf 1984); and,
• the role of 'cognitive distortions' and denial in sex offending (Salter 1988).

Use of dynamic risk factors

3.31 Dynamic risk factors have been described broadly as those factors which change over time, or which can be made to change through treatment and intervention (Quinsey et al, 1998). In the assessment of general offender recidivism, such factors have been labelled as 'criminogenic needs' (e.g. within the LSI-R and OASys) . Whilst it is generally agreed that they do not out-perform static actuarial predictors ⁷, the role of dynamic factors in establishing treatment and intervention plans is now well established (HMIP, 1998a; Raynor, 1997). As Quinsey et al (1998) point out, their assessment is often more complex due to their variable nature, for example some may relate to an offender's environment, others to social networks. Some may change naturally with the passage of time, for example levels of maturity, others may need specific interventions such as housing and employment. Still others, such as treatment impact may be difficult to assess as discrete from other variables in the offender's life such as gaining a stable life-style.

3.32 Whilst dynamic variables are important , how they should be weighted within risk assessments can present significant problems (Raynor, 1997). Notwithstanding the problems of measuring re-offending rather than re-conviction, dynamic variables are more difficult to measure than criminal history as they are often compiled from differing sources (including the self-report of the offender), and are open to interpretation by the assessor. May's study of over 7,000 offenders concluded, however, that whilst criminal history is the best predictor of re-offending, those offenders with multiple problems are more 'at risk' (1999). Dynamic factors such as drug misuse, accommodation and employment were found to have a 'clear link' to reconviction (p.26), and knowledge of social factors was particularly helpful in predicting reconviction for those cases with little criminal history (p.38). May's study does acknowledge the varying interpretations of social factors by assessors and variations in their recording, and that some so-called 'social factors', such as ethnicity and having been the victim of abuse, are not dynamic. May concludes, however, that the identification of relevant social factors 'could help to confirm the notions of the factors that need to be tackled to reduce re-offending' (p.49).

3.33 The contribution of dynamic factors to the assessment of violent offending has also been explored. Hagell (1998) notes that whilst there is 'an emerging consensus that multiple indicators are likely to be more successful than individual factors' (p.56), which dynamic factors apply in each individual case and how factors overlap is still problematic. For some factors, such as substance abuse and the use and availability of weapons, research evidence is either 'unclear or insufficient' (p.57). Personal factors, such as the offender's general disposition or temperament, and cognitive factors, have also been shown as features of dangerousness assessments (Blackburn, 1994; Howls, 1987). Bush (1995) for example identified cognitive distortions as significant features of decisions to offend violently, with 'anti-social logic' used to justify violent behaviour towards others. His Cognitive Self-Change Programme is based upon challenging such anti-social thinking patterns (Bush 1995). In a West Midlands study of Section 18 and Section 20 offences, Genders and Morrison (1996) found that offenders tended to blame others and justify their actions as 'out of control'. Other personal and temperamental factors such as lack of self-control, lack of victim empathy and high levels of hostility and aggressiveness have been raised as dynamic indicators of violent behaviour (Hare, 1993; Blackburn, 1994; Menzies et al, 1994). However, whilst important in indicating areas for potential intervention, none of these factors can out-predict past history and convictions.

3.34 In their study of partner abuse and familial, violence Limandri and Sheridan (1995) noted that violence assessment is enhanced by the addition of key dynamic factors such as 'disinhibiting agents', use and availability of weapons, and access and proximity to victims. Whilst their work is limited to partner abuse, they importantly suggest that violence prediction is likely to benefit from research into multiple paths to offending.

3.35 In sex offending assessment, cognitive distortions, the cycle of offending (Wolf, 1984) and grooming patterns have gained significance, particularly in probation and prison officer assessments for group work programmes (Abel et al, 1987; McEwan and Sullivan, 1996). Integrated or multi-factoral theories have also been proposed by research in this field (Finkelhor, 1984; Marshall and Barbaree, 1990; Prentky, 1995; Wolf, 1984). These theories stress personality factors such as: egocentricity, poor self image, defensiveness, distorted thinking, obsessive thoughts and behaviours, social alienation, sexual preoccupation (Scottish Office, 1997).

3.36 Prentky (1995) has outlined a number of factors significant to sex offending (although all will not necessarily be present in each individual case):

• impaired relationships with adults;
• lack of victim empathy;
• extent and nature of anger, particularly whether instrumental or expressive;
• cognitive distortions and rationalisations for offending;
• sexual fantasy and deviant sexual arousal;
• antisocial personality; and,
• impulsivity.
  (Prentky, 1995:159-167).

3.37 However, as with dynamic factors for violent offending, dynamic factors in sexual offending can vary between offender types (for example sexual offending against children and offending against adults). Furthermore, the identification and relevance of each variable in individual cases is somewhat dependent upon the judgement of assessors (for example criminal justice and social workers). As with dynamic factors for general offending and violent offending, dynamic variables in sexual offending have tended to have most significance in the design and delivery of interventions (Proctor, 1996).

COMBINED RISK ASSESSMENTS

3.38 It is now generally accepted that the accuracy and consistency of risk assessments is enhanced by assessment tools which combine actuarial calculations of probability with detailed clinical interviewing to establish the conditions and circumstances under which risky behaviour(s) might occur (Milner and Campbell, 1995; Quinsey et al, 1998). Such tools combine the use of well-established static risk factors such as previous history of behaviours and convictions, with growing clinical and research knowledge on a wide range of dynamic factors. Dynamic factors are increasingly assessed through the use of 'aides memoire' or structured interviewing tools which emphasise those factors most proven by research (Raynor, 1997; Webster et al, 1997). Whilst the accuracy of pure actuarial methods is not always outperformed by the addition of clinically assessed dynamic factors (Ditchfield, 1997; Raynor, 1997), combined methods have an important 'value-added' component by identifying behavioural traits, environmental stressors, personal characteristics and social variables which can trigger offending or exacerbate risk. Multi-variate analysis of the risk of reconviction in general offending, and in sexual and violent offending, is now widely accepted as the most useful approach to risk assessment, providing both predictive utility and significant information for the design of treatment programmes and case interventions.

DEFENSIBLE DECISIONS

3.39 Carson (1996) notes that risk assessment is a highly fallible undertaking, and that it is unlikely that any method can be found which will provide certainty and 100 per cent levels of accuracy for worker, agency and public. He argues that in a situation where accuracy cannot be guaranteed, the key to decisions withstanding subsequent accountability and public scrutiny is their ' defensibility'. In other words, how decisions are evaluated with hindsight after negative outcomes have occurred, and whether decisions can be considered to be 'reasonable'. As Carson (1996: 4) expresses it, whether a 'responsible body of co-professionals would have made the same decision'. This is particularly pertinent for those agencies who carry out risk assessments in the public eye, and where risk assessment failures can be very costly to organisational credibility.

3.40 Monahan (1993) has suggested several elements that need to be present for a decision to be defensible, and these can be translated into minimum standards for risk assessment. A defensible decision is therefore made when:

• all reasonable steps have been taken;
• reliable assessment methods have been used;
• information is collected and thoroughly evaluated;
• decisions are recorded;
• staff work within agency policies and procedures; and,
• staff communicate with others and seek information they do not have.
  (Kemshall, 1996a; 1996b; 1998a; 1998b).

3.41 Defensibility is also likely to be tested by Human Rights legislation, and as such it is important that risk assessments are transparent, accountable, based on the most reliable tools, grounded in empirical evidence, and that risk management plans are proportionate to the level of risk identified. The Scottish Executive has recognised that some of the recommendations of the Cosgrove report will have legal considerations under the European Commission on Human Rights (ECHR) and the Data Protection Act, particularly in the areas of medical confidentiality, disclosure of information especially on those sex offenders against whom no charge has been proven, and transfer of information under the Data Protection Act.

INTER-AGENCY CO-OPERATION

3.42 Information exchange has been seen as crucial to effective risk assessment and management (Prins 1999), and there is consensus that this is best supported by well functioning inter-agency arrangements (Maguire et al 2001). Three areas have been identified as barriers to effective inter-agency work:

• Incompatible systems of data storage, difficulties in information access and retrieval, and inefficient computerised storage systems.
• Professional mistrust and rivalries.
• Misplaced confidentiality.
• (Cosgrove, 2001; Maguire et al 2001).

3.43 Effective inter-agency work is enhanced by formal protocols addressing these issues and formalised arrangements for information sharing and exchange (Maguire et al 2001). Such protocols should be drawn up at senior management level of the relevant agencies, and checked for both Data Protection and Human Rights legislation compliance.

« Previous | Contents | Next »