« Previous | Contents | Next »
Listen
CHAPTER EIGHT additional data
Introduction
8.1 In an attempt to obtain additional data for those on ISMS, particularly to examine outcomes, we considered a number of other sources where work has been ongoing from other evaluation teams. We are grateful to Includem for providing additional self-report data from a number of ISMS cases. We are also grateful to Nina Vaswani at Glasgow City Council for supplying data on a number of ISMS and intensive support service cases that are part of her evaluation of ISMS in Glasgow.
8.2 This data is rather limited by covering a relatively low number of cases and relying on a mixture of self reporting and perception of change rather than actual evidence of change. However, it does provide some additional information that allows us to profile ISMS/intensive support service cases as well as some tentative data on outcome measures. A full breakdown of the results is provided in Appendix E.
Self-report data
8.3 This sub-section reports on self-report offending data from Includem's Events and Changes Database. This database provides details on self report offending data from 1 st April 2006 to 7 th May 2007. This data is entered into a database by Includem project workers on a weekly basis. The data is based on young people reporting committing any offence at all.
8.4 Includem's two stated main aims when working with young people are: to reduce their offending behaviour and to increase their social inclusion. This work has included intensive support provision under the ISMS programme provided in the phase 1 LA areas. Where ISMS has been offered as an option, Includem has provided the external intensive support element of the package in five of the phase 1 areas. The following analysis therefore only covers areas where Includem has provided a service.
8.5 While Includem's database covers 157 cases in total, the sample includes 104 (66%) intensive support service cases, and 53 (34%) ISMS cases. Throughout this section, we are dealing with averages of relatively small sample sizes.
Time Input
8.6 The average number of months young people had with Includem under ISMS is 17.4 months (this includes assessment period, the period of the MRC and aftercare). The average time young people have with Includem under intensive support service is 15.1 months. Therefore, those cases receiving ISMS service provision on average had slightly more input from Includem, although this is not a statistically significant difference.
Offending
8.7 Of the total of 157 cases, 43% admitted committing an offence while receiving either ISMS or intensive support service provision. A further 18% had charges from pre-service offending coming up to be dealt with while receiving services. Of a total of 95 young people who either admitted to offending in service or who had pre-service offences to be dealt with, 36 were ISMS cases (68% of total ISMS cases) and 59 were intensive support service cases (57% of total intensive support service cases 52). This is in contrast to the findings above, which show limited use of referrals to Hearings on offending grounds. These self report statistics show a continued level of offending for these cases (although, again, we can say little about reductions in persistency and seriousness).
8.8 Those who were most likely to admit offending while receiving Includem service provision, either as ISMS or intensive support service, were those in the older age groups. While those least likely to report offending were aged 12 or 13. Table 8.1 below shows a summary of admission to offending according to age.
Table 8.1. Age of those admitting to new offences under service provision
Age of ISMS or intensive
support serviceyoung person | Number of those admitted
offending under service provision | % rounded
to nearest point |
|---|
16 | 18 | 12% |
15 | 14 | 10% |
17 | 15 | 10% |
18 | 10 | 7% |
14 | 6 | 5% |
19 | 2 | 1% |
12 | 1 | 1% |
13 | 2 | 1 % |
Source: Analysis of Includem's Events and Changes Database
YLS assessments
8.9 This information below is based on an initial Youth Level of Service ( YLS) assessment and later repetition of the same assessment by Includem. YLS is a tool for assessing general recidivism in 12-17 year-olds. It focuses on categories that represent risk and need factors such as antisocial attitudes; antisocial peers; antisocial personality; family structure and family relationships; substance use; leisure; issues of vulnerability, care and protection; and risk of harm to self and others 53. Scores are given across a number of indicators and a composite overall risk score is also assigned. Levels of risk associated with overall scores are summarised in Table 8.2.
Table 8.2. YLS Score and associated level of risk
Risk | Score |
|---|
Low | 0-8 |
|---|
Moderate | 9-22 |
|---|
High | 23-34 |
|---|
Very High | 35-42 |
|---|
Source: Includem
8.10 In this analysis, changes in risk are indicated when a young person moves across a risk category, e.g. from low risk to moderate risk. There may have been small changes in score within a risk category and this is not shown here because it is not considered to indicate marked change. A significant change is indicated when a young person has moved by two categories (i.e. high risk to low risk or vice versa, or very high risk to moderate risk or vice versa).
8.11 The sample of cases is drawn from Includem monitoring data. It includes all cases where a YLS assessment has been carried out and repeated at a later date. YLS is only suitable for offenders and it therefore has not been used in non-offence ISMS/intensive support service cases. Assessments have not been repeated in all cases, either because the young person has not yet reached a suitable point in the programme or because the young person exited the programme before a repeat assessment was undertaken. A large number of young people receiving ISMS/intensive support service are therefore excluded from this sample.
8.12 There are four types of case in the sample.
- Current ISMS. These cases are receiving intensive support and electronic monitoring and were still engaged with the programme at the time of the analysis. There are 17 of these cases in the sample.
- Current intensive support service. These cases are receiving intensive support only and were still engaged with the programme at the time of the analysis. There are 8 of these cases in the sample.
- Exited ISMS. These cases received intensive support and electronic monitoring, but had either completed the programme or exited before completion. There are 8 of these cases in the sample.
- Exited intensive support service. These cases received intensive support only, but had either completed the programme or exited before completion. There are 20 of these cases in the sample.
8.13 It should be noted that the bases for this analysis are small and results should therefore be treated with caution. There is also significant variability in the time elapsed between the first YLS assessment and the most recent repeat assessment.
8.14 The single largest group across all cases is young people in the 'high risk' category and, as expected for an intensive intervention, no young people were considered 'low risk' at the beginning of either programme. Those at low risk of recidivism are also probably excluded from YLS assessments. It may be that those considered 'moderate risk' in the YLS assessment (which only considers risk of recidivism) are experiencing other risk factors around care and protection, explaining their involvement with such an intensive intervention.
8.15 Figures 8.1 and 8.2 look at changes in the risk profile of the sampled cases.
Figure 8.1. Risk profile of sample at first test and most recent test
Note: Bases are: Current ISMS (17), Current ISS (8), Exited ISMS (8); Exited ISS (20).
Figure 8.2. Overall change in YLS Scores
Note: Bases are: Current ISMS (17), Current ISS (8), Exited ISMS (8); exited ISS (20).
8.16 None of the young people in the sample who have exited ISMS were in an increased overall risk category on final testing. None of the cases (in any category) show significantly increased risk, although there were a number of cases where the risk factor did increase by one category. This occurred, for example, in 5 of the 20 intensive support only cases
8.17 Generally, Figures 8.1 and 8.2 seem to show little improvement in risk category for current and exited cases on both programmes. However, assessments made during the ISMS or intensive support service programme ('current cases') may give a misleading indicator of outcomes, as some attitudinal change can be slow to materialise, particularly in circumstances where significant behavioural change is underway. While it is difficult to generalise about what might have happened in the absence of ISMS/intensive support service, it seems likely that stabilisation of risk ('no change') can be considered a positive outcome in many cases. Likewise, an increase in the overall risk category may still indicate an improved trajectory and does not necessarily imply that the intervention has not had a positive impact.
8.18 It is also the case where changes in these scores have been monitored and measured in LAs (specifically in Dundee and Glasgow, see Chapter 3) that more noticeable improvements have been detected.
Additional data from Glasgow City Council
8.19 As part of its own evaluation, Glasgow City Council ( GCC) made available data on interviews with social workers (from the Glasgow ISMS Team and the Area Team) to the DTZ evaluation.
8.20 These interviews provided some useful additional outcomes data as they asked social workers about the changes that they observed in the young person during and after they were on ISMS or intensive support services. Below we provide some analysis of this data, but we need to be cautious in interpreting it for several main reasons.
- It is based on social workers' perceptions of change, which may not be accurate in terms of actual behavioural change. The GCC team found that, for a number of cases, the perception of social workers was not accurate when compared with actual levels of change that were identified in offending data, YLS scores, etc.
- The data covers changes in behaviour or attitudes by the young people. We do not know the extent to which these changes were as a result of the programmes.
- The sample sizes are relatively small, making it difficult to draw strong conclusions. Sample sizes become particularly small the further we drill down into the analysis to look at particular groups of young people. The bases for all charts are included in Appendix E. Bases of less than 6 are highlighted in red and have been omitted from figures in this Section.
- Comparisons between the ISMS and intensive support service cases are difficult because the intensive support service cases tend to be lower tariff cases. ISMS is only meant to be for the highest risk cases in Glasgow.
8.21 Nevertheless, given the difficulties that the DTZ evaluation has faced in acquiring outcome data, we believe that this is a useful additional source of this data despite these qualifications.
Effectiveness of ISMS/intensive support service
8.22 Figure 8.3 presents social workers' responses to how effective they felt that ISMS/intensive support services had been for the young person overall
Figure 8.3. The overall effectiveness of ISMS/intensive support service
Note: This is based on 16 ISMS cases and 7 intensive support only cases.
8.23 We can see from this analysis that, generally, both programmes are seen as being effective in the majority of cases, with ISMS having the higher perceived effectiveness among this sample of social workers. However, in sizeable minorities of cases, social workers viewed the programmes as ineffective. Many respondents qualified their 'ineffective' response by claiming it was the result of factors such as the young person's refusal to engage or other life circumstances getting in the way.
Perceived changes in behaviour by different behavioural types
8.24 Figure 8.4 considers perceived changes in the young people pre, during and post ISMS/intensive support service by three main categories of behavioural type.
Figure 8.4. Perceived changes in behaviour by different behavioural types
Note: Actual bases are: Offending ( ISMS 19, ISS 10); Alcohol use ( ISMS 19, ISS 10); Drug use ( ISMS 14, ISS 8).
8.25 Figure 8.4 demonstrates that most of the social workers interviewed believed that there had been improvements in the young people's offending behaviour after the programmes (this is consistent with the falls in actual offending uncovered by the GCC research team). However, the results were more mixed for alcohol and drug use, although more social workers thought that there had been improvement than the problem getting worse. Interestingly, this is a slightly different picture to that presented in Figure 8.3, where ISMS had a clear lead on views of effectiveness compared to intensive support service. In Figure 8.3, we were asking for overall views on effectiveness while in Figure 8.4 we are asking about types of improvements on actual cases, therefore these are different questions. It may also be the case that even if there was no change in behaviour or the young person got worse, the programme may still have been effective in others parts of the young person's behaviour, or the young person behaviour may have deteriorated even further without the programme.
Perceived changes in mental health
8.26 Figure 8.5 covers perceived changes in the young people's thoughts of or actual self-harming and other mental health issues.
Figure 8.5. Perceived changes in mental health
Note: Actual bases are: Thoughts of or actual self harming ( ISMS 6); Other mental health issues ( ISMS 7, ISS 6).
8.27 Based on this evidence, post- ISMS, there seemed to be reduced incidents of self-harming for a number in the sample, but the interesting result is the sizeable proportions of these of young people whose condition deteriorated after the programme. Their deterioration is unlikely to be as a result of the programme, instead being more likely to be part of a trend and/or more information coming to light about the mental health of the young person as they are monitored more closely while on the programme. We do not have the information to enable us to clarify whether ISMS/intensive support service helped to reduce the rate of this decline in these young people.
Perceived changes in attitudes towards others
8.28 Figure 8.6 looks at the changes in behaviour towards others pre, during and post the programmes. Although we do make some general comments in interpreting the analysis, we are careful not to go too far with this due to the small samples involved and the fact that we are looking at perceptions of change.
Figure 8.6. Perceived changes in attitudes towards others
Note: Actual bases are: Controlling their temper ( ISMS 18, ISS 10);
Getting on with family members ( ISMS 17, ISS 9); Dealing with officials ( ISMS 19, ISS 6).
8.29 Figure 8.6 suggests that attitudes towards others did improve for large proportions of the young people after the programmes in a number of ways, particularly for those on intensive support service. However, no change was again exhibited by a substantial minority of the sample.
Perceived changes in attitudes towards oneself
8.30 Figure 8.7 covers perceived changes in a number of attitudes towards themselves.
Figure 8.7. Perceived changes in attitudes towards oneself
Note: Actual bases are: Building self-confidence ( ISMS 10, ISS 8); Speaking their minds at panels/reviews ( ISMS 10); Feeling that there were people who would actually help ( ISMS 17, ISS 7); Feeling in control over their future ( ISMS 13, ISS 7); Taking responsibility for their own actions ( ISMS 18, ISS 9).
8.31 This figure shows that there were perceived changes in a sizeable number of cases across the different categories, particularly in building self-confidence. This was the case for those on both ISMS and intensive support services.
Perceived changes in attendance rates
8.32 Figure 8.8 breaks down perceived changes in attendance rates for various categories of appointments.
Figure 8.8. Perceived changes in attendance rates
Note: Actual bases are: Keeping appointments medical ( ISMS 11), Keeping appointments non medical
( ISMS 12, ISS 6); Attending education/employment ( ISMS 17, ISS 9); Attending court ( ISMS 7).
8.33 This figure, again, demonstrates noticeable improvement for the samples as a whole across a range of categories. Again, this was the case for both programmes. However, there was no perception of change for substantial proportions of the samples across all of these categories.
Summary of findings from the additional data
- ISMS tends to have a slightly longer period of Includem support than intensive support service cases and averages nearly one year and a half.
- Over 40% (65 cases) of ISMS and intensive support service cases combined admit to offences while on the programmes, especially those in the older age groups.
- Analysis of YLS data from a sample of cases seems to show little improvement in risk for young people before and after ISMS. However, none of the young people in the sample who have exited ISMS were in an increased overall risk category on final testing. Furthermore, none of the cases showed significantly increased risk, although there were a number of cases where the risk factor did increase by one category. Stabilisation of risk could also be seen as a positive outcome.
- Generally, social workers in Glasgow perceive both ISMS and intensive support service to be effective, particularly in improving offending behaviour, attitudes towards oneself and others and in attendance rates.
« Previous | Contents | Next »