The Scottish Government

« Previous | Contents | Next »

Listen

CHAPTER FIVE Survey of wider stakeholders

Overview of survey approach

5.1 To gain a more complete view of the ISMS process, a range of stakeholders were surveyed. These stakeholders represented a number of agencies and organisations that have or should have some involvement in ISMS, such as those working in Social Work, Education, Mental Health, local ASB coordinators, those working in procurator fiscal offices and Includem or NCH staff. It was not necessary to have had a direct involvement in ISMS to reply; we wanted to understand stakeholders' awareness and perceptions of ISMS as well as the nature of any involvement. The purpose of this survey was to ascertain the level and nature of involvement of wider stakeholders in the ISMS programme; and to identify their opinions of the programme, the way in which it was implemented and is being operated, as well as their views on its use and effectiveness. A copy of the survey questionnaire is provided in Appendix D.

5.2 The survey was initially distributed by DTZ in late February 2007, with a stamped-addressed envelope. DTZ worked with Research Resource, a company specialising in market research and surveys were returned to Research Resource for data inputting. Two weeks after the survey was issued, Research Resource sent out a reminder letter to all non-respondents. Research Resource then sent all of the returns on to DTZ for analysis in a SPSS database. A postal methodology was chosen as it allowed us to contact a wide range of stakeholders, drawing on contact details already held by the Scottish Executive. It also meant that respondents could complete the survey at work or at home, depending on the nature of their involvement, and take time to consider their response.

5.3 This survey was conducted in February-March 2007 and refers to views and opinions up to this time.

5.4 We contacted 385 individuals and received 181 completed surveys (a response rate of 47%). This is a good response for a survey of this type, as postal surveys usually have low response rates ⁴⁷ if the subject is not considered to be of interest to potential respondents - this indicates interest in and awareness of ISMS among stakeholder groups. The breakdown of respondents by professional group is shown in Figure 5.1 and by LA area in Figure 5.2.

Figure 5.1. Respondents by professional group

Note: base of 181.

Figure 5.2. Respondents by local authority area

Note: base of 181.

Involvement with ISMS

5.5 Of the 181 respondents, 117 (65%) had been in direct contact with an ISMS case, a further 61 individuals (34%) had not had direct contact with an ISMS case. The nature of involvement is discussed in more detail below.

5.6 The nature of the contact with the ISMS programme varied, but the majority of people have been involved in more than one stage of the process. Two-thirds of respondents (66%) had been involved in the case review process (reviews are undertaken at planned intervals or in response to non-compliance or other issues). Similarly, assessment and service delivery areas had large proportions of stakeholder involvement (60% and 59% respectively). It is interesting to note that only 37% of respondents had been involved in the aftercare element of the process (i.e. following removal of the MRC but before the programme has reached completion), although this may reflect the lack of cases in the early stages in some of the areas, meaning that there was little opportunity for other stakeholders to be involved by the time of the survey. The involvement levels across the LAs were generally reflected by the proportions shown in Figure 5.3. However, the involvement levels across all stages of the process in Dundee and Moray were significantly higher: in these regions, stakeholder participation averaged 62% and 66% respectively across all stages. In comparison, the overall average was 49% across all the LAs.

Figure 5.3. Nature of direct involvement in ISMS cases

Note: totals sum to more than 100% because multiple responses were possible. Proportions are expressed relative to all respondents who answered this question (116 out of 181 respondents)

Understanding of the reasons underpinning the introduction of the ISMS programme

5.7 The stakeholders were asked about their understanding of the underlying reasons for the ISMS programme. The most common response (given by 89% of respondents) was 'to help young people'. This was followed by 'to offer a cost effective alternative to secure accommodation' (cited by 72% of respondents). Another popular response (59% of respondents) was 'to control young people to protect communities'. Very few people (3%) cited 'to punish young people' as a reason for the ISMS programme. A full breakdown of responses is shown in Figure 5.4.

Figure 5.4 The reasoning behind the ISMS programme

Note: Totals sum to more than 100% as multiple responses were possible. Proportions are expressed relative to all respondents who answered this question (173 out of 181 respondents).

5.8 Stakeholders were supportive of the ISMS approach as they perceive it; with almost three-quarters (74%) stating that they either agree or strongly agree with the approach underpinning the programme. Only 6% stated that they disagreed with the approach.

Figure 5.5. Do you agree or disagree with the ISMS approach?

Note: proportions are expressed relative to all respondents who answered this question (172 out of 181).

Effectiveness of the referral process

5.9 A significant proportion of the respondents (43%) believe that the ISMS referral process is effective. Just over one-quarter (27%) of respondents said that they did not know whether the system was effective and a further 16% viewed the system as neither effective nor ineffective. This may well simply reflect a lack of direct experience of this stage.

Figure 5.6. Is the ISMS referral process effective?

Note: proportions are expressed relative to all respondents who answered this question (172 out of 181).

Effectiveness of the assessment process

5.10 The stakeholder views on the assessment process told a similar story, with almost one-third of respondents (31%) not knowing whether the assessment process was effective. Again, this probably reflects a limited number of respondents having experience of this stage, particularly given that 40% of respondents said that had not been directly involved in the assessment stage of an ISMS case (see Figure 5.3).

5.11 37% thought that the assessment process was effective, with a further 17% viewing the process as very effective. Those LAs who found the assessment process most effective were in Dundee and Moray, with 35% and 33% respectively viewing the assessment process as very effective. In Dundee the comments from respondents indicate that the success of the assessment process is the result of expert leadership that links with other agencies well, and allows an in-depth bespoke assessment to be carried out. Respondents in Moray indicated that they feel the strength of the assessment process is in its intensity, which allows issues to be identified but also provides a clear indication of whether ISMS will work and can sometimes bring about improvement in behaviour in its own right.

Figure 5.7. How effective is the ISMS assessment process?

Note: proportions are expressed relative to all respondents who answered this question (173 out of 181).

5.12 A large proportion of stakeholders (42%) did not express a view on the appropriate length of the assessment process. Of those that did express a view, most thought the length of the process should stay the same (41%). Relatively few thought that it should be extended and only a small number supported it being reduced.

Figure 5.8. Should the assessment process be…..

Note: proportions are expressed relative to all respondents who answered this question (169 out of 181).

Service delivery

5.13 The crisis response ⁴⁸ mechanism was considered by 84% of respondents to be either effective or very effective. This was closely followed by the providers Includem/ NCH, which 80% of respondents deemed to be either effective or very effective. Accommodation services were viewed by 28% of respondents to be neither ineffective nor very ineffective and a large proportion of respondents (38%) viewed it as neither effective nor ineffective. Employment services, mental health services and addiction services were the other parts of the programme that were rated by less than 50% of respondents as being effective. An overview of all the responses is shown in Figure 5.9.

Figure 5.9. Effectiveness of elements of the ISMS package

Note 1: Proportions are expressed relative to all respondents who indicated a measure of effectiveness for a particular element. The exact respondent base varies for each element, however the smallest sample is 88, and is still a statistically robust sample.

Note 2: Actual base numbers for each of the above categories are: Crisis response (121); Includem/ NCH (115); Assisting family/carer (124); Offending behavior programmes (115); Respite and time-out (97); Monitoring service (119); Education (134); Training services (98); Addiction services (106); Mental health services (97); Employment services (88); Accommodation services (95).

5.14 Just over half of the respondents (51%) believed that the ISMS process could be improved, with only a small proportion (7%) claiming it could not. The remainder did not know. Suggestions for improving the programme varied, below are some of the suggestions given.

• Improved inter-agency working and improved communication between the agencies
• Introduce consequences for breaching MRC
• Assessment period should be lengthened
• More work with the families of young people on MRC
• Focus on the retention of intensive support staff should be improved and staff turnover kept to a minimum
• Clarify who the scheme is for and more guidance

5.15 There were also a number of areas identified where stakeholders felt that there are some unmet needs that ISMS should address. A selection of quotes that reflect the most commonly cited responses are given below.

• Accommodation and addiction services are poor and over-stretched
• More effective links with mental health agencies and alcohol/substance misuse agencies
• Improve mental health provision
• Early referral not possible and reporting period too long
• Not addressing child protection issues within the family home
• Limited weekend support - this is when most needed

Good practice

5.16 Stakeholders were asked whether there were any examples of good practice in the provision of ISMS. The most commonly cited responses are below. Some of these coincide with the responses given for areas of improvement, partly due to differences in service organisation and delivery across the LAs.

• Enthusiastic staff
• Cognitive programmes and mental health services
• Communication and sharing of information between agencies
• Educating offenders
• Includem and crisis response
• Ethos of supporting young people without removing them from the local community

5.17 The length of time that the stakeholders expected young people to take to engage with ISMS varied from one to two weeks to up to one year. Interestingly, a number of stakeholders expected an immediate change in behaviour as a result of ISMS. The length of time that stakeholders expected before changes in behaviour would occur ranged from 0 weeks to 12 months, with the most common response being 12 weeks.

Inter-agency working

5.18 Stakeholders were asked about their relationships with other organisations - how frequently they work together and how effective they feel contact to be. Most of the stakeholders had some contact with the ISMS Team and (of all the other partners within the ISMS programme), this was a partner that most stakeholders (64%) worked closely with. Relationships with social workers also formed a key role in the stakeholder relationships.

5.19 Few stakeholders felt that they had a close relationship with some of the service based agencies, particularly employment, addiction and mental health services, and this was cited as in need of improvement. Interestingly, these are the agencies with some of the lowest scores for effectiveness and this perceived lack of effectiveness may be related to lack of close working. However, it is reassuring to see that around half of the respondents had contact with the ISMS Team, Social Work, Includem/ NCH and Education.

Figure 5.10 Stakeholders level of involvement with service providers

Note 1: proportions are expressed relative to all respondents who provided an answer to the relationship with the agencies. The exact respondent base varies for each question and the lowest base is 111, which is still deemed to be a robust sample.

Note 2: Actual base numbers for each of the above categories are: ISMS team (122); Social Work (120); Includem/ NCH (111); Education (121); Addiction services (119); Serco (117); Mental Health Services (120); Training services (119); Employment services (120).

5.20 Most stakeholders that had a relationship with agencies felt that it was an effective one: in fact, 86% of respondents who had a relationship with the ISMS Team viewed it as effective or very effective. It is also reassuring to see that, for all agencies, the number of negative responses was relatively low. The highest figure for ineffective relationships was for mental health services, where 19% of respondents stated that their relationship with them was ineffective.

5.21 A complete breakdown of the effectiveness of stakeholder relationships with service providers is shown in Figure 5.11. Again, the core services score relatively highly and it is what may be perceived as the more peripheral parts of the ISMS programme that score less well. This may be because these services are only used occasionally (depending on the needs of the young person on the programme), but the fact that relatively large proportions of stakeholders view these services as ineffective and the partnership relationships as ineffective is a source of concern.

Figure 5.11. Effectiveness of stakeholder relationships with service providers

Note 1: proportions are expressed relative to all respondents who stated their level of involvement with a particular service provider. The exact respondent base varies for each question the lowest base is 62, which is still deemed to be a robust sample.

Note 2: Actual base numbers for each of the above categories are: ISMS team (115); Social Work (117); Includem/ NCH (105); Education (102); Serco (62); Training services (85); Addiction services (90); Mental Health Services (86); Employment services (78).

5.22 Some suggestions to improve the relationships with agencies included the following.

• A single point of contact
• More information sharing
• Increased accountability
• More communication/better communication structures
• Working outside office hours
• Improved holiday/sick cover
• Interagency training
• Dedicated resources
• Using the same assessment tools

5.23 Figure 5.12 shows how opinion on the effectiveness of the ISMS team and the Includem/ NCH team changes when contact moves from close contact to occasional contact. Those respondents who had close contact with ISMS and Includem/ NCH almost entirely stated that they had an effective working relationship with them, 99% and 95% respectively. This declined to 60% and 56% for ISMS and Includem/ NCH respectively when the frequency of contact was reduced to occasional. This would seem to imply that it is the closeness of contact that has an influence on the effectiveness rating, although we cannot determine the cause and effect here, i.e. we cannot say whether the working relationship becomes closer because it is more effective or it becomes more effective because it is closer.

Figure 5.12. Close contact with ISMS Team and Includem/ NCH

Note 1: proportions are expressed relative to all respondents who provided an answer to having either a close or occasional relationship with the agencies.

Note 2: Actual base numbers for each of the above categories are: ISMS Close (79); ISMS Occasional (32); Includem/ NCH Close (55); Includem/ NCH Occasional (33).

5.24 A similar pattern emerges for mental health services, where 71% of those who work closely with them consider them to be either effective or very effective, falling to just 26% of those who have only occasional contact.

Comparisons with other measures

5.25 In comparison to the measures used by the Children's Hearings System ⁴⁹, ISMS was generally viewed as at least as effective as these measures. ISMS was viewed as more effective than home supervision by 71% of the participants. ISMS also compared favourably with secure accommodation; accommodation other than secure, and intensive support (without MRC). Compared to all three of these other measures, at least half of the respondents said ISMS was more/much more effective.

Figure 5.13. ISMS compared to the measures in the Children's Hearings System

Note 1: proportions are expressed relative to all respondents who stated their level of involvement with a particular service provider. The exact respondent base varies for each question the lowest base is 113, which is still deemed to be a robust sample.

Note 2: Actual base numbers for each of the above categories are: Home supervision (124); Other accommodation (119); Secure accommodation (115); Intensive support only (113).

5.26 A similar story emerged when comparing ISMS to the adult system, again with at least half of the participants viewing ISMS as more or much more effective than the measures under the adult system. In particular, 86% of respondents said that ISMS is at least as effective as a Supervised Attendance Order and 84% said that ISMS is at least as effective as a Probation Order. About one-quarter of respondents felt that ISMS was less effective than a fine (26%), a Community Service Order (22%), a custodial sentence (27%), or a Restriction of Liberty Order (26%). However, on the whole, ISMS compared positively with other adult disposals.

Figure 5.14. ISMS compared to the adult system

Note 1: proportions are expressed relative to all respondents who stated their level of involvement with a particular service provider. The exact respondent base varies for each question the lowest base is 59, which is still deemed to be a robust sample.

Note 2: Actual base numbers for each of the above categories are: Fine (62); Probation Order (64); Community Service Order (64); Custodial sentence (64); Antisocial Behaviour Order (65); Supervised Attendance Order (59); Restriction of Liberty Order (62).

Support after the Movement Restriction Condition has been removed (Post- MRC support)

5.27 The vast majority of stakeholders (93%) viewed the post- MRC support as important or very important, with only a small minority (2%) viewing it as not very or not at all important.

5.28 Over half of the respondents (60%) thought that the post- MRC element should remain in place for the same length of time as is currently the case, while 39% thought the period of post- MRC support should be extended. The length of time that post MRC support is in place varies across the phase 1 LAs and individual cases, but is generally in place for the same amount of time as the MRC has been in place. Only 1% thought that this period should be shortened.

Training and guidance

5.29 Just over half (53%) of the stakeholders received training, meaning that a large proportion of stakeholders had received no ISMS training. The reasons for this were not detailed in the survey. The proportion that had received no training was lowest in Edinburgh (19%) and highest in Glasgow (57%). Of those responding to the survey, none of those working in secure accommodation or children's units; legal professions; and ASB services had received any ISMS training, while almost all respondents from SCRA (91%) had.

5.30 The percentage of respondents who had received training, by type of training, is shown in Figure 5.15. Over two-thirds of these respondents (69%) received reading material on ISMS, a similar proportion (66%) had attended a presentation on ISMS. Only one in ten of those who had received training (11%) attended a formal training course on ISMS.

Figure 5.15. Type of training received

Note: totals may sum to more than 100% because multiple responses are possible. Proportions are expressed relative to all respondents who said that they had received training or guidance on ISMS and then went on to describe the nature of that training (90 out of 181 respondents).

5.31 Of those who received training, 93% of respondents said that this training was either useful or very useful and only 2% viewed the training as not useful. Strengths of the training that were highlighted are given below.

• Answering questions
• Giving a good insight into the process and the aims and objectives of the service
• Peer support
• Clear explanation
• Context clearly set/good background
• The information suited my needs
• Allowed us to see how it works and why it works
• Comprehensive
• Directly related to practical issues
• Carried out in parallel with the process.

5.32 Reasons given for the training not being useful included:

• Some of the ISMS criteria changed and it was difficult to learn the correct legislation
• The training did not give practical advice
• No local focus to service provision
• Material not fully understood
• More formal training would have been helpful
• Unaware that training was available
• Would like to have more knowledge of how it works for those who are actually tagged.

Strengths and weaknesses of ISMS

Main strengths of the ISMS programme

5.33 Stakeholders were asked about the main strengths of the ISMS programme, the key themes from the responses are given in the list below.

• A good Youth Justice System to implement it
• Ability to resource cases at short notice
• Adapts to the individual
• Alternative to secure accommodation
• Builds relationships with young people
• Committed staff and excellent communications
• 24/7 support
• Continuity of care plan with regular reviews and a multi-disciplinary approach
• Direct support to the family and young person at the time of crisis
• Intensity of support and consequent amount of time spent with the young person
• Provides the young people with a caring environment where they can feel safe and provides them with social skills that they may have missed out on
• Encourages young people to take responsibility
• Tagging element
• Collaborative inter-agency approach
• Young people are usually keen to take part as it is an alternative to secure accommodation

5.34 The responses above are not exhaustive, but the key themes that emerge are that most respondents cited ISMS as a good alternative to or last chance before secure accommodation, and that ISMS benefited from good multi-agency working.

Main weaknesses of the ISMS programme

5.35 Some of the comments below may contradict the strengths stated above, and this reflects differences in opinion between stakeholders. The most common concerns that emerged over ISMS were in relation to its costs and the number of individuals that the young person had to maintain relationships with.

• Can result in too many people (agencies) in the young person's life
• Capacity to provide education/training during the day is limited
• Consistency of approach
• Costs, time and money
• No evidence of long-term behaviour change
• Seen as a badge of honour
• Not an effective deterrent
• Electronic monitoring at odds with welfare principles
• Success rate too low
• Lack of clarity from the Scottish Executive in guidance
• Lack of communication
• Only as effective as the young person involved wants it to be
• Staff turnover
• Training and level of knowledge of social workers too low

Summary of findings from the wider stakeholders' survey

• Two-thirds of respondents had been involved in an ISMS case review, so there was a reasonable level of general knowledge and experience working with the programme in the sample.
• Generally, there was strong support for the ISMS programme among the stakeholder group as a way of helping young people and their communities as well as offering a cost effective alternative to secure accommodation.
• Many of the key parts of the ISMS programme were perceived as effective, especially crisis response and Includem/ NCH. Accommodation, employment services, mental health services and addictions services all scored relatively lowly in terms of perceptions of effectiveness. This may be related to actual ineffective service provision, but other evidence suggests a correlation between the closeness of the working relationship and perception of effectiveness.
• Broadly, this also corresponds with views on the effectiveness of partnership working between the different agencies.
• In terms of perceived effectiveness, ISMS compares favourably among the stakeholders with alternative measures within the Children's Hearting System and the adult system.
• Just under half of all the stakeholders who responded said that they had received no ISMS training. Very few had been on a formal training programme. Most training had been done by provision of reading material and presentations. The training that had been provided was well received.

« Previous | Contents | Next »