Scottish Government

THE NATIONAL EVALUATION OF THE CAREERS SCOTLAND INCLUSIVENESS PROJECTS

ANNEX 3 SURVEY OF PARTNERS

Introduction

3.1 The purpose of the survey of partners is to provide a fuller understanding of the levels of awareness and influence of the Inclusiveness projects at a more strategic and operational level within a wide range of partner organisations. It complements the other evaluation elements, including the case study evaluations and the additional project survey.

3.2 The project surveys provided a comprehensive list of partner organisations. A partner contact database was developed on the basis of this list. Partners contacted as part of the case study work were excluded from the survey. Senior personnel within each partner organisation were identified and telephone interviews were conducted with 30 partner organisations. These included representatives from local authority departments, voluntary sector organisations, further education, local enterprise companies, central government ²³, and one private training provider. We selected two partner organisations from each Careers Scotland (SEn) area ²⁴. In view of the different administrative arrangements across the Highlands and Islands area we conducted an additional eight interviews to ensure that each Local Authority area would be represented.

3.3 Table A3.1 below provides a breakdown of the different organisational type of each partner contacted.

Table A3.1: Interviewees by organisational type

3.4 The vast majority of interviews were held with Chief Officers, Managers, Principal Psychiatrists and Project Co-ordinators. All but the latter have a wide range of responsibilities, of which employability issues are only one of many. In two voluntary sector cases, we spoke with project workers as they were the individuals with the links to the Inclusiveness projects.

Partner Background

3.5 In terms of client group, all partners interviewed had responsibility for young people between the ages of 16 and 25 as part of their wider client group. They shared at least one client group in common with the Inclusiveness projects. These included:

• looked after children
• young homeless people between 16 and 25
• young offenders between 16 and 25
• young people between 16 and 25 with physical disabilities
• young people between 16 and 25 with learning/emotional difficulties
• young people between 16 and 25 with mental health difficulties
• young people between 16 and 25 with addiction issues

3.6 In terms of cross organisational networking, it is worth noting that all respondents attended one or more multi-agency network that considers major strategic issues for young people between the ages of 16 and 25. These networks were often linked to the partner's particular client group. Box 1 below provides an example of the diverse array of networks that two partners interviewed attend.

Box 1

The Principal Educational Psychiatrist for West Dunbartonshire Council sits on the Children's Services Planning Group, the Early Years Multi-Agency Forum, the Joint Assessment Team Co-ordinators for Secondary Schools Group and the Positive Futures Steering Group. Employability is part of the agenda of the latter two. The Links Centre Manager for the Scottish Prison Services, Lanarkshire, is on the Scottish Offenders Employment Forum for Forth Valley. Employability is clearly a key element of this Forum. Forum attendees include JobCentre Plus, Scottish Enterprise, APEX Scotland and the Scottish Prison Services

3.7 The findings indicate that there are a wide range of networks in each Careers Scotland area. While some of these may have a narrow focus, many others have a wider focus of which employability can be a part. This would suggest that when Careers Scotland is considering the development of new networks or other initiatives aimed at influencing partners, account must be taken of existing networks. This will avoid duplication and people feeling 'over-networked'.

Relationships with the Inclusiveness Projects

3.8 All partners interviewed were aware of the Inclusiveness projects in their area. We used the locally employed term when conducting the interviews. For example, in Ayrshire we referred to the Positive Futures project, whereas in Forth Valley we referred to The In Team project. In two cases, the interviewee was not aware of the project by its official name but when we mentioned Key Workers or Individual Action Plans they stated that they did know of the project.

3.9 It is interesting to note that in terms of partner descriptions of the projects all respondents were consistent in describing the project in their area as about improving access to employment, education and training for young people finding transition difficult. However, only one respondent referred to the project purpose as being about better linking of services, despite the fact a specific purpose of the project is better linking of services. This indicates the need for projects to be clearer about their purpose from the outset. The pie chart (Figure A3.1) below reveals the extent to which partners felt that the Inclusiveness projects were relevant to their work.

Figure A3.1: Relevance of Inclusiveness Project for the Partner Organisation

3.10 In terms of relevance, the vast majority of partners felt that the Inclusiveness projects were relevant to their own organisation's work. Only one partner felt that the project was not particularly relevant to what they did and none felt that the projects were not relevant.

3.11 In terms of partner involvement with the project, the majority of partners were involved at both strategic and operational levels ²⁵. This is clearly an important finding. Partner engagement at both levels can facilitate learning, information sharing and relationship building. It is also worth noting that even where partners had no involvement with the Inclusiveness projects, they were aware of the concepts of Key Workers and Individual Action Plans. This illustrates the extent to which the Inclusiveness approach has gained credence.

3.12 Table A3.2 highlights the different involvement of different partners in the Inclusiveness projects.

Table A3.2: Respondent Partner's Involvement with the Inclusiveness Projects

Impact and Service Change

Impact on partners' systems and service provision

3.13 Figure A3.2 below highlights the different responses from partners in terms of service delivery impact.

Figure A3.2: Impact on Service Delivery

3.14 Thus, in terms of partners improving service delivery as a result of their links with the Inclusiveness projects, the greatest impact was on the following (numbers of respondents expressing each opinion provided in brackets):

• greater awareness of client needs (17)
• increased emphasis on employability issues, such as the need for better preparation for employment e.g. CV and interview skills, relevant training etc. (13)
• changed relationships with existing partners (15)
• enhanced relationships with new partners (12).

3.15 In addition to the four improvements highlighted above, a few partners observed the following benefits:

• the benefits of having an additional resource (financial and personnel) to ensure better support to young people's employability needs
• sharing of information and resources (financial and technical) to better support young people
• increased access to most vulnerable young people, through contacts made by the Key Workers

3.16 These findings are important as the key objectives of the Inclusiveness projects include:

• increasing service providers' employability focus for young people
• improved linking of services

3.17 Thus important steps have been taken to meeting these objectives: service providers now have an increased employability focus and different services have new and improved relationships (see Box 2 below).

Box 2: One partner's impression as to the improved linking of services:

"It (the project) has been very worthwhile and very successful. It has changed the culture and improved partnership work in our area. Before there were lots of people doing similar jobs, people were very protective, now we can see the benefits of shared work and the different support that each of us gives young people".

3.18 It is worth noting that the most significant improvement in relationships with particular partners was the relationship with Careers Scotland itself. This was specifically noted by 7 respondents. These respondents felt that they now had a better knowledge of Careers Scotland as an organisation and that they could now engage more directly with Careers Scotland through the Key Workers. Relationships between interviewee partners with new partners included with JobCentre Plus, with Scottish Enterprise, with Training providers and between council departments.

3.19 In terms of the development of new systems or procedures, the majority of respondents had not developed any formal procedures or tools as a result of their links with the Inclusiveness projects.

3.20 There were 7 examples of formalised systems being put into place. These include:

• inter agency referral protocol
• case level reviews joint referral mechanism
• information sharing protocols
• APSET: a joint assessment tool
• Rickter scale
• Future Education and Training Series
• vocational profiling: this has since been rolled out across the partner organisation (LEAD Scotland)

3.21 The absence of formal developments in terms of tools and processes is slightly worrying. Much progress has been made through the Projects but there is a risk that if this is not formalised it might be lost. Much of the progress would also appear to be dependent upon the individuals involved, particularly at an operational level. Sometimes it was apparent that when individuals changed roles joint working could suffer. The survey would suggest that there is a need to plan more effectively in terms of influencing the provision of services by partners involved in pilot schemes such as this one.

3.22 It should also be noted that a fifth of partners report no influence upon the work they do. There were also some vigorously asserted criticisms of the project. A small number of partners did not feel there had been an improvement in their links with other service providers. Several respondents cited the top-down and increasingly centralised nature of the Projects as a factor (see Box 3 below).

Box 3

"(The Project) was just another layer onto an already existing network, it did not build on existing multi-agency links. There have been difficulties with info sharing and joint tracking".

"The concept is right, but it was not well thought out. It was fraught with difficulties from the start. There was a short timescale for setting it up, for getting a shared vision. It was directed from the top with no local negotiation. It was a top down exercise."

Impact on working with clients

3.23 As a result of the influence of the Project, partners noted some concrete impacts for their clients (See Box 4 below). Benefits cited for clients include:

• Better one to one support for clients (16):
  • person-centred, intensive support that other services do not have capacity/expertise to provide
  • informality of the key workers' outreach work ensures that those too intimidated to contact service providers are reached
  • involvement of families through Key Worker facilitates the support of young people
  • improved self esteem and confidence
  • increased opportunities to get information on employability issues, including funding for courses
• Smoother transition for clients between services (17):
  • Key Worker as single referral point improves seamless progression to public and voluntary sector services
  • more integrated approach to the wider problems the young people face in accessing employment/education/training, including addiction, homelessness, health issues
  • greater continuity for clients, better sharing and transfer of information between service providers.

Box 4

"It has made a valuable contribution. As a small authority we are aware of the individuals whose lives have significantly improved through the project"

"It has been invaluable. If it continues, it can be built on and not just sustained. It has been a very important development and done very, very effective work with young people."

3.24 A negative impact was also noted by a small number of respondents. This relates to client group focus. One partner felt that young people with very limited employment potential, such as those with severe learning difficulties, were now less supported by Careers Scotland than was previously the case. Another respondent also felt that the needs of young people with mental health issues were being neglected as a result of the Inclusiveness projects.

Sustainability and Mainstreaming Issues

3.25 Partners found the question of the sustainability of the benefits of the Inclusiveness project a difficult one to answer. The vast majority of partners felt that without the added input of the Key Workers they would be unable to sustain the level of support provided, particularly in terms of outreach work and specific employability knowledge.

Mainstreaming of Learning

3.26 Respondents were encouraged to explore the impact the Inclusiveness projects may have had for their wider organisation. Most partners felt that within their organisations, all those who should be aware of the project were. Almost all did feel, however, that more could be done in terms of considering the lessons from the Inclusiveness projects within their own organisation.

3.27 We went on to ask respondents about how learning was shared within their organisation. For the majority of partners this has taken place through day to day operational work with Key Workers and through the sharing of information at strategic meetings. Another key source of sharing learning has been through the holding of conferences, seminars and other events.

3.28 The fact that the majority of partners were engaged in learning is an important accomplishment and indicates commitment by the Inclusiveness Co-ordinators and by partners to shared learning. It will be important to build on this success in the future.

3.29 It is also important to note, however, that eight partners felt that they had not received any sharing of learning but there is nothing to indicate that this figure cannot be improved in the future. Partners were constructive in making recommendations as to how to improve learning in the future. Overall, the survey would suggest that systems for disseminating learning from the Projects could have been improved, through more proactive forms of dissemination and through measuring and reporting client progress in terms of 'softer' or 'distance travelled' indicators.

Main barriers to learning

3.30 The survey went on to explore barriers to learning. The three main barriers identified were:

• competing priorities for resources (14)
• lack of resources in terms of qualified people (14)
• lack of dissemination of learning (14).

3.31 Three other barriers were also identified by individuals:

• strict internal policies and procedures make it difficult to make changes as a result of learning
• data protection legislation can be used to avoid sharing information
• attitudinal barriers- for example schools see their role as over once a young person is over 16 and yet it is important to plan for this transition in the curriculum.

3.32 The respondents were able to suggest a number of ways of overcoming these barriers. Suggestions included:

• more resources earmarked for learning purposes
• better dissemination of learning, with more emphasis on face to face activity
• more influencing at the senior level with those in decision making positions
• more joint training which provides a platform for the dissemination of learning and practice
• formalising relationships and the development of joint tools.

3.33 These are practical, attainable recommendations which would be worth taking on board in future planning.

Conclusions

3.34 Partners on the whole have been extremely positive about their experiences with the Inclusiveness projects and feel them to be a valuable and fundamental contribution to addressing the employability needs of young people in the transition between education and work or training. The survey identifies a broad range of opinion and experience. Certain patterns do emerge however:

• there is a good level of awareness as to the work of the Inclusiveness projects and in most instances the work of the Projects is seen as being very relevant to the work of the partners
• there is a considerable degree of inter-agency networking at strategic and operational level in most areas
• there has been an impact on partner service delivery, particularly in terms of an increased emphasis on employability issues
• improved links between service providers are also evident
• there have been important benefits for clients but there is some concern that non-priority groups may be neglected by Careers Scotland
• there has been some mainstreaming of learning, particularly in terms of client needs and the value of better linked up services provision but more emphasis needs to be placed on shared learning at the outset of new initiatives and to becoming 'smarter' in how learning is promoted.