The Scottish Government

« Previous | Contents | Next »

Listen

CHAPTER FIVE IMPACT ON BENEFICIARIES

5.1 Whilst ERP has not had the desired level of impact on SLDR figures, the study did find that the pilot has had a number of positive benefits for the school pupils who received support. This chapter examines SLDR data, sustainability data and findings from our telephone surveys with ERP pupils and their parents to explore the impact that the intervention has had on beneficiaries. It should be noted that no control group was involved, therefore, it is unclear to what extent any reported impacts would have happened anyway in the absence of the intervention.

5.2 The telephone survey was first carried out in May 2007 with 81 ERP pupils who had received support in the 2006/7 academic year and 83 parents of ERP pupils. In almost all cases interviews were carried out with parents of the pupils who were interviewed, however, on occasion a pupil was not available or their parent was not available, therefore only one or other was surveyed. The survey was repeated in May 2008 with 86 ERP pupils who had received support in the 2007/8 academic year and 91 parents of ERP pupils. For the purpose of this report, survey analysis focuses primarily on the findings of the year 2 survey as the pilot was more fully embedded in the school structures by this time. Findings that vary from those reported in year one are highlighted in the text. However, caution should be exercised when comparing the results of the 2 years due to the self selection bias which is inevitable within this type of research.

5.3 The chapter explores:

• Post-school destinations of those receiving support
• Sustainability of positive destinations
• Impact on expectations and aspirations
• Changes in attitudes and behaviour
• Personal development.

Destinations of beneficiaries

5.4 During the pilot ERP Advisers worked with ⁷:

• 686 pupils in 2006/7, of whom 246 returned to school, 436 left school, 4 moved out with Scotland
• 561 new pupils in 2007/8, of whom 237 returned to school, 321 left school, 3 moved out with Scotland
• A total of 1247 individual pupils over the 2 years of the pilot.

5.5 In addition to working with the 561 new ERP pupils in 2007/8, ERP Advisers continued to work with many of the 246 pupils who had received support in 2006/7 and had returned to school in 2007/8. Of these 246 pupils, 168 left school in 2007/8, meaning there were a total of 489 school leavers in 2007/8.

5.6 Figure 5.1 illustrates the nature of initial destinations achieved by ERP participants in each year of the pilot. The graph shows that:

• In year one of the pilot 75% of school leavers who had received ERP support entered a positive destination, 24% entered a negative destination and for 1% their destination is unknown
• In year 2 of the pilot 65% of school leavers who had received ERP support entered a positive destination, 32% entered a negative destination and for 3% their destination is unknown.

5.7 In the second year of the pilot there was reduction in the proportion of beneficiaries moving in to further education, employment and training, and an increase in the proportion becoming unemployed upon leaving school. The reduction in pupils moving into employment could reflect the current economic climate, whereby unemployment is rising. As far as we are aware there has been no reduction in training or further education provision in the pilot areas. The change could be explained by there being a different cohort of pupils in year 2, with ERP Advisers perhaps working more with pupils in need of additional support.

Source: 2006/7 ERP Sustainability Data, N=436, 2007/8 ERPSLDR Data, N=489

Notes to figure: Data refers to ERP participants from the 13 pilot schools plus 2 additional schools with ERP provision in 2006/7 and 3 additional schools in 2007/8

Sustainability of destinations

5.8 Careers Scotland tracks ERP participants at 3, 6, 9 and 12 month intervals to understand the sustainability of positive destinations achieved following participation in the pilot. Figures 5.2 to 5.4 show the sustainability of positive destinations for 3 tranches of ERP participants (those who left in December 2006, May 2007, and December 2007). For those leaving in December 2006 and for those leaving in May 2007 data is available for a 12 month period, but for those leaving in May 2007 data for the 9 month tracking point was not available. For those leaving in December 2007, data is available for a 9 month period, and data on their initial destination was not available.

5.9 In all 3 cases a peak is achieved at the 3 month tracking stage (82% among December 2006 leavers, 81% among May 2007 leavers and 59% among December 2007 leavers). Following this a decline in the percentage of participants in positive destinations is observed, with figures for the first 2 groups falling to 77% at the 6 month tracking stage and figures for the third group falling to 52%. This decline continues with 71% of December 2006 leavers and 66% of May 2007 leavers sustaining a positive destination for one year.

Notes to figure: Data refers to ERP participants from the 13 pilot schools plus 2 additional schools with ERP provision

Source: ERP Sustainability Data, Careers Scotland, N=171

Notes to figure: Data refers to ERP participants from the 13 pilot schools plus 2 additional schools with ERP provision

Source: ERP Sustainability Data, Careers Scotland, N=273

Notes to figure: Data refers to ERP participants from the 13 pilot schools plus 3 additional schools with ERP provision

Source: ERP Sustainability Data, Careers Scotland

5.10 Results for those in the December 2007 group show a much lower proportion of positive destinations than for the other 2 groups. Although the data is affected by a much higher number of leavers in the "unknown" category (which decreases the proportion of positive destinations), if the unknown leavers had a similar distribution as others, the proportion of positive outcomes would still be just 68% at 3 months, falling to 64% at 6 months and 58% at 9 months.

5.11 It can be concluded that there is usually a high proportion of positive outcomes 3 months after leaving school, with, on average, over three quarters of leavers settling in a positive destination at this time. However, this proportion decreases gradually, and after one year an average of two thirds of ERP leavers sustain a positive outcome.

Impact on expectations

5.12 In the telephone survey we explored with pupils the extent to which ERP support had influenced their intended destination. We found clear evidence that the ERP intervention has led to a positive change in anticipated outcomes among ERP participants. This impact appears to have been greater in relation to participants involved in the second year of the pilot. This could reflect refinements in the referral processes, with the focus shifting to working with pupils who are still attending school. It could also be due to ERP Advisers having developed their skills over the duration of the pilot.

5.13 Figure 5.5 illustrates the destinations that respondents expected to achieve before and after engagement with the ERP Adviser. Prior to receiving ERP support, 28% of pupils had either no expectations about their future career (23%) or expected to become unemployed (5%). ERP support has had a significant impact on the expectations of these groups. The proportion of those with no expectations fell by 16 percentage points to 7% following support. Similarly, none of the 5% of pupils who had previously expected to become unemployed had this expectation after receiving support.

5.14 Among year one survey respondents, the percentage of pupils with no expectations for their future career decreased from 14% to 3% following engagement with the ERP Adviser. None of the respondents to the year one survey anticipated being unemployed either before or after involvement in the pilot.

5.15 The largest percentage of year 2 respondents (42%) had expected that they would move on to college or university following completion of their school education. After working with the ERP Adviser, this figure increased by 13 percentage points, with 55% expecting to go to college or university after receiving support. This is compared to 54% and 59% respectively of survey respondents in year one.

5.16 A similar pattern is observed in the analysis of responses from parents (Figure 5.6). The pattern highlighted by Figure 5.6 is broadly in line with the responses of parents interviewed in year one.

Notes to figure: % do not equal 100 due to multiple answers
Source: 2008 Research Resource Telephone Survey, N=86

Source: 2008 Research Resource Telephone Survey, N=91

5.17 Further analysis of the survey responses of these key transition groups (those who now expect to move on to college or university and those who previously had no expectations for their post-school career in the year 2 survey) is provided in Figure 5.7. It shows that among those who now expect to go to college or university, the largest percentage (74% (34 pupils)) had expected to do this prior to their engagement with the ERP Adviser. Thirteen per cent (6 pupils) have changed their expectations from one positive destination to another following their participation in ERP.

5.18 More significantly, 9% of those who now expect to move on to college or university had no expectations for their future prior to working with the ERP Adviser and a further 4% had expected to be unemployed after leaving school. This equates to 4 and 2 pupils respectively.

Source: 2008 Research Resource Telephone Survey, N=46

5.19 Figure 5.8 shows the anticipated career path of those pupils who indicated that they had "no expectations" for their future prior to participation in the pilot. In 30% of cases involvement in ERP has not made a significant difference to the aspirations of this group of pupils, however, the remaining 70% indicated that they now expect to enter a positive destination after leaving school. This is evidence of the significant positive impact of ERP on those with no expectations.

5.20 Throughout the evaluation, pupil's expectations about their post-school destination was never mentioned as a factor that was considered when making a referral to ERP. This omission could mean that some pupils, for whom ERP could make a difference, are being missed.

Source: 2008 Research Resource Telephone Survey, N=20

5.21 In the telephone survey we asked pupils whose expectations of their future career options had changed as a result of working with the ERP Adviser, why this has been the case. We found that:

• 49% feel that the ERP Adviser encouraged them to have greater expectations of themselves
• 17% feel that the ERP Adviser helped them to set realistic expectations
• A further 5% could not say why their expectations had changed as a result of working with the ERP Adviser.

5.22 These findings were reinforced in focus groups with pupils and by anecdotal evidence gathered in interviews with school staff. The ERP Advisers can be very effective in assisting those who are uncertain about their post-school career to identify their interests and strengths, narrow their options and focus their thinking to achieve a positive destination. Pupils and teachers alike provided examples of how the ERP Adviser has helped in this way. One teacher also said they were "astounded" that some of the pilot participants have achieved positive destinations.

5.23 The senior management contacts within the schools reported that one of the most significant impacts of ERP has been raised awareness of the wide variety of options that are available to pupils after leaving school. Many pupils highlighted this as the most useful aspect of their work with the ERP Advisers and something that is often not covered comprehensively by class and pastoral care teachers. This finding emphasises the importance of the specialist knowledge of ERP Advisers in assisting vulnerable pupils with careers guidance.

5.24 This role is also highly valued by staff in some of the pilot schools, many of whom were not fully aware of the wide variety of post-school options out with further and higher education. One teacher went as far as to attribute a change in awareness and attitude among school staff to the information and awareness raising work facilitated by the ERP Adviser.

5.25 One school commented on the role of ERP Advisers in managing the expectations of young people. As an example they described the case of one pupil who had decided to leave school after S4. The pupil wished to study social subjects but had no idea of the options or potential entry routes available. The work of the ERP Adviser was considered very important in helping this pupil to research and understand their options and hopefully move on to a positive destination.

5.26 ERP Advisers seem to have played a significant role in helping pupils make their applications to further or higher education establishments, employers or training providers. Among those who had submitted applications at the time of the survey we found that:

• 36% believe they would not have made their application without the support of the ERP Adviser
• 32% feel that the ERP Adviser helped them to improve the quality of their applications
• 12% feel that they completed their applications quicker as a result of the support received from the ERP Adviser.

5.27 Only 12% of pupils who were interviewed indicated that the ERP Adviser had played no part in their decision to submit applications for employment, education or training and had not assisted them in this process.

5.28 Parents were also asked if they thought their child would have made these applications if they had not visited the ERP Adviser. Of the 66 parents who said their child had made applications, 38% said they would not have made them.

5.29 The above findings are in line with those of the year one survey and once again are supported by evidence from focus groups with young people which highlighted many instances in which ERP Advisers have assisted pupils in completing applications for employment, education and training. Pupils commented that work to develop CVs and opportunities to develop their interview skills in mock sessions were particularly beneficial. As a direct result of assistance from the ERP Advisers a number of young participants have gained entry to further education, apprenticeships and other training programmes.

5.30 Some pupils did, however, say that they chose to approach pastoral care staff to help them complete their applications. In these cases pupils also said that they were aware that they could have asked the ERP Adviser to assist them with this. When asked why they sought help from their pastoral care teacher rather than the ERP Adviser they were not able to explain. The reason seemed to be just that they know that pastoral care teachers are able to help with applications and this was the first person they thought of approaching for help.

5.31 Interviews with school staff and ERP Advisers themselves reinforce the importance of the ERP Adviser in helping young people to complete applications and secure a positive post-school destination. The fact that ERP Advisers have the capacity to provide intensive support to ensure that applications are completed and submitted on time appears to be key to this achievement.

Changes in attitudes and behaviour

5.32 While not an explicit goal of the pilot, positive changes in the attitudes and behaviours of young people can be considered a positive impact on the young people.

5.33 Participants were asked to comment on their attitudes and behaviour patterns prior to and following participation in ERP. Pupils and parents were asked a series of questions concerning:

Attendance at school

5.34 When asked about their attendance, the majority (58%, 50 pupils) indicated that they had been attending school "regularly" prior to their involvement with the ERP Adviser. Only 6% of respondents (5 pupils) stated that they had stopped attending school prior to their engagement with the pilot.

5.35 ERP Advisers had a positive impact on attendance for 27% of participants (23 pupils). Reasons included:

• Additional encouragement and support they had received (52% (12 pupils))
• Realisation that they needed to improve their marks in order to achieve their goals (39% (9 pupils))

5.36 This finding is similar to that of the year one survey in which 33% of pupils felt that their involvement with the ERP Adviser had had a positive impact on their attendance.

5.37 While school staff provided some anecdotal evidence of an improvement in attendance among some ERP participants (see case study example 5.2), no quantitative evidence of this was available. Further, school staff and ERP Advisers do not generally consider improved attendance to be a commonly attainable outcome of the pilot, expressing an opinion that those engaging with the pilot in school will tend to be regular attenders; for non-attenders, the principal aim of the pilot will be to assist in finding an appropriate post-school option. This may include exceptional entry to college.

Attitude to school

5.38 Pupils commented on their enjoyment of school before and after working with the ERP Adviser. A range of responses were recorded with the largest number of respondents indicating that they "enjoyed school a bit" before their involvement with the pilot. Two thirds of respondents did not feel that their work with the ERP Adviser had made any difference to their enjoyment of school. Once again this is very similar to the findings in year one where just under two thirds of respondents held this view.

5.39 In both years this was supported by the findings of qualitative focus groups and in interviews with ERP Advisers, many of whom commented on a change in the attitude of pupils towards the pilot. ERP Advisers felt that this was due to the establishment of relationships but did not feel that this would be translated into any real change in enjoyment of school per se.

5.40 Among those pupils who said that they began to enjoy school more after working with the ERP Adviser (34% (29 pupils)), the most significant explanations cited by pupils included:

• The advice received (41% (12 pupils))
• A desire to achieve better grades (35% (10 pupils)).

5.41 As table 5.1 shows, the majority of pupils felt that they had been making some or significant effort to achieve good marks in their exams. Forty-two per cent of pupils stated that their involvement with the ERP Adviser had had a positive impact on their attitude to their school work and exam marks. This figure is below that recorded in year one where 67% of respondents were of this opinion.

Table 5.1 Attitude towards school

Source: 2008 Research Resource Telephone Survey, N=86

5.42 When asked to explain why attitudes to school work may have changed following engagement with the ERP Adviser, the most common responses were:

• Improved chances of getting better results (39%)
• Began trying harder (33%).

5.43 A number of school teachers commented on a perceived improvement in the attitude of ERP participants towards their general school work. One teacher said that this was especially true for those on the margins of disengaging as opposed to those who have already given up on school. A possible explanation for this was provided by another teacher who suggested that ERP Advisers are instrumental in building the confidence and self-esteem of their pupils by helping them to take control of their lives and plan for their future (see case study examples 5.1 and 5.2).

Career aspirations

5.44 The survey of ERP participants asked respondents to comment on their career aspirations before and after involvement with the ERP Adviser. Figure 5.9 shows a 25% increase in the number of pupils who were confident of their career path after leaving school. A similar pattern was observed in the survey of parents (Figure 5.10).

Source: 2008 Research Resource Telephone Survey, N=86

Source: 2008 Research Resource Telephone Survey, N=91

5.45 To try and isolate the impact of ERP in assisting pupils, survey respondents (pupils and parents) were asked about the role of the ERP Adviser in helping them reach their decisions (Figure 5.11). It was found that:

• 23% of pupils feel that the ERP Adviser was instrumental in helping them make a decision about their future career options
• Around a quarter of pupils feel that the support of the ERP Adviser has helped them to make better choices
• 14% believe it would have taken them longer to make their decisions without the support of the ERP Adviser.

5.46 Of 76 parents that said their child was more certain now, 38% gave no comment as to why this was the case. However, 25% said that this change had come about as a direct result of the pupil's involvement with the ERP Adviser.

Source: 2008 Research Resource Telephone Survey, N=86

5.47 In line with the findings above, the year one survey reported that around a quarter of pupils felt that the ERP Adviser had played a pivotal role in helping them come to a decision about their future career options.

5.48 However, the findings show that respondents in year one appear to have been less certain about their careers options from the start, with only 9% stating that they made their career decisions independently. This is compared to 24% of year 2 respondents who stated that they made their decision without the help of the ERP Adviser. In year one, 46% of respondents felt that while they would have made a careers decision on their own, the support of the ERP Adviser had enabled them to make a better decision. This is significantly above the 24% of year 2 respondents who gave this response.

5.49 These differences may reflect changes in the way pupils were selected for ERP support over the course of the pilot, for example declining focus on non-attenders, as well as the impact of other support within the school environment as the focus on careers education and planning intensifies.

5.50 At the time of the survey 7% of respondents in year one had yet to make a decision regarding their future career path. This is compared to 13% in year 2.

Personal development

5.51 Figure 5.12 and 5.13 illustrate the impact of the Enhanced Resource Pilot on the personal development of young participants. They highlight a significant impact in relation to:

• Knowledge of information sources concerning further and higher education, training and employment
• Knowledge of how to present yourself for an interview
• Confidence
• Understanding of the link between achievement at school and progressing into a positive destination and understanding of the importance of achieving a positive destination after leaving school.

5.52 All of these may be considered important factors in helping young people to achieve and sustain positive post-school destinations. More than half of the young people questioned in year 2 also indicated that they now feel more certain about what they will do upon leaving school as a result of working with the ERP Adviser. These findings are similar to those of the year one survey in which positive (if slightly higher) impacts in relation to personal development outcomes were also found.

5.53 This is a positive finding for the pilot but as yet it is not possible to tell if this positive sentiment will be translated into an improvement in this year's SLDR.

Source: 2008 Research Resource Telephone Survey, N=86

Source: 2008 Research Resource Telephone Survey, N=86

Summary

• During the pilot ERP Advisers have worked with 1247 pupils, the majority of whom have moved into positive destinations. In year one 75% of ERP pupils moved into a positive destination and in year 2, 65% achieved a positive destination.
• In year 2 of the pilot there was a significant decrease in the numbers moving into training and a significant increase in the numbers staying at school. This could be explained by a shift in the demographic of the pupils who were referred.
• Sustainability data shows that the majority of ERP pupils are in a positive destination three months after leaving school, with, on average, over three quarters of leavers settling in a positive destination at this time. However, this proportion decreases gradually, and after one year an average of two thirds of leavers sustain a positive outcome.
• The vast majority of pupils surveyed in year 2 (75%) reported that prior to meeting the ERP Adviser they had expected to move into a positive post-school destination.
• ERP Advisers have raised the expectations of those who, prior to engaging with the ERP Adviser thought they would be unemployed when they left school or had no expectations about their future career. The number of pupils in this group decreased from 14% (11 pupils) to 3% (2 pupils) in year one and from 28% (24 pupils) to 7% (6 pupils) in year 2.
• School staff never mentioned considering pupils' expectations about their post-school destination when determining whether they should be referred to the ERP Adviser. Given that pupils with low expectations were found to be a group that ERP Advisers have had a significant impact on (70% of this group raised their expectations), we would recommend that this group be targeted in future to maximise the impact that similar interventions could have.
• Over a third of those surveyed in year 2 were supported by the ERP Adviser to complete an application for employment, further education or training place, which they would not have applied for without support.
• Around a third of pupils in both years reported that they were attending school more frequently as a result of the support they had received from the ERP Adviser. This was a positive outcome, although not an aim of the pilot.
• At the time of the survey a quarter of pupils in both years believed that they would not yet have made a decision about their future career had it not been for the ERP Adviser. A further 25% of those surveyed in year 2 and 42% of those in year one believed that although they would have made a decision the ERP Adviser had helped them to make a better one.
• ERP Advisers have had a significant impact on personal development, with 69% of pupils feeling more confident; better understanding the link between attending school, working hard and getting a job or college place; and better understanding the importance of getting a job, training place, college or university place or apprenticeship. A significant impact in relation to personal development was also observed in year one.
• Employability skills have also improved with upwards of 70% of pupils in both years having more knowledge about information sources that will help them find out about the various post-school destinations; better understanding of how to present themselves for an interview; more awareness of career options; and more understanding about how to complete an application form.

« Previous | Contents | Next »