« Previous | Contents | Next »
Listen
SECTION 5: THE PHASE 2
EMA PILOTS IN THEIR FIRST
YEAR
5.1 The Phase 2
EMA Pilots began in autumn 2001, in
Glasgow, Dundee and West Dunbartonshire. Thus, they were in
operation in session 2001/2 when cohort 3 were in their
first year after S4, and cohort 2 in their second year
after S4. In this section, we look at the early impact of
the Phase 2
EMA Pilots in their first year of
operation.
5.2 The analyses compare the participation and
attainment in national qualifications of young people in
the Phase 2
EMA Pilot areas with that of young
people in other areas of Scotland who attended schools with
similar characteristics to those in the
EMA pilot areas. There are 48 schools in
the Phase 2
EMA Pilot areas, 31 in Glasgow, 10 in
Dundee and 7 in West Dunbartonshire. A "control group" of
48 comparator schools was selected to match as closely as
possible the schools in the
EMA pilot areas (Appendix 2).
Participation in national qualifications in the
first year after S4
5.3 Figure 5.1 summarises overall levels of
participation in national qualifications in the first year
after S4, which is the S5 stage at school or its equivalent
at college. Trends are shown separately for "any"
participation, that is the percentage attempting at least
one national unit or course, and "full-time" participation,
that is the percentage attempting twelve or more national
units or three or more national courses.
Figure 5.1: Percentage attempting
NQ in first year after S4 in Phase 2
EMA pilot and control areas
5.4 There are no clear trends for the percentage
attempting "any" national qualifications (1+ national units
or courses), owing to the inflated figures for 2000 (the
first year of Higher Still) and apparent drop between 2000
and 2001, and subsequent increase by 2002. The inflated
figures for 2000 probably arise from over-reporting of 1-2
units - the problem has been described in Appendix 1 with
respect to the national picture. It is apparent from Figure
5.1 that overall participation in any national
qualifications is broadly similar in the
EMA pilots and the control sample.
5.5 There are, however, much clearer upward trends in
the proportion entered for "full-time" national
qualifications (12+ units or 3+ courses), and the trends
are more marked in the
EMA pilot areas than control areas.
There was an upward trend in full-time participation
between 2000 and 2001 in each of the pilot areas, which
occurred before the introduction of the
EMA. Subsequently, however, the increase
in full-time participation in 2002 was greater in the
EMA pilot sample than the control
sample.
5.6 Participation in national qualifications in the
first year after S4 is strongly influenced by the young
person's Standard Grade attainment, with lower levels of
participation by young people who did not achieve awards at
credit level. Figure 5.2 focuses on young people in the
comparative sample with no credit level awards at Standard
Grade, and shows that an increasing percentage of them were
attempting a full-time programme of national
qualifications. The upward trend is especially marked in
Glasgow and West Dunbartonshire. The increase in full-time
participation between 2000 and 2002 by young people with no
credit awards was significantly greater in the
EMA pilot sample than the control
sample.
Figure 5.2: Percentage of low-attainers
attempting
NQ in first year after S4 in Phase 2
Pilot and control areas
5.7 We used statistical models to test whether the
upward trend in full-time participation in national
qualifications was significantly greater in the
EMA pilot areas than the control areas,
after taking account of Standard Grade attainment, whether
living in a low
SES area (using the definition based on
MOSAIC), and school intake
characteristics as measured by free-meal entitlement (
FME). The modelling was carried out in
two stages: the first model focussed on factors influencing
the overall pattern of full-time participation, and the
second model tested whether participation was different
among a sub-set of young people with low Standard Grade
attainment, ie no credit-level awards. The detailed results
are given in Appendix Table A5.1.
5.8 The results show that the likelihood of full-time
participation:
- increased in line with Standard Grade
attainment;
- decreased in line with school average free-meal
entitlement.
- was lower for young people living in low
SES areas.
- There was a significant rise in full-time
participation between 2000 and 2002.
After taking account of Standard Grade attainment,
school
FME and low
SES, the increase in full-time
participation was significantly greater in
EMA areas than in the control areas in
both 2001 and 2002.
5.9 A further statistical model tested these factors
among young people with low Standard grade attainment. It
shows that levels of full-time participation by low
attainers in the
EMA pilots increased to a greater extent
in 2002 than participation by their counterparts in the
control areas.
Participation in national qualifications in the
second year after S4
5.10 Fewer young people take national qualifications in
the second year after S4 than in the first year after S4
(Appendix 1). However, in the
EMA pilot and control areas there was a
strong upward trend in the proportion of young people
attempting national qualifications, and this was most
marked for full-time participation, and was highest in
Dundee and West Dunbartonshire (Figure 5.3).
5.11 The statistical models used to analyse the effect
of
EMA group all the
EMA pilots together, and do not
distinguish between the different areas (Appendix Table
A5.2). The first statistical model shows that the
likelihood of full-time participation in the second year
after S4:
- increased in line with Standard Grade
attainment;
- decreased in line with school average free-meal
entitlement;
- was lower for young people from low
SES areas.
- Overall full-time participation increased between
2001 and 2003.
In 2001, full-time participation in the second
post-compulsory year was higher in the
EMA Pilots than in the control areas,
but this advantage was no longer evident in 2002.
Figure 5.3: Percentage attempting
NQ in the 2nd year after S4 in Phase
2
EMA Pilot and control areas
5.12 The second model, which focuses on a sub-set of
young people with low Standard Grade attainment shows that
on average, low attainers in the
EMA pilots were more likely to study
full-time than those in the control areas, but this
difference did not change between 2001 and 2003.
Attainment in the first year after S4
5.13 As described in Appendix 1, throughout Scotland the
Higher Still reforms and the introduction of new national
qualifications have led to a marked increase in the average
numbers of national qualifications attempted. The
EMA pilots and control areas shared in
this upward trend (see Appendix Table A5.3).
5.14 Figure 5.4 shows that average attainment in the
first year after S4, measured by the attainment point
score, increased between 2000 and 2002. The trend lines in
the upper part of Figure 5.4 show average attainment in the
first year after S4 of all young people in the
EMA pilot and control areas. Overall,
average attainment increased in all areas, including the
control areas. However, the increase, especially by 2002,
was greater in Dundee and West Dunbartonshire than in the
control areas. In 2000, average attainment in Glasgow was
lower than that in the control areas, but the increasing
rate of attainment between 2000 and 2002 resulted in
Glasgow having higher average attainment than the control
areas by 2002.
5.15 The trend lines in the lower part of Figure 5.4
show average attainment of young people with low Standard
Grade attainment. For these young people, with no credit
level awards at Standard Grade, we find that the upward
trend in attainment is less steep than for the overall
average. Attainment increased to a greater extent in the
West Dunbartonshire Pilot than the control areas, but
trends in Glasgow and Dundee are a little steeper than the
control areas.
Figure 5.4: Average attainment score in 1st
year after S4, in Phase 2 Pilot and Control areas,
comparing scores of young people with low Standard
Grade attainment with overall average
5.16 We used statistical models to test whether the
upward trend in attainment was significantly greater in the
EMA pilot areas (taken together) than
the control areas, after taking account of other factors;
the results are given in Appendix Table A5.4. The results
of the first model confirm that attainment in national
qualifications in the first year after S4:
- increased in line with young people's prior
attainment;
- decreased in line with school average
FME;
- was lower for young people living in low
SES areas;
- rose significantly between 2000 and 2002.
After controlling for these factors, attainment was
lower in the
EMA pilot areas than in the control
areas in 2000, but increased to a significantly greater
extent between 2000 and 2002 in the
EMA areas than in the control areas.
5.17 A second model tested the effect of these factors
on a sub-sample of young people with low Standard Grade
attainment. It found that average attainment by
low-attainers in the
EMA pilots increased to a significantly
greater extent than attainment by their counterparts in the
control areas.
5.18 There are differences between areas in levels of
participation at college, and to take account of these
differences, a third statistical model controlled for
differences associated with school and college (Model 3 in
appendix table A5.4). Results from this model are different
to those described in Section 4. It found that:
- In 2000, young people attending college for all or
part of their
NQ studies had higher average
attainment than their peers (note: the opposite effect
is found in section 4);
- In 2001 and 2002 young people studying at college
did not share to the same extent in the overall
increase in attainment;
- Trends in attainment at college in the Phase 2
EMA Pilots were no different to
those in control areas.
- After controlling for different patterns of
attainment between school and college students, we can
identify more clearly the effect of
EMA in increasing attainment by
young people who stayed on at school. After taking
account of differences in college participation, the
increase in attainment in 2001 and 2002 was
significantly greater in the
EMA pilot areas than in the control
areas.
Attainment by the end of the second year after
S4
5.19 Trends in the average numbers of national
qualifications attempted and passed in the second year
after S4 are summarised in Appendix Table A5.5. Measures of
attainment such as the average number of courses attempted
and passed, and the overall attainment point score
increased to a greater extent in Dundee and West
Dunbartonshire than in the control areas, but the trends
for Glasgow were similar to those in the control areas.
Figure 5.5: Average attainment score over two
years after S4, in Phase 2 Pilot and control areas,
comparing scores of young people with low Standard
Grade attainment with overall average
5.20 Figure 5.5 summarises cumulative attainment over
the two years after S4, separately for all students (the
trend lines in the upper part of Figure 5.5) and those with
low Standard Grade attainment (the trend lines in the lower
part of Figure 5.5). If we focus on the overall trends for
all students, we find an upward trend in attainment in all
areas. The increase in attainment over the three years is
higher in Dundee and West Dunbartonshire than in the
control areas. Although average attainment was lower in
Glasgow before the pilot than in the control areas and
remains so, upward trends there are similar.
5.21 Turning to young people with low prior attainment
at Standard Grade, we find that there is not such a steep
increase in attainment over the three years as that shown
for attainment by all students. However, the upward trends
for this group are greater in all the
EMA pilot areas than in the control
areas. In West Dunbartonshire average attainment by this
group over the two years is higher than in the control
areas and other pilots. In Dundee and Glasgow the upward
trend in attainment by this group is slightly steeper than
that in the control areas, especially between 2002 and
2003.
5.22 We used statistical models to test the overall
EMA effect on attainment in the two
years after S4 (Appendix Table A5.6). The results confirm
that attainment in national qualifications over the two
years after S4:
- increased in line with young people's Standard
Grade attainment.
- decreased in line with school average
FME
- was lower for those living in low
SES areas.
- There was a significant rise in attainment between
2001 and 2003.
In 2001, average attainment in the
EMA pilot areas was no different from
that in the control areas. However, the increase in
attainment between 2001 and 2002 was significantly greater
in
EMA areas than in the control areas.
5.23 Focusing on young people with low Standard Grade
attainment we find that average attainment increased by
2001, and the increase was significantly greater among low
attainers in the
EMA pilots than among their counterparts
in the control areas.
The Effects of Phase 2
EMA Pilots: Summary
5.24 The
EMA pilots had a positive effect on
participation in the first year of operation, both overall
and among low attainers. Full-time study of national
qualifications in the first year after S4 (S5 or its
college equivalent) rose substantially between 2000 and
2002, and the increase was greater in the Phase 2 pilot
areas than the control areas. Rising participation was
evident in the Phase 2 pilot areas before the introduction
of the
EMA, and increased thereafter. Similar
upwards trends in participation were evident among young
people with low Standard Grade attainment, and were greater
on average in the
EMA pilots.
5.25 Participation in the second year after S4 increased
between 2001 and 2003 in both pilot and control areas, but
there is no evidence that the increase was different in the
EMA pilots than in the control
areas.
5.26 The
EMA pilots had a positive effect on
attainment in their first year of operation, both overall
and among low attainers. Attainment of national
qualifications in the first year after S4 also rose
substantially between 2000 and 2002. As with participation,
average attainment rose to a greater extent in the Phase 2
pilot areas than the control areas. Rising attainment was
evident in the Phase 2 pilot areas before the start of the
EMA, and increased further in 2002 after
the start of the
EMA. Average attainment increased among
young people with low Standard Grade attainment, and the
increase was greater on average in the
EMA pilots.
5.27 Young people who took all or part of their national
qualifications at college had higher average attainment in
2000 but did not share in rising levels of attainment to
the same extent as those who took all their national
qualifications at school. This pattern did not differ
between pilot and control areas.
5.28 There were differences between the Phase 2 pilots
in the extent to which participation and attainment rose
over the three years, with steeper upward trends in West
Dunbartonshire than elsewhere.
5.29 Between 2001 and 2003 cumulative attainment by end
of the second year after S4 rose substantially. Although in
2001, before the start of the
EMA, there was no difference in
attainment between Phase 2 pilot and control areas, the
increase in attainment in 2002 and 2003 was significantly
greater on average in the Phase 2 pilots than in the
control areas. There was some increase in average
attainment among young people with low Standard Grade
attainment, and the increase was greater on average in the
EMA pilots. In other words the
EMA pilots had a positive effect on
attainment that was cumulative in the two post-compulsory
years. The beneficial effect was evident for the whole
sample and among low attainers.
« Previous | Contents | Next »