The Scottish Government

« Previous | Contents | Next »

Listen

D: Numeracy attainment estimates

D.1 Assessing and reporting numeracy attainment

The design of the 2006 numeracy assessment was the same as that of 2005. The details of the assessment materials and procedures are available in Annex II.

Every pupil involved in the assessment of numeracy was randomly allocated two booklets, each containing a mix of test items at three levels. In order to establish a range of achievement at each stage, pupils were assessed in numeracy at the level expected for the stage, the level above this and the level below. The two exceptions are at P3 and P5. At P3 the expected level is level A, with no level below. This allowed us the opportunity to use the extra survey space to assess pupils at level C, which is two levels above that expected, in order to investigate further the extent of the progress pupils in this stage are making beyond the expected level. At P5 there is no expected level. Pupils at this stage were assessed at level B which is the expected level for P4, at level C which is the expected level for P6, and at level D which is the expected level for P7.

The results of the survey for numeracy are reported using criteria for performance on the assessment items as follows: 'very good skills' at a level, indicated by 80% or more of the test items at that level answered correctly; 'well-established skills', at least 65% of items answered correctly but fewer than 80%; and 'made a good start', at least 50% of the items answered correctly but fewer than 65%.

In total, just over 13,000 pupils took part in the numeracy assessments. All estimates of numeracy attainment shown in the following section are adjusted for the over-representation of reporting authorities in the national sample, and also for sample imbalance in terms of gender and deprivation. As with all sample surveys there is a level of uncertainty inherent in the results. Where appropriate, standard errors are quoted alongside the attainment estimate and the size of the standard errors should be taken into account when interpreting the data. Further details of the survey design are available in Annex I, including the data weighting procedure.

D.2 National estimates of numeracy attainment

Chart D1 shows the percentage of pupils in each of P3, P5, P7 and S2 categorised into attainment bands at levels A-F for numeracy. Detailed results are available in Table D1.

Chart D1
Numeracy attainment estimates at P3, P5, P7 and S2^*
(% pupils classified into attainment bands by 5-14 level)

*Standard errors are ½-1½ percentage points for 'well-established skills or better', varying with stage and level.

The chart shows that 90% of pupils at P3 were well established or better at the level expected of them (level A). At P5, 81% of pupils were well established or better at level B (the expected level for P4) and 48% of P5 pupils were well established or better at level C (the expected level for P6). In P7, 67% of pupils were well established or better at level D (the expected level for the stage) and in S2 46% of pupils were well-established or better at level E (the expected level for the stage) in numeracy.

Considering the levels below those expected for the stage, the proportion of pupils who were well established or better is generally around 80%, although this varied from 85% at P7 level C to 76% at S2 level D.

Many pupils were working at levels above those expected for their stages. For example, 33% of P3 pupils were already well-established or better at level B and 5% at level C. In S2, 18% of pupils were already well established or better at level F.

D.3 Gender differences in numeracy attainment

Chart D2 compares the attainment of boys and girls within the survey.

Chart D2
Numeracy attainment estimates, by gender
(% pupils having "well-established" skills or better at the given level)

The thin black lines attached to each attainment are the 95% confidence intervals

There is no consistent pattern in the differences in attainment between boys and girls and in almost all cases the size of the difference is not significant. There is, however, a tendency for boys to do better than girls at the most advanced level assessed within each stage. These findings are in line with the results of the 2005 survey. Detailed results for attainment in numeracy by gender are available in Table D2.

D.4 Deprivation differences in numeracy attainment

Chart D3 compares the attainment of pupils from the 20% most deprived areas in Scotland against those from less deprived areas.

Chart D3
Numeracy attainment estimates, by deprivation
(% pupils having "well-established" skills or better at the given level)

The thin black lines attached to each attainment are the 95% confidence intervals

Pupils from the most deprived areas had lower levels of attainment than pupils from less deprived areas, at all stages and levels. The differences are small at P3 and are not statistically significant, however there is a consistent pattern which suggests that this is may not be due to random variation within the sample. The differences for P5, P7 and S2 pupils are statistically significant and pupils from the most deprived areas had lower levels of attainment than pupils from less deprived areas at all levels within these stages. The differences in attainment between the pupils from the most and the less deprived areas increases as the stage assessed increases. Detailed results for attainment in numeracy by deprivation are available in Table D3.

D.5 Teachers' judgements about attainment in numeracy

In addition to the written assessments for numeracy skills, teachers were also invited to submit a level judgement for each pupil sampled in the survey. Judgements were submitted for around 80% of the sampled pupils. Chart D4 shows the national picture of numeracy attainment based on these class teachers' judgements. Detailed results are available in Table D4.

Chart D4
Teachers' judgements of numeracy attainment
(% pupils judged to be at indicated levels by their teachers)

In general, teachers judged pupils to be at or around the expected level for their stage. At P3, teachers judged 97% or more pupils to be either at the expected level or the level above. At P7 the judgements are more widely spread, although 75% of pupils were judged to be at the expected level or above. At S2 64% were judged to be at the expected level or above with 23% judged to be at level D. There is no evidence of any gender difference in level judgements at any stage.

Teacher judgements and SSA test results are not necessarily equivalent indicators of pupils' numeracy levels, so it might not be surprising to find differences emerging in the results of the two approaches. Judgements are based on a range of knowledge that the classroom teacher has about each pupil, which has been gained over a period of time. These judgements will be locally moderated within the school. The SSA numeracy tests, on the other hand, are centrally administered and designed to provide comparable performance opportunities for all pupils in the survey wherever they might live. In this case, the results are based solely on performance on the day of the test and pupils' responses to the material that is contained within them.

Table D5 shows the coincidence rates for numeracy, which compare the teachers' judgements about numeracy attainment with the test-based results. The statistics in this table derive from unweighted sample data and, as noted above, that although these attainment measures appear similar there are important differences between them which means care should be taken when making comparisons.

The highest coincidence rate is seen at the expected level for each stage. For example, 81% of pupils who attained level A at P3 on the basis of their SSA test results were judged to be at level A by their teachers. In comparison, only 3% of pupils who attained level C at P3 on the basis of test results were judged to be at level C by their teachers.

D.6 Changes in numeracy attainment over time

Around 40% of the items used in the 2006 survey had also been used in the 2005 survey. The remaining items were newly developed. On average, pupils performed about the same on the new items compared with those repeated from 2005. This means we can be reasonably confident in our ability to compare year-on-year trends for numeracy using data from all of the items, without resorting to a comparison based on the common items alone. More detail can be found in Annex II.4.

The comparison between 'well established or better' attainment in numeracy between 2005 and 2006 is shown in Chart D5.

Chart D5
Numeracy attainment between 2005 and 2006
(% pupils having "well-established" skills or better at the given level)

The thin black lines attached to each attainment are the 95% confidence intervals.

There are a number of small shifts in the attainment estimates across all stages, but with no consistent pattern of increase or decrease. Only two of the changes are statistically significant. The estimated proportion of pupils in S2 showing well established or better skills at level F has increased from 13% in 2005 to 18% in 2006, while the estimated proportion of P5 pupils showing well established or better skills at level D has decreased from 22% to 17%. More detailed results are available in Table D6.

It is also possible to look at trends in attainment over time by comparing the teachers' level judgements. These are shown in Chart D6. It should be noted that the 2005 and 2006 data derive from the sample-based information collected as part of the SSA, whereas those relating to earlier years are based on the results collected on all pupils in Scotland (via the 5-14 National Survey of Attainment). These differences in collection methods need to be borne in mind when looking at longer term trends.

At P3 the high proportion of pupils achieving the expected level has grown only very slowly over the period, and now an inevitable ceiling effect has kicked in. At P7, there has been a steadier rise in attainment proportions, with a levelling off between 2005-2006. At S2, however, there has been a continued increase in the proportion of pupils judged to be achieving level E. Detailed results are shown in Table D7.

Chart D6
Trends in numeracy attainment - teachers' judgements
(% pupils judged to be at indicated levels by their teachers)

D.7 Detailed numeracy results

Tables D1 to D7 provide detailed results to support the information provided in this section.

Table D1
2006 SSA - National numeracy attainment estimates
(% pupils categorised into each attainment band*)

* '< 50%' means fewer than 50% of the items in the level-based task answered correctly, 'good start' means between 50% and 64%, 'well-established' is 65% to 79%, and 'very good' is 80% or more. ' SE' is the standard error for the 'well-established or better' estimate.

Table D2
2006 SSA - National numeracy attainment estimates, by gender
(% pupils categorised into each attainment band ^*)

* '< 50%' means fewer than 50% of the items in the level-based task answered correctly, 'good start' means between 50% and 64%, 'well-established' is 65% to 79%, and 'very good' is 80% or more. ' SE' is the standard error for the 'well-established or better' estimate.

Table D3
2006 SSA - National numeracy attainment estimates, by deprivation** category
(% pupils categorised into each attainment band ^*)

* '< 50%' means fewer than 50% of the items in the level-based task answered correctly, 'good start' means between 50% and 64%, 'well-established' is 65% to 79%, and 'very good' is 80% or more. ' SE' is the standard error for the 'well-established or better' estimate.
**Most Deprived means the pupil attends a school in one of the 20% most deprived postcodes in Scotland based on the 2004 Scottish Index of Multiple Deprivation ( SIMD), the remainder of pupils are classified as less deprived

Table D4
2006 SSA - National numeracy attainment estimates: teachers' judgements
(% pupils judged to be at indicated levels by their teachers ^*)

* For 'all pupils' the standard error is between 0.1 and 1.1 for non-zero estimates, and between 0.1 and 1.9 for each non-zero gender estimate.

Table D5
Test-based numeracy estimates compared with teachers' judgements
The statistics in this table derive from unweighted sample data

Table D6
Trends in reading attainment: 2005 and 2006 SSA*
(% pupils having "well-established" skills or better at the given level)

Table D7
Trends in numeracy attainment - teachers' judgements
SSA results compared with the 5-14 National Survey*
(% pupils judged to be at indicated levels by their teachers)

*Pre-2005 data is based on the National Survey of 5-14 Attainment. This census was replaced by the sample-based SSA in 2005.

« Previous | Contents | Next »