The Scottish Government

« Previous | Contents | Next »

Listen

C: Reading attainment estimates

C.1 Assessing and reporting reading attainment

The design of the 2006 reading assessment was the same as that of 2005. However, thirty five of the seventy two reading tasks (or tests) were newly developed in order to introduce a social subjects context into this survey. The details of the assessment materials and procedures are available in Annex II.

Every pupil involved in the assessment of reading was randomly allocated three tasks, at three consecutive levels. In order to report on a range of achievement, pupils were assessed in reading at both the level above and the level below that expected. The two exceptions are at P3 and P5. At P3 the expected level is A, so there is no level below. This allowed us the opportunity to use the extra survey space to assess pupils at level C, which is two levels above that expected, in order to investigate further the extent of the progress pupils in this stage are making beyond the expected level. At P5 there is no expected level. Pupils at this stage were assessed at level B (the expected level for P4), at level C (the expected level for P6) and at level D (the expected level for P7).

The results of the survey for reading are reported using criteria for performance on the assessment tasks as follows: 'very good skills' at a level, indicated by 80% or more of the test questions at that level answered correctly; 'well-established skills', at least 65% of questions answered correctly but fewer than 80%; and 'made a good start', at least 50% of the questions answered correctly but fewer than 65%.

In total, just over 14,000 pupils participated in the reading assessments. All estimates of reading attainment shown in the following sections are adjusted for the over-representation of reporting authorities in the national sample, and also for sample imbalance in terms of gender and deprivation. As with all sample surveys there is a level of uncertainty inherent in the results. Where appropriate, standard errors are quoted alongside the attainment estimate, and the size of the standard errors should be taken into account when interpreting the data. Further details of the survey design are available in Annex I.

C.2 National estimates of reading attainment

Chart C1 shows the percentage of pupils in each of P3, P5, P7 and S2 categorised into attainment bands at levels A-F for reading. Detailed results are available in Table C1.

Chart C1
Reading attainment estimates at P3, P5, P7 and S2^*
(% pupils classified into attainment bands at the given level)

* Standard errors are 1-1½ percentage points for 'well-established skills or better', varying with stage and level.

The chart shows that 79% of pupils at P3 were well established or better at the level expected of them (level A) and 48% of P5 pupils were well established or better at level C (the expected level for P6). However, attainment levels were lower in later stages. In P7, 48% of pupils were well established or better at level D, and in S2 43% of pupils demonstrated well-established or better reading skills at level E.

Considering the levels below those expected for the stage, the proportion of pupils who were well established or better also decreased through the stages, from 84% at P5 level B to 63% at S2 level D. On the other hand, many pupils were working at levels above those expected for their stages. For example, 52% of P3 pupils were already well-established or better at level B and 19% were at level C. In S2, 18% of pupils demonstrated well established or better reading skills at level F.

C.3 Gender differences in reading attainment

Chart C2 compares the attainment of boys and girls within the survey.

Chart C2
Reading attainment estimates, by gender
(% pupils having "well-established" skills or better at the given level)

The thin black lines associated with each attainment bar are the 95% confidence intervals.

There is consistently higher attainment in reading amongst girls compared to boys at all stages and levels, although in most cases the size of the difference is not statistically significant . Nevertheless, the trends shown are in line with the results of the 2005 survey as well as other research evidence. Detailed results for attainment in reading by gender are available in Table C2.

C.4 Deprivation differences in reading attainment

Chart C3 compares the attainment of pupils from the 20% most deprived areas against those from less deprived areas.

Chart C3
Reading attainment estimates, by deprivation
(% pupils having "well-established" skills or better at the given level)

The thin black lines associated with each attainment bar are the 95% confidence intervals.

There was consistently higher attainment in reading amongst pupils from less deprived areas than pupils from the most deprived areas at all stages and levels. In almost all cases, the size of the difference is statistically significant. There is also a clear trend towards increasing gaps as children get older. Detailed results for attainment in reading by deprivation are available in Table C3.

C.5 Teachers' judgements about attainment in reading

In addition to the written assessments for reading skills, teachers were also invited to submit a level judgement for each pupil sampled in the survey. Judgements were submitted for around 80% of the sampled pupils. Chart C4 shows the national picture of reading attainment based on these class teachers' judgements. Detailed results are available in Table C4.

Chart C4
Teacher judgements of reading attainment
(% pupils judged to be at indicated levels by their teachers)

In general, teachers judged pupils to be at or around the expected level for their stage. At both P3 and P5, teachers judged at least 85% of pupils to be either at the expected level or the level above. At P7, the judgements are more widely spread but still with just under 80% judged to be at the expected level or the one above. At S2, however, a lower number (65% of pupils) were judged to be at level E or F and 23% were judged to be at level D (one below that expected).

In terms of differences by gender, teacher judgements show similar trends to the test based results with greater proportions of female pupils consistently judged to be at higher levels.

Teacher judgements and SSA test results are not necessarily equivalent indicators of pupils' reading levels, so it might not be surprising to find differences emerging in the results of the two approaches. Judgements are based on a range of knowledge that the classroom teacher has about each pupil, which has been gained over a period of time. These judgements will be locally moderated within the school. The SSA reading assessments, on the other hand, are centrally administered and designed to provide comparable performance opportunities for all pupils in the survey wherever they might live. In this case, the results are based solely on performance on the day of the test and pupils' responses to the material that is contained within them.

Table C5 shows the coincidence rates for reading, which compare the teachers' judgements about reading attainment with the test-based results. The statistics in the table derive from unweighted sample data and, as noted above, although these attainment measures appear similar there are important differences between them which means care should be taken when making comparisons.

The results show that there is a clear relationship between the SSA assessment results and the teachers' judgements about pupils' levels. There is highest coincidence around the levels expected for the stage. For example, for P3 pupils estimated at level A in the test results, teachers were in agreement in 62% of cases. However, for P3 pupils estimated at level C from the test, teachers had provided a lower estimate for 95% of them.

It is also interesting to note that, for pupils assessed at the lower levels by the SSA tests, teachers tended to have provided slightly higher level judgements. For example, 52% of the P7 pupils estimated at level C on the basis of their test results were judged to be at level D by their classroom teachers.

C.6 Changes in reading attainment over time

As noted in the introduction, one of the main objectives of the SSA is to help support the development of assessment skills and approaches in a range of subject areas. Therefore, each survey has a subject focus which changes year-on-year. The focus this year was on Social Subjects while in 2005 it was English language. In order to estimate attainment levels in reading in the context of the social subjects, some new reading tasks were introduced between 2005 and 2006. Around half of the tasks used in the 2006 survey had also been used in the 2005 survey. The remaining tasks were newly developed. The comparison of the results from the 2006 survey with those from 2005 has been made using data from those tasks which were common to both years. This controls for the 'task' effect and gives a truer comparison of pupil ability between the two years. The results are shown in Chart C5.

Although new tasks are designed to perform in a similar way to the existing tasks it is possible that new tasks (although validated and pre-tested) may perform differently to our expectation introducing 'task' bias to our results. The extent of this potential bias was investigated and the results show that, on average, pupils found the new tasks to be consistently harder compared with those tasks repeated from 2005. More detail can be found in Annex II.3. Because of these differences, comparisons of the results from the 2006 survey with those from 2005 have been made using data from those tasks which were common to both years. This controls for the 'task' effect and gives a truer comparison of pupil ability between the two years.

The comparison between 'well established or better' attainment in reading between 2005 and 2006 (using data from the common tasks) is shown in Chart C5.

Chart C5
Reading attainment between 2005 and 2006
(% pupils having "well-established" skills or better at the given level)

The thin black lines associated with each attainment bar are the 95% confidence intervals.

At the expected levels, we can see that there have been insignificant increases in estimated reading attainment of 3 percentage points at P3 Level A and 2 percentage points at P5 Level B. At P7, the proportion attaining Level D has increased insignificantly to 55%. At S2 Level E, however, we see a significant decrease in estimated attainment of 8 percentage points from 55% to 47%. There has also been a significant decrease in the proportion of S2 pupils who are well established or better at Level D. More detailed results are available in Table C6.

It is also possible to look at trends in attainment over time by comparing the teachers' level judgements. These are shown in Chart C6 with more detailed results available in Table C7. It should be noted that the 2005 and 2006 data derive from the information collected as part of the SSA, whereas those relating to earlier years are based on the results collected on all pupils in Scotland (via the 5-14 National Survey of Attainment). These differences in collection methods need to be borne in mind when looking at longer term trends.

At P3 the high proportion of pupils achieving the expected level has increased slowly over the period, and is now levelling off as they become subject to an inevitable ceiling effect. At P7, however, there has been a continued increase in the proportion of pupils judged to be achieving level D. At S2, there has been a similar steady increase in attainment proportions, but the upward trend is currently levelling off.

Chart C6
Trends in reading attainment - teachers' judgements
(% pupils judged to be at indicated levels by their teachers)

C.7 Detailed reading results

Tables C1 to C7 provide detailed results to support the information provided in this section.

Table C1
2006 SSA - National reading attainment estimates
(% pupils categorised into each attainment band ^*)

* '< 50%' means fewer than 50% of the items in the level-based task answered correctly, 'good start' means between 50% and 64%, 'well-established' is 65% to 79%, and 'very good' is 80% or more. ' SE' is the standard error for the 'well-established or better' estimate.

Table C2
2006 SSA - National reading attainment estimates, by gender
(% pupils categorised into each attainment band ^*)

* '< 50%' means fewer than 50% of the items in the level-based task answered correctly, 'good start' means between 50% and 64%, 'well-established' is 65% to 79%, and 'very good' is 80% or more. ' SE' is the standard error for the 'well-established or better' estimate.

Table C3
2006 SSA - National reading attainment estimates, by deprivation** category
(% pupils categorised into each attainment band ^*)

* '< 50%' means fewer than 50% of the items in the level-based task answered correctly, 'good start' means between 50% and 64%, 'well-established' is 65% to 79%, and 'very good' is 80% or more. ' SE' is the standard error for the 'well-established or better' estimate.
** Most Deprived means the pupil attends a school in one of the 20% most deprived postcodes in Scotland based on the 2004 Scottish Index of Multiple Deprivation ( SIMD), the remainder of pupils are classified as less deprived

Table C4
2006 SSA - National reading attainment estimates: teachers' judgements
(% pupils judged to be at indicated levels by their teachers ^*)

* The standard error is between 0.1 and 1.1 for non-zero estimates of the "all pupils" group and between 0.1 and 1.9 for each non-zero gender estimate

Table C5
Test-based reading estimates compared with teachers' judgements
The statistics in this table derive from unweighted sample data

Table C6
Trends in reading attainment: 2005 and 2006 SSA*
(% pupils having "well-established" skills or better at the given level)

* using only tasks which were common to both surveys

Table C7
Trends in reading attainment: teachers' judgements
SSA results compared with the 5-14 National Survey*
(% pupils judged to be at indicated levels by their teachers)

*Pre-2005 data is based on the National Survey of 5-14 Attainment. This census survey was replaced by the sample-based SSA in 2005.

« Previous | Contents | Next »