The Scottish Government

« Previous | Contents | Next »

Listen

Section 3 Reading, Listening/Watching, and Knowledge About Language

Assessing and reporting Reading attainment

3.1 At each of Levels A to F, 12 different reading tasks were administered in the survey (72 tasks in all). The tasks shared the format of national assessments in reading, each comprising a stimulus text and set of test questions. Both the demand of the texts and the number and complexity of questions increased from level to level.

3.2 Every pupil involved in the assessment of reading was randomly allocated three tasks, at three consecutive levels. At P3, pupils took tasks at Levels A, B and C; at P5 the tasks were at Levels B, C and D; at P7 the tasks were at Levels C, D and E; at S2, the tasks were at Levels D, E and F. At any particular level the same tasks were used at all relevant stages (e.g. Level C at P3, P5 and P7).

3.3 The results of the survey for reading are reported using criteria for performance on the assessment tasks as follows: 'very good skills' at a level, indicated by 80% or more of the test questions at that level answered correctly; 'well-established skills', at least 65% of questions answered correctly but fewer than 80%; and 'made a good start', at least 50% of the questions answered correctly but fewer than 65%.

3.4 The task performance results suggest that teachers' judgements about standards and progression across levels represented by the assessment materials used in the survey were generally sound, so that the tasks were appropriate to levels in terms of content and demand. There was inevitable variation in attainment rates across tasks within a level, but markedly greater differences across levels.

3.5 The evidence indicates that narrative and functional texts and tasks may be equally challenging or accessible for pupils, with no genre effect, except at P3 where narrative texts proved easier for pupils than functional texts, understandable given their likely more limited experience across genres.

Attainment results for Reading

Figure 1
Reading attainment profiles at P3, P5, P7 and S2*
(% pupils classified into attainment bands within 5-14 level: SSA 2005)

* Pupils have 'very good skills' at a level if they successfully answer 80% or more of the test questions in their level-based task; they have 'well-established skills' if they successfully answer 65% or more of the questions but not as many as 80%; they have 'made a good start' at the level if they answer correctly at least 50% of the questions but not as many as 65%. Approx. pupil sample sizes: 3,800-3,900 at primary stages, 3,300-3,500 at S2. Margins of error are of 1_-2 percentage points for 'well-established skills or better', varying with stage and level.

3.10 The picture in Figure 1 shows the pattern of attainment for P3, P5, P7 and S2 at Levels A-F. Levels B and C are the levels slightly above those expected for P3 and P5. For P7 and S2 Levels D and E are those expected for their stage. At these levels the picture is of strong attainment for P3 at Level B. The pattern is sustained into P5 at Level C.

3.11 At P7, however, the evident gains at P5 do not seem to have been fully sustained. The proportion of pupils with well-established skills or very good skills decreases slightly and the number making a good start, or not attaining the expected level, which had been successfully reduced from P3 to P5, increases slightly.

3.12 Between P7 and S2, schools appear to be building on the very good attainment of more able readers from Level D to Level E, although the proportion of S2 at Level F compared to P7 at Level E is almost halved.

3.13 Looking more closely at lower attainment, Figure 1 also shows attainment in P3, P5, P7 and S2 at Levels A, B, C and D, respectively. For P7 and S2 this is the level below that expected for their stage. For P3 and P5, the levels are those expected for almost all pupils.

3.14 Again, from P3 to P5 the proportion of pupils estimated as having well-established skills at the level slightly increases. The proportions estimated as having made a good start at Level B and Level C skills increases from P5 to P7, and again more sharply from P7 to S2. However, more than eight out of ten S2 pupils demonstrated at least a good start in Level D reading skills. The proportion not achieving at least a good start at the level below their expected level increases from one in ten in P5/P7 to nearly one in five at S2.

Reading in local authorities

3.15 The survey sample was drawn to allow some reporting at local authority as well as national level in 16 of the 32 Scottish local authorities. The results provide a snapshot in time, giving an indication of levels of attainment in reading in the summer of 2005. In future years it will be possible to show trends for each authority over two- or three-year periods.

3.16 There is a well-documented link between reading attainment and deprivation. The survey looked at the differences between authority attainment and national attainment at the various stages and levels in the 16 reporting authorities, taking account of income deprivation.

3.17 As might be expected, the results suggest that pupils in the three authorities with the lowest levels of income deprivation were among those with the highest pupil attainment rates. However, the patterns of attainment for reading varied across the 16 local authorities in focus in this survey. For example, the proportions of P3 pupils estimated as having well-established skills or better at Level B vary from 40% to over 60%; at P5 Level C estimates for well-established skills vary from over 50% to 65%; at P7 the variation at Level D is 40% to 65%; and at S2 at Level E the range is from around 50% to 60%. The variation at every stage and level shows a steady gradation rather than discrete shifts.

3.18 Among authorities with slightly higher levels of deprivation, P5 pupils in Edinburgh City seemed to perform particularly well as did P3 pupils in Highland and P7 pupils in Aberdeen City.

3.19 In the three authorities with the highest levels of income deprivation, Inverclyde clearly stands out for its generally positive picture of attainment. Pupil performance in other authorities with high levels of deprivation showed a more expected pattern.

3.20 Looking particularly at the lowest attaining pupils in the survey (not made a 'good start' at Levels A, B, C and D for the four reported stages), again as might be expected the results suggest that the proportions of sampled pupils in this category were generally lowest in East Dunbartonshire and East Renfrewshire, and highest in North Lanarkshire at P3, and in North Ayrshire. The proportions of pupils in this group were much lower than might be expected in South Ayrshire, especially at P3, in Renfrewshire at all primary stages and in Highland at S2; and higher than might be expected in Aberdeen City at P5 and S2.

3.21 As in the country as a whole, the results of the survey for some of the local authorities show a particularly encouraging picture in P3 and P5, suggesting that their recent approaches to raising attainment in the early years might be having a particularly positive impact on attainment in reading, and particularly for lower achieving pupils - 'closing the gap' at the earlier stages.

Teachers' judgements about attainment in Reading

3.22 In reading the proportion of pupils for whom the teacher's level judgement coincided with the survey result is around 40% overall and at each stage. The lowest rates of coincidence occurred at Level D at both P7 and S2, at just under 30% in each case.

3.23 At P3 and P5 teachers classified almost all the sampled pupils into just two levels, A and B at P3, and B and C at P5. The survey results, in contrast, distribute pupils over four separate levels: below A, A, B and C for P3; and A, B, C and D for P5. The differences between the teachers' and survey's 'well-established skills' attainment profiles are statistically significant. In particular, teachers judged 40% of P3 pupils to be at Level B while the survey estimates 50% as having well-established skills at that level. Teachers judged 2% of P3 pupils to be at Level C, and 10% of P5 pupils to be at Level D, while the survey puts the proportions at around 20% at each stage at these levels.

3.24 On the other hand, whilst under 10% of P3 and P5 pupils were judged by their teachers to be, respectively, below Level A or at Level A (but not yet B), the survey estimates the proportions at over 25% at P3 and just under 20% at P5. At P7 and S2, differences between the teachers' and survey's attainment profiles for well-established skills are again statistically significant, but the pattern is different. At these stages, both the survey results and teachers' judgements distribute pupils' attainment across several levels, but teachers' estimates are uniformly more generous to pupils than the survey results.

3.25 The match for a good start at a level, however, is very much closer. The survey and the teachers put around 70% and 78%, respectively, of P7 at Level D, and around 70% and 66%, respectively, of S2 at Level E. While the survey estimates just under 10% of P7 pupils and just under 20% of S2 pupils as not having at least made a good start with Level C and Level D skills (the levels below those expected), teachers' equivalent estimates are 4% and 13% respectively.

3.26 Coincidence rates varied across the authorities, with a range of roughly 50-65% at P3, 40-55% at P5, 45-65% at P7, and 40-65% at S2. These findings suggest that there are significant differences from one authority to another in the 'standards' being applied by teachers when they judge pupils' attainment levels.

3.27 At P7, Highland showed a significantly better than average agreement rate between its teachers and the survey, as did Inverclyde at S2. On the other hand, Inverclyde showed a relatively low agreement rate at P7, as did South Lanarkshire and East Ayrshire. East Ayrshire also shows a significantly lower than average agreement rate at S2, as did North Lanarkshire and West Lothian.

Gender differences in Reading

3.28 Girls performed better than boys across reading tasks and genres at Primary stages. This is illustrated in Figure 2.

3.29 Topic effects also seem to have come into play, with larger gender differences emerging for some tasks compared with others. This is consistent with the findings from the 2002 and 2003 AAP surveys in social subjects and science.

3.30 There were also significant gender differences evident in the level judgements of the teachers, with girls rated more highly for reading attainment than boys at every stage surveyed.

Figure 2
The gender gap in reading attainment
(% pupils attaining indicated levels or higher)

Changes in Reading attainment over time

3.31 The comparison between attainment in reading in 2001 ( AAP) and in 2005 ( SSA) is shown in Figure 3, which provides information about 'well-established skills' attainment for P7 and S2 only, because P3 and P5 did not feature in the survey in 2001. The data suggest that attainment has improved since 2001 at both stages and at all levels, with the exception of Level F at S2.

Figure 3
Reading attainment rates at P7 and S2 in 2005 and 2001
(% pupils having well-established skills or better at the given level)

3.32 Figures 4a and 4b compare the trends in reading attainment evident in teachers' level judgements (the 5-14 National Survey of Attainment) and AAP/ SSA survey results.

Figure 4a
Trends in reading attainment at P7
(% pupils 'having well-established skills' or better at the indicated levels)

Figure 4b
Trends in reading attainment at S2
(% pupils having 'well-established skills' or better at the indicated levels)

Note: National Survey results are 1998/99-2003/04 National Survey of 5-14 Attainment Levels, 2004/05 are sampled teacher judgements as provided in the SSA. AAP/ SSA results are from the AAP in 2001 and the SSA in 2005.

3.33 In both sets of data the trend in reading attainment is increasing, although, as before, the figures based on teachers' estimates are consistently higher than those from the AAP/ SSA. The exception is S2 attainment at Level F, where the attainment proportion as measured by the AAP in 2001 is slightly higher than the results from the national survey for that year.

Listening/Watching attainment

3.34 Twelve different CD or video-based tasks were used in the survey to assess listening/watching, covering Levels A-F. Four of the tasks involved questions for Levels A, B and C, and were used with P3 and P5 pupils. Four tasks had questions at Levels B, C and D and four had questions at Levels C, D and E. Both of these sets of written questions were used with pupils at P5, P7 and S2. Finally, four tasks had questions at Levels D, E and F and were used with pupils at P7 and S2. Where there was overlap at levels and stages, the same tasks and questions were used, so that progression could be evaluated.

3.35 Pupils' listening and watching skills were assessed within the practical component of the survey, in a subsample of schools. Field officers conducted the test sessions with three or four of the sampled pupils from each school visited. Between 35 and 75 pupils attempted each listening test, a total of around 300 pupils at each stage.

3.36 Survey results for listening are reported as average item facility values, which show the proportion of pupils that successfully answered each question, rather than as percentage scores on the whole set of questions for each task, since there were too few questions on any one task for percentage scores to be meaningful.

Knowledge About Language ( KAL)

3.39 Between 250 and 320 pupils at each stage answered orally questions that related to the Knowledge about Language strands in the 5-14 English language guidelines. The assessments covered understanding of terms such as verb, tense, fiction, etc. There were three mixed-level tests per stage. At P5, P7 and S2 the tests between them contained 15 items at each level ( BCD at P5, CDE at P7, DEF at S2). At P3 they contained a total of 27 Level B items and nine Level C items ( KAL does not feature at Level A). At Level B, a good proportion of P3 pupils and a very high proportion of P5 pupils correctly answered questions relating to relevant terms.

3.40 For Levels C and D, the proportion of P5 and P7 pupils successfully answering relevant questions increased steadily from stage to stage. On the other hand, there is relatively little difference between P7 and S2 in the proportions successfully answering questions at Level D, or in the proportions of pupils at S2 answering questions correctly at Levels D, E and F, that is between five and six out of ten.

3.41 Further analysis of these results would be needed to determine whether particular terms caused particular difficulties.

« Previous | Contents | Next »