« Previous | Contents | Next »
Listen
Programme for International Student Assessment (PISA) 2003: Initial Report on Scotland's Performance in Mathematics, Science and Reading
Executive summary
Introduction
The OECD Programme for International Student Assessment (PISA) assesses the performance in reading literacy, mathematical literacy and scientific literacy of 15 year old students in its member countries.
PISA examines young people's ability to use their knowledge and skills to meet real-life challenges rather than whether they have mastered a specific school curriculum.
In PISA 2003, the main subject assessed was mathematics, with reading and science forming the minor domains.
Scotland took part in PISA 2003 as an independent National Centre, meaning that it participated fully in all PISA activities as though it were a full country including separate quality monitoring and adjudication of test administration.
Over 275,000 students took part in PISA 2003 from 41 participating countries (all 30 OECD countries and 11 non-OECD 'partner' countries). Although the United Kingdom as a whole failed to meet the rigorous sampling criteria for the study (meaning that its results are not considered sufficiently reliable to be reported in full) Scotland and Northern Ireland did, separately, meet the criteria and their results are reported in an annex to the international report.
The intended sample for Scotland was of 108 schools and 35 pupils in each of these. The school sample was a stratified, random sample representative of all mainstream secondary schools in Scotland (local authority, grant-aided, and private schools). Of the 108 Scottish schools that were recruited to the study, 98 returned completed tests and questionnaires, a response rate of 90%. Just over 2,700 students completed tests and questionnaires.
This report presents results for Scotland and the 29 OECD countries that fully met the OECD criteria for acceptability. It does not report on results from the 'partner countries'.
This present report is concerned with the main set of test results from the study. Further themed reports planned for 2005 are likely to report on the questionnaire data gathered from schools and pupils, the relationships between these data and the results obtained in mathematics, and the domain of problem solving (which was additional in PISA 2003).
Student Proficiency in Mathematical Literacy
Scotland's mean score in mathematics is 524. This is significantly above the OECD average. Eight OECD countries have mean scores higher than Scotland but only Finland, Korea, and The Netherlands have mean scores that are significantly higher.
In mathematics literacy, male students outperformed female students in all countries except Iceland. For the 29 OECD countries as a whole, the average difference is close to 11 points. In Scotland it is 7 points.
PISA 2003 reports on student performance at six 'proficiency levels' - descriptions of the kind of mathematical competency demonstrated by students. Just below one-third (31%) of students across the OECD area as a whole performed at the top three proficiency levels (Level 4 or above) and 3.5% at the highest level, Level 6. In Scotland, 41.2% of students attained Level 4 or better and 3.9% achieved Level 6.
As well as assessing overall performance in mathematics, PISA 2003 assessed pupil performance in 4 'content areas': Space and shape (which draws upon the discipline of geometry); Change and relationships (which relates most closely to algebra); Quantity (which covers those aspects of mathematics bearing upon number); and Uncertainty (which lies within the area of probability and statistics). The mean scores for Scotland in each of the four content areas are significantly higher than those for the OECD as a whole.
Both in respect of the OECD taken as a whole and relative to the individual countries that constitute the OECD, Scottish students do very well in the content area of Uncertainty. Only one OECD country had a mean score in this area that was significantly higher than that of Scotland. Change and Relationships is the next strongest area. Only two OECD countries have higher mean scores than Scotland. In the other two content areas, Scotland's performance is good but not strong (nine countries have a mean score that is significantly higher than that of Scotland's in Space and Shape and five countries have a mean score that is significantly higher in Quantity) .
In each content area, male students attain higher mean scores than females both in the OECD as a whole and in Scotland.
It is not possible to compare overall performance in mathematics in PISA 2000 and 2003 (for any participating country). The only valid comparison is confined to the two content areas of Space and Shape and Change and Relationship. Performance could be compared across 25 OECD countries and Scotland.
The average score in Space and Shape across these 25 countries as a whole did not change significantly. On a common scale, the mean score in PISA 2000 was 494, while that in PISA 2003 was 496 points. Scotland showed a slight, though non-significant drop in performance, with a mean score in PISA 2000 of 511 points and in PISA 2003, of 507 points.
The average score in Change and Relationship showed a significant increase. On a common scale, the mean score in PISA 2000 was 488, while that in PISA 2003 was 499 points. Although Scotland's mean score in this content area (529) is one of the highest in PISA 2003, this mean decreased since PISA 2000 by 1 point.
PISA 2003 provides encouraging evidence that Scotland is succeeding in 'closing the gap in attainment'. One of the ways in which PISA measures equity in attainment is by comparing the score gap between students at the 75 th percentile level and those at the 25 th percentile. The smallest gap (115 points) is found in Finland. Scotland has the same score gap as Finland.
Student Proficiency in Reading Literacy
Scotland's mean score of 516 is (statistically) significantly above that for the OECD as a whole. Only the three top performing OECD countries on the PISA 2003 reading literacy scale (Finland, and Korea and Canada) had mean reading scores that were significantly higher than that of Scotland.
In every OECD country the mean reading score for female students is significantly higher than that for male students. On average, across the OECD, the difference in performance between female and male students is 34 points. In Scotland, the difference was 24 points.
Overall, 55% of students in the OECD attained the top 3 proficiency levels in reading literacy (Levels 3,4 and 5) and 8% attained the highest level, Level 5. In Scotland, 68% performed at the top three levels and 9% at Level 5.
Between PISA 2000 and 2003 mean score in reading literacy in Scotland dropped by 11 points - a drop that is of borderline statistical significance (i.e. significant at the 10% level of significance but not at the 5% level). 1
Although there has been a drop in Scotland's mean score, this drop has not been experienced across all ranges of ability. Despite the trend across the OECD as a whole, the performance of our lowest 25% of students has improved (though not significantly).
Scotland's score gap in reading between students at the 75 th percentile level and those at the 25 th percentile level is 116. Only 4 countries had a score gap that was narrower than this. The smallest gap (105 points) is again found in Finland.
In PISA 2000, Scotland's score gap (between the 25 th and 75 th percentiles) in reading was 136 points. This means that between 2000 and 2003, our score gap narrowed by 15%. No other OECD country narrowed its score gap by as much as this.
Student Proficiency in Scientific Literacy
Scotland's mean score in science literacy (514) is significantly above the OECD average. Of the eight OECD countries with higher mean scores than Scotland, only the top three countries (Finland, Japan and Korea) can be said with certainty to have students who do better, on average, than those in Scotland.
While in general male students did better than females in the OECD at large, this is not universally the case. In Scotland there is a difference of 8 points in favour of male students, but this difference is not statistically significant.
Scotland's mean science score fell between PISA 2000 and PISA 2003, but the 8 point drop, from 522 to 514 points, is not statistically significant.
The drop in performance in Scotland was more or less uniform across the whole ability range. Performance by students at the very lowest levels of ability, those at the 5 th percentile and 10 th percentile levels fell by approximately 5 points, while those at the 25 th 75 th 90 th and 95 th percentile levels fell by between 7 and 9 points.
In Scotland, the score gap between students at the 75 th percentile level and those at the 25 th level was 140 points. Scotland's rating on this equity scale is neither particularly good nor particularly bad, compared with the OECD countries.
In PISA 2000, Scotland's score gap for science was 142. This means that between 2000 and 2003 we narrowed the gap by only 2 score points. However, only three OECD countries managed to narrow their gap in science between these two sweeps of PISA: Belgium (by 1 score point); Denmark (by 3 score points) and Hungary (by 8 score points). In all other OECD countries the gap increased.
« Previous | Contents | Next »