The Scottish Government

« Previous | Contents |

Listen

Programme for International Student Assessment ( PISA) 2006: Highlights from Scotland's Results

© OECD 2007

What is PISA?

The OECD Programme for International Student Assessment ( PISA) assesses the performance of 15 year-old students in three domains, reading literacy, mathematical literacy and scientific literacy through a rolling programme of tests taking place every three years. PISA focuses on testing the knowledge and skills required for participation in society and assessing the extent to which students can apply skills gained in school in everyday adult life thus moving beyond the student's ability to master the school curriculum.

Each cycle of assessment emphasises one of the three specific domains included although the others are still tested. In 2006 the emphasis was on scientific literacy. Alongside the test questionnaires students and head teachers also complete context questionnaires providing socio-economic data alongside data on attitudes and motivations. In 2006 101 schools and around 2,700 students took part in the assessment. The fieldwork in Scotland was undertaken by the Scottish Council for Research in Education ( SCRE) in March 2006.

The first round of PISA took place in 2000. 43 countries participated in the 2000 round rising to 57 in 2006. All 30 OECD members participate but the number of non-member countries taking part is growing rapidly. This paper highlights the main Scottish results in the context of the results of other OECD countries.

What do we mean by scientific literacy ?

Source: OECD

How did Scotland perform overall?

In SCIENCE, the main focus in 2006, Scotland's overall score was 515. This was significantly higher than the OECD mean score of 500

While in MATHS Scotland's mean score of 506 was above the OECD average of 498

And in READING Scotland's mean score of 499 was also above the OECD average of 492

The vast majority of Scottish students exceed basic levels of attainment in science

The lowest levels of attainment in PISA are below level 1 and level 1. Scotland's performance at both of these levels can be compared to other countries however it is important only to recognise differences that are statistically significant. 3.6% of Scottish students scored below level 1 and 11% at level 1 compared to OECD averages of 5.2% and 14.1%. The only country that significantly outperforms Scotland in this respect is Finland where 0.5% and 3.6% of students score below level 1 and at level 1 respectively. The percentages of students scoring below level 2 in total in Canada, Japan and New Zealand, all countries outscoring Scotland overall ranged from 10% in Canada to 13.7% in New Zealand. In other parts of the UK, the percentage falling below level two was 16.7% in England, 18.1% in Wales and 20.3% in Northern Ireland. None of these scores are significantly different to Scotland.

At higher levels of achievement in science Scottish students also perform well

As with the lower levels of proficiency, Scotland's performance at the higher levels of 4, 5, and 6 can also be compared to other countries and significant differences noted at each level. Scotland has 10.1% of students achieving level 5 and 2.4% achieving level 6 compared to OECD averages of 7.7% and 1.3%. Only one country, Finland has significantly more students, 17%, scoring above Scotland at level 5. Looking at level 4, Scotland has 20.7% of its students in this category which is around the OECD average of 20.3% however a number of countries significantly outperform Scotland at this level, namely, Australia (24.6%), Canada (27.7%), Finland (32.2%) Japan (27%), Korea (25.5%), Netherlands (25.8%).

Figure 1: Percentage of students at selected levels on the science scale

There is a gap between the highest and lowest scorers in Science

We use the gap between the 25 ^th percentile and the 75 ^th percentile to measure the degree of equity in the Scottish system by looking at the gap between those who do well and those who do not. The gap between high and low scorers in science is 139, close to the OECD average of 134 indicating an average performance. However a narrower gap can be seen where overall attainment is uniformly low, as is the case for Mexico and Turkey. The smallest gap in a high scoring country is that of 116 achieved by Finland

Figure 2: Gap between high and low scorers in science

The vast majority of students exceed basic levels of attainment in maths

15.5% of Scottish students scored below level 2 compared to an OECD average of 21.3%. The only country that significantly outperforms Scotland at both below level 1 and level 1 is Finland where 1.1% and 4.8% of students score below level 1 and level 1 respectively compared to 3.8% and 11.7% for Scotland. Canada and Korea also do significantly better at level 1 with 8% and 6.5% scoring at level 1. In other parts of the UK, the percentage falling below level 1 was 6% in England, 6% in Wales and 7.3% in Northern Ireland; the last of these results is significantly different. At level 1 the scores were 13.9% in England, 16.1% in Wales and 15.3% in Northern Ireland. Scotland's score is significantly different to that of Wales at level 1.

Figure 3: Percentage of students at selected levels on the maths scale

At higher levels of achievement in maths Scottish students performance is average

Scotland has 9.4% of students achieving level 5 and 2.7% achieving level 6 compared to OECD averages of 10% and 3.3%. A number of countries, Belgium (16%), Canada (13.6%), Finland (18.1%), Japan (13.5%), Korea (18%), Netherlands (15.8%), New Zealand (13.2%) and Switzerland (15.9%) scored significantly above Scotland at level 5. At level 6 a number of the same countries also significantly outscored Scotland, namely Belgium (6.4%), Finland (6.3%), Korea (9.1%), Netherlands (5.4%), New Zealand (5.7%) and Switzerland (6.8%) plus the Czech Republic (6%). Looking at level 4, Scotland has 20% of its students in this category which is around the OECD average of 19.1% with three countries significantly outperforming Scotland at this level, Canada (25.1%), Finland (28.1%) and Korea (25.5%).

Figure 4: Gap between high and low scorers in maths

And we also have a narrow gap between the lower and higher performers in maths

In maths the score gap between students at the 75 ^th percentile and those at the 25 ^th percentile is 117. Of those countries with overall higher means than Scotland, 4 have similar or lower score gaps, Canada, Finland, Denmark and Ireland.

Many students exceed basic levels of attainment in reading

16.7% of Scottish students scored below level 2 compared to an OECD average of 20.1%. Only two countries significantly outperform Scotland at both below level 1 and level 1; Finland with scores of 0.8% and 4% and Korea with scores of 1.4% and 4.3% compared to Scottish scores of 5.2% and 11.5% respectively. In addition Canada significantly outscores Scotland at level 1 with a score of 7.6% In other parts of the UK, the percentage falling below level 1 was 6.8% in England, 7.6% in Wales and 7.7% in Northern Ireland; none of these results is significantly different from Scotland. At level 1 the scores were 12.1% in England, 14.4% in Wales and 13.2% in Northern Ireland. Again, none were significantly different to Scotland.

Figure 5: Percentage of students at each level on the reading scale

At higher levels of achievement in reading Scottish students performance is good

In reading the scale stops at level 5. Scotland has 8.5% of students achieving level 5 compared to OECD average of 8.6%. Four countries, Canada (14.5%), Finland (16.7%), Korea (21.7%) and New Zealand (15.9%) scored above Scotland at level 5. The same four countries also significantly outscored Scotland at level 4. Looking at level 3, Scotland has 30.9% of its students in this category which is around the OECD average of 27.8%. No country performs significantly better than Scotland at this level.

And we also have a narrow gap between the lower and higher performers in reading

In reading the score gap between students at the 75 ^th percentile and those at the 25 ^th percentile is 125. Only 3 higher scoring countries have a narrower gap, Finland, Denmark and Korea. The results in maths and reading point to a relatively equitable education system in Scotland.

Figure 6: Gap between high and low scorers in reading

How does our 2006 performance compare to previous years?

In SCIENCE our performance apparently has not changed while others have improved

However changes in individual country scores over time are not directly comparable as the test framework for science was changed for the 2006 test. So while in 2003 3 countries could be said to have students who performed significantly better than those in Scotland compared to 4 in 2006, this difference could be due to the use of the revised testing framework.

While in MATHS our performance has declined

Figure 7: Changes in mean scores in maths between 2003 and 2006

In 2003 3 countries (Finland, Korea, and the Netherlands) had mean scores that were significantly higher compared to 8 in 2006. Scotland's mean score has dropped by 18 points since 2003 recording the largest drop in scores of all the OECD countries (France is next with a drop of 15 and Japan third with a drop of 11). This is a significant decline.

In reading our results have also declined

Figure 8: Changes in mean scores in reading between 2003 and 2006

In 2003, 3 countries Finland, and Korea and Canada had mean reading scores that were significantly higher than that of Scotland compared to 5 in 2006. Scotland's results have decreased since 2003 dropping by 17 points, a significant drop. Most other countries scores remained the same with Norway dropping by 16 and Korea increasing by 22.

Our attainment gap in science is unchanged

In science the score gap between students at the 75th percentile level and those at the 25th level was 139 which is similar to 2003 and 2000. Scotland's rating on this equity scale is neither particularly good nor particularly bad, compared with the other OECD countries.

While the attainment gap in maths was also unchanged

In maths the score gap between students at the 75th percentile level and those at the 25th level was 117 in 2006. This was one of the smallest gaps recorded in line with Finland, Canada, Denmark, and Ireland who achieved similar performances with gaps ranging from 111 to 116. In 2003, the gap between performance at the 75 ^th and 25 ^th percentile levels in Scotland of 115 was the smallest in the OECD, along with Finland.

And in reading the score gap has increased slightly

In PISA 2000, Scotland's score gap was 136 points while in PISA 2003 it was 116. This means that between 2000 and 2003, the score gap narrowed by 15%. In 2006 the score gap was 125, a minor increase on 2000.

Our performance at the upper levels of attainment has reduced in maths and reading

In maths, the total percentage of students scoring at level 3 and above fell from 69% in 2003 to 60% in 2006. Amongst males the percentage dropped from 71% to 64% while amongst females it dropped from 68% to 56%. The percentage of students scoring at level 3 and above is still greater than the OECD average of 57% although just above the OECD average female score of 55%

In reading 60% of students scored at level 3 or above in 2006 compared to 68% in 2003. Split across males and females, the female percentage reaching this level fell from 73% to 65% while the male percentage fell from 63% to 55%. Compared to OECD averages in 2006 of 50% for males and 65% for females, Scottish males and females are still scoring at or above the OECD average.

Is there a difference between boys and girls?

Some score differences between boys and girls still exist

Figure 9: Score differences between boys and girls in science, maths and reading

Gender does not make a difference in science scores

In science, the gender difference is insignificant with males outscoring females by 4 points. This is in line with the OECD average of 2 points and a narrowing of the 2003 gap which was 8 points.

In maths, gender differences have become significant

Gender differences are slightly higher than the OECD average with boys recording a significant 16 point advantage compared to an OECD average of 11. This is an increase from 2003 when gender differences were marginal with boys showing a 7 point advantage. This is further highlighted when comparing male and female performance between 2003 and 2006. Male performance has dropped by 14 while female performance has dropped by 23. Both of these drops are significant.

For reading, gender differences are significant and unchanged

The gender difference between males and females is 2006 is 26 in favour of females compared to an OECD average of 38. This is a similar result to 2003.

Performance in science and the impact of socio-economic background

Compared to other countries, socio-economic status is a major determinant of attainment

Scotland demonstrates an above average level of performance in science but also an above average impact of socio-economic background. This means that the economic, social and cultural background of Scottish students had a larger impact on their performance than in many other countries. Close to 16% of the total variance in student performance can be explained by their score on the index of economic, social and cultural status ( ESCS). This compares to 6.7% for Iceland and 8.3% for Finland and Canada but is lower than France at 21.2% and Germany at 19%.

See Figure 10 below which focuses on those countries whose score on the ESCS is significant.

Figure 10: Strength of the relationship between student performance and the ESCS

How do students feel about their studies?

Scottish students have neutral feelings about their abilities

Alongside the test items, students were also asked a range of questions to establish their beliefs about their own abilities in Science which were used produce two indices:

1) The index of Self Efficacy - this examines student's confidence in overcoming difficulties in science

2) The index of Self Concept - this examines student's belief in their academic abilities

A plus score on the indices indicates higher than average Self Efficacy and Self Concept.

Scottish students score close to average on both indices. Compared to students from those countries that score higher than Scotland in science, Canada, Finland, Japan and New Zealand the attitudes of Scottish students are in line with other high performing countries. This comparison highlights the cultural component of any link between Self Efficacy, Self Concept and attainment.

Figure 11: Top country scores in Self Efficacy and Self Concept

and do not express a high level of interest in science

On the index of general interest in science, Scotland's minus score indicates a low level of interest. This low level of interest is shared by other high performing countries such as Finland, The Netherlands and Australia while students in lower performing countries such as Turkey, Portugal and Mexico display a high level of interest.

Figure 12: Scores on the index of general interest

How will the results of PISA be used?

Want to know more?

For more information about the PISA 2006 study in Scotland, please contact Audrey MacDougall (tel: 0131-244-0906; e-mail: Audrey.macdougall@scotland.gsi.gov.uk).

For more information about the PISA 2006 International Report please go to the relevant OECD website at www.pisa.oecd.org

Other international studies

If you would like to learn more about international studies, that include Scotland, please visit, www.scotland.gov.uk/Topics/Education/Schools/Excellence/IE

The Scottish Survey of Achievement

The Scottish Survey of Achievement ( SSA) uses a sample survey to find out how well students are learning in Scotland as a whole. The information is used to help plan for improvement to support quality learning and teaching. For more information about the SSA and results, please visit Learning and Teaching Scotland's assessment website, www.ltscotland.org.uk/assess/of/ssa

HMIE

HM Inspectors of Education ( HMIE) promote sustainable improvements in standards, quality and achievements for all learners in Scottish education through independent evaluation. If you

would like information about inspections of Scottish schools, or are interested in knowing more about good practice in Scottish education please visit, www.hmie.gov.uk

Your child's progress and achievements

If you would like to know more about how your own child is progressing, or you have concerns about their learning, you should get in touch with the school and talk to your child's teachers.

You will find more information about education in Scotland, and advice on supporting your child's learning on the Parentzone website, www.parentzonescotland.gov.uk

If you have any enquiries about this web only education research report please contact:

Tel: 0131 244-0894
Fax: 0131 244-5581
Email: recs.admin@scotland.gsi.gov.uk
Website: www.scotland.gov.uk/insight

« Previous | Contents |