« Previous | Contents | Next »
Listen
PROGRAMME FOR INTERNATIONAL STUDENT ASSESSMENT 2000 - SCOTLAND ANALYSIS
SECTION 1 - BACKGROUND OF STUDY
1. PISA looked at young people's ability to use their knowledge and skills in order to meet real-life challenges rather than how well they had mastered a specific school curriculum. For instance, PISA defines reading literacy as the ability to understand, use and reflect upon written texts to participate effectively in life.
2. PISA provides a broad assessment of comparative learning outcomes towards the end of compulsory schooling. This can guide policy decisions and provide insights into the factors that contribute to the development of knowledge and skills, and the extent to which these factors are common to different countries. In the 2003 and 2006 studies a proportion of the test items from the previous study are retained to provide a measure of performance change over time.
3. The results of PISA 2000 were published by the OECD in 2001 in ' Knowledge and Skills for Life: First Results from PISA 2000'. There have been two thematic reports since; ' Reading for Change: Performance and Engagement Across Countries' and 'Literacy Skills for the World of Tomorrow'. Scotland took part in PISA as part of the UK 1. This meant that the Scottish results were not disaggregated in the international reports. Instead, our performance in the 3 subjects was published in our own national report in January 2002. The purpose of the analysis that is described below is to provide further interrogation of our data. The analysis is not, however, exhaustive and we encourage researchers to undertake further analysis by accessing the database at www.pisa.oecd.org2.
1. Wales did not participate in PISA 2000 so throughout this report the reference to the UK is to Scotland, England and Northern Ireland.
2. The subnational code for Scotland is 1
Design of PISA
4. In 2000 over a quarter of a million students took part in PISA from the 32 participating countries. A further 13 countries undertook PISA in 2002 and the results for these countries are to be reported separately. PISA used pencil and paper assessments, lasting two hours for each student, including multiple choice questions and questions requiring students to construct their answers. Each student's assessment tasks were drawn from a total of seven hours of assessment items, with different students taking different combinations of these items. Students also completed a background questionnaire that took about 30 minutes and head teachers completed a questionnaire about their schools. This analysis concentrates on student assessments and questionnaires.
5. In Scotland, the intended school sample was 120 and completed tests and questionnaires were received from 99 schools, an 82 per cent response rate, which was within the PISA sampling limit. The sample of schools was drawn as a stratified, random sample representative of all secondary schools in Scotland. In total over 2,500 students in the sample schools completed tests and questionnaires, the sample drawn randomly from students with a date of birth during a specified period.
6. The PISA scores for each subject area were scaled so that across the OECD the average score is 500 and around two-thirds of students achieve between 400 and 600 points.
Reading Literacy
7. Reading literacy is the ability to understand, use and reflect on written texts in order to achieve one's goals, to develop one's own knowledge and potential, and to participate effectively in society. This definition goes beyond the notion that reading means decoding written material and literal comprehension. Rather, reading also incorporates understanding and reflecting on texts, for a variety of reasons and in a variety of contexts. PISA's assessment of reading literacy reflects three dimensions: aspect of reading task; form of reading material; and the use for which the text is constructed.
8. Reading literacy was measured on three scales - on retrieving information, on interpreting, and on reflecting and evaluating. The scores on each scale represent degrees of proficiency in particular aspects of reading literacy. Each of the three reading literacy scales is divided into six levels of knowledge and skills. Students at a particular level not only demonstrate the knowledge and skills associated with that level but also the proficiencies required at lower levels.
9. Students proficient at Level 5 (over 625 points) are capable of completing sophisticated reading tasks, such as managing information that is difficult to find in unfamiliar texts; showing detailed understanding of such texts and inferring which information in the text is relevant to the task; and being able to evaluate critically and build hypotheses, draw on specialised knowledge, and accommodate concepts that may be contrary to expectations.
10. Students proficient at Level 4 (553 to 625 points) are capable of difficult reading tasks, such as locating embedded information, construing meaning from nuances of language and critically evaluating a text.
11. Students proficient at Level 3 (481 to 552 points) are capable of reading tasks of moderate complexity, such as locating multiple pieces of information, drawing links between different parts of the text, and relating it to familiar everyday knowledge.
12. Students proficient at Level 2 (408 to 480 points) are capable of basic reading tasks, such as locating straightforward information, making low-level inferences of various types, deciding what a well-defined part of the text means, and using some outside knowledge to understand it.
13. Students proficient at Level 1 (335 to 407 points) are capable of completing only the least complex reading tasks developed for PISA, such as locating a single piece of information, identifying the main theme of a text or making a simple connection with everyday knowledge.
14. Students performing below Level 1 (below 335 points) are not able to show the most basic skills that PISA sought to measure. Such performance should not be interpreted to mean those students have no literacy skills at all, but they could be said to have serious deficiencies in their ability to use reading literacy as a tool for the acquisition of knowledge and skills in other areas.
Mathematical and Scientific Literacy
15. Mathematical literacy in PISA concerns students' ability to recognise and interpret mathematical problems encountered in their world, to translate these problems into a mathematical context, and to use mathematical knowledge and procedures to solve the problems within their mathematical context. It also involves the way that students interpret the results in terms of the original problem, reflect upon the methods applied, and formulate and communicate the outcomes.
16. Scientific literacy reflects students' ability to use scientific knowledge, to recognise scientific questions and to identify what is involved in scientific investigations, to relate scientific data to claims and conclusions, and to communicate these aspects of science.
Analysis
17. The analysis contained within this document is based largely on the relationships between reading literacy scores and responses to the student questionnaires. Reading scores were used because this was the main subject assessed. No causal links are implied, as many factors not covered by the questionnaires are likely to influence the performance of individual pupils. However, the analysis aims to inform policy makers of factors which are to some extent correlated with test scores, so that policy in these areas may be further explored.
18. Standard errors naturally occur with the methodology used for sampling and testing. However, these are not high (see tables 1 & 2), and so have been ignored in subsequent tables.
19. International comparisons are taken from the OECD publication: Education at a Glance 2002. Calculation of indices included in the tables is explained in the annex to the OECD publication. Scotland, UK and OECD figures have been extracted from the OECD PISA database.
Further Information
20. More details can be found in the Scottish PISA Report and in the OECD executive summary of First Results from PISA: Knowledge and Skills for Life:
http://www.scotland.gov.uk/library3/education/pisa-00.asp
http://www.pisa.oecd.org/Docs/Download/PISAExeSummary.pdf
21. Details on methodology can be found in the annex to Education at a Glance, and on the PISA website, from which data can also be downloaded:
http://www.oecd.org/els/education/eag2002
http://www.pisa.oecd.org
22. Public enquiries (non-media) about the information contained in this Statistics Publication Notice should be addressed to Donna Hosie, Assistant Statistician, The Scottish Executive Education Department, Education and Children Statistics Unit, 1-A Victoria Quay, EH6 6QQ. Tel. 0131 244 0304 or email to: educ.outcomes.stats@scotland.gsi.gov.uk
Media enquiries about the information in this Statistics Publication Notice should be addressed to Marion MacKay (0131) 244 3070.
« Previous | Contents | Next »