The Scottish Government

« Previous | Contents | Next »

Listen

1. Introduction

1.1 Overview

The aims of this last survey in the Assessment of Achievement Programme ( AAP) series ¹ were to:

• assess the mathematics attainment of pupils in P3, P5, P7 and S2;
• assess the core skills attainment of pupils in P3, P5, P7 and S2;
• compare attainment across the four stages and between boys and girls;
• compare attainment in 2004 with that in 2000;
• report attainment in terms of 5-14 levels whenever possible;
• explore pupils' perceptions about mathematics;
• provide a learning context against which to reflect on the attainment and attitude findings.

The survey involved around 15500 pupils in almost 850 mainstream schools across Scotland (see Appendix B for sampling details), and testing took place in May and June 2004. Around two-thirds of the pupils attempted written and mental mathematics tasks, between them attempting more than one thousand different atomistic tasks across the six levels, while the remainder participated in core skills reading and writing assessment. In over two-fifths of the survey schools the 'mathematics' pupils also took part in practical mathematics assessments while the 'core skills' pupils undertook ICT tasks.

To provide a context for the attainment findings, both pupils and teachers were invited to complete subject-relevant questionnaires. Those pupils attempting mathematics assessments were invited to complete questionnaires seeking their views about their learning experiences in mathematics and exploring their perceptions about the importance of mathematics in the world of work. In half the participating schools, managers and class/subject teachers were invited to complete questionnaires about the resources available to them for mathematics and about their subject provision.

1.2 The assessment framework and task selection/development

1.2.1 Mathematics

The survey was designed primarily to assess and to report pupil attainment for mathematics and for mental mathematics, in terms of specific 5-14 levels at each stage: P3 attainment would be assessed at Levels A and B, P5 at Levels B, C and D, P7 at Levels C, D and E, and S2 at Levels D, E and F. In addition, it was planned to offer comment on pupil performance on the 'written' items for each outcome separately, in the form of average scores on items classified at the same level in each area.

The current 5-14 framework for mathematics, as described in the National Guidelines ², comprises the following outcomes and strands:

Information handling:

• Collect information
• Organise information
• Display information
• Interpret information
• Probability

Number, Money & Measurement:

• Range and type of numbers
• Money
• Add and subtract
• Multiply and divide
• Round numbers
• Fractions, percentages and ratio
• Patterns and sequences
• Measure and estimate
• Time
• Perimeter, formulae and scales
• Functions and equations
• Algebra

Shape, Position & Movement:

• Range of shapes
• Position and movement
• Symmetry
• Angle

Problem solving and enquiry:

• Problem solving.

A Mathematics Reference Group, comprising subject specialists, practising primary teachers, practising secondary mathematics teachers and other education professionals (membership details are given in Appendix A), advised on how best to implement this framework to meet survey aims within the inevitable practical constraints of this large-scale exercise. The way in which the general framework was eventually represented in the set of atomistic assessment tasks ('test items') administered in the survey is illustrated in Chapter 2.

The 5-14 National Assessment Bank furnished the majority of the 1000+ test items needed for the survey (for examples see Chapter 2). Remaining gaps in intended coverage were filled through new item development, organised by staff in the SQA and involving the contributions of practising primary teachers and secondary mathematics teachers. All newly developed items were piloted before survey use.

In addition, a small number of multi-item 'mathematics literacy' tasks were developed. These were intended to offer pupils the opportunity to apply their numeracy skills in response to a coherent set of test questions, all linked to the same paper-based stimulus information.

Over-time attainment comparisons are offered on the basis of 'written' items previously administered in the 2000 mathematics survey and reused on this occasion. Given that P4 was not assessed in the 2004 survey, and given that Level F did not feature strongly in the 2000 survey, over-time comparisons are presented for P7 and S2 only at Levels C, D and E.

1.2.2 Practical mathematics

This survey included an important component of practical assessment in mathematics. Practical mathematics tasks, administered by itinerant field officers, directed pupils to work through a number of atomistic but linked activities. The four tasks focused, respectively, on money, time and measure, fractions, percentages and ratio, and shape, angle and direction.

1.2.3 Core skills

In addition to numeracy, which naturally featured in the general mathematics assessment, the core skills of reading, writing and using ICT were also assessed within the survey, with tasks set within a mathematics context.

Several reading and writing tasks were newly developed for use here. The reading tasks took the same general form as those used in previous AAP subject surveys, including English Language, Social Subjects and Science, and in National Assessments, viz. a source text, whose length and difficulty increased with level, followed by a series of test questions exploring comprehension of that text. The difference between these new tasks and those already available from previous surveys is the fact that here the texts focused on some issue or personage in mathematics, though without any requirement for prior knowledge and understanding of mathematics per se. Writing tasks were based on the same stimulus material as the reading tasks.

A small number of ICT tasks were also developed for use in the survey. These required pupils to work with laptop computers supplied by assessing field officers, undertaking various activities involving text manipulation and use of spreadsheets.

A number of experimental problem solving tasks were also piloted in the survey.

1.3 Task administration and marking

All the survey pupils were involved in some form of written assessment, and around one-third also participated in some form of practical assessment.

The pupils' own teachers organised and supervised written test sessions, but the practical assessments were the responsibility of itinerant field officers (see section 1.3.3). Once the written testing had taken place, the survey schools returned completed scripts to SQA for processing and marking. Field officers completed their assessments of pupils as they engaged in the practical activities.

1.3.1 Mathematics 'written' assessment

The atomistic written mathematics test items were presented to pupils in the form of mixed-level mixed-outcome test booklets, lasting around 40-45 minutes at P3/P5 and 50-60 minutes at P7/S2. It was intended that each pupil would attempt two such booklets, booklet pairs being randomly allocated to them. Pupils also undertook a mental test before they started on their written items. The schools themselves organised and supervised the assessment sessions, which were to take place within the period mid-May to mid-June 2004. Test booklets were despatched to and received back from schools by SQA. Completed booklets were processed centrally by SQA, with students marking pupils' responses. The 'mathematical literacy' tasks were marked by the field officers who administered them (see section 1.3.3).

1.3.2 Reading and writing

It was intended that each pupil would attempt two different reading booklets, at different levels and assigned at random, plus a single writing task. Reading tasks were presented to pupils in the form of test booklets, one reading task forming a single booklet. A source booklet contained passages for the two reading tasks, and two associated writing tasks. Pupils had the freedom to choose one or other of the writing tasks. Reading booklets were designed to have the same general time requirement as the mathematics booklets, and it was expected that schools would organise both types of assessment to occur in the same test sessions. Writing was undertaken in a third session. Completed reading booklets, like completed mathematics booklets, were processed centrally, using students to mark responses and prepare mark sheets for keying. Random samples of pupils' writing were selected for evaluation by the practising teachers who had served as field officers in the survey (see section 1.3.3).

1.3.3 Practical tasks

Education authorities throughout the country provided the 137 practising teachers, both primary class teachers and secondary mathematics teachers, who served as field officers. The field officers worked in pairs, half at P3/P5 and half at P7/S2, spending five days visiting schools located within reasonable travelling distance of their home areas. In preparation for this work, the field officers were given a day of task orientation in May/June 2004. They then spent one day in each of their five assigned schools, setting up and supervising the practical sessions, rating pupils on the spot, using checklists to record observations and judgments as pupils worked through their practical mathematics (see Chapter 3) and ICT tasks (see Chapter 4), or participated in experimental group discussions. Their involvement in the practical assessment was formally ended in a debriefing day held in June 2004.

1.4 Reporting pupil attainment

Pupils' mathematics attainment, as well as their reading and writing attainment, is reported in terms of the percentage of pupils at each stage who were deemed to have attained specific 5-14 levels. In mathematics and reading, cut-off scores were applied to identify attainment groups (see Appendix B, section B.4, for details), whereas in writing pupils were classified into levels through the application of a 'best fit' evaluation scheme (further details are given in Chapter 4). By their nature, the 'mathematical literacy' tasks, the practical mathematics tasks and the ICT tasks do not lend themselves to the same kind of level-based reporting. In these cases, pupil performance is either averaged over test items or behaviours classified at particular levels, and average facility values or rating distributions given, or performance is reported item by item.

The resulting attainment findings are presented in Chapters 2, 3 and 4. Chapter 2 focuses on the assessment of written and mental mathematics, Chapter 3 presents the results of the assessment of practical skills in mathematics, and Chapter 4 presents the results of the core skills assessment.

Chapter 5 presents findings from the pupil questionnaire enquiries, while Chapter 6 presents findings from the school questionnaire. Finally, Chapter 7 summarises and reflects on the principal findings of the survey, and addresses some of the issues arising.

« Previous | Contents | Next »