« Previous | Contents | Next »
Listen
Section 2 Survey Overview
2.1 This document presents the findings of the first Scottish Survey of Achievement ( SSA). The SSA differs from its antecedent, the Assessment of Achievement Programme ( AAP), in its broader aim to report pupil attainment at the level of local authorities as well as nationally
2.2 The tests and tasks in the SSA survey are designed to assess pupils' progress and attainment against the levels described in 5-14 curriculum guidelines, A-F. The 2005 SSA focused on English Language and on core skills applied within a language context, so the relevant guidelines used were English Language, Mathematics, and Information and Communications Technology. Reference was also made to the learning outcomes that define Core Skills for National Qualifications, published by SQA.
2.3 Where the results are reported for 'expected' levels, these are as follows: Level A attainable for almost all of P3; Level B attainable by most of P4; Level C attainable by most of P6; Level D attainable by most of P7; Level E attainable by most of S2 *.
2.4 Just over 28,000 randomly sampled pupils in 1,300 mainstream schools across Scotland were assessed, that is just over 7,000 at each of the three primary stages and almost 6,500 at S2. Testing took place in May and June 2005.
2.5 In 16 local authorities *, school and pupil samples were boosted to allow some reporting at local authority level in reading and numeracy. Non-participation on the part of some schools, combined with pupil absence in those schools that did take part in the survey, reduced the intended pupil sample sizes in some authorities, but only in one or two cases did this threaten the representativeness of the sample, and only at S2. Margins of error for national attainment estimates are around two percentage points, rising to between five and seven percentage points for authority estimates (depending on stage and level).
2.6 Assessment results are not reported for individual schools or for individual pupils, and indeed the pupils assessed in the survey will remain anonymous.
2.7 All the participating schools undertook written assessments, assessing both reading and numeracy with separate groups of pupils. In addition to being assessed in reading or numeracy, over a quarter of the pupils also provided a piece of school-based writing for assessment. A team of teachers who had been nominated by their local authorities subsequently moderated a representative sample of the writing submitted.
2.8 Schools organised and supervised their own written test sessions within the given period, mid-May to mid-June, but administration of the practical assessments was the responsibility of field officers. The field officers, all practising teachers recruited from local authorities throughout the country, were given one day of task orientation prior to the survey, and then worked in pairs, visiting one school each per day for five days to carry out their assessments.
2.9 Pupils took part in practical assessments in around a quarter of the survey schools, attempting tasks in Listening, Talking, Knowledge about Language or Writer's Craft. They worked in pairs on ICT tasks, and in pairs or groups on Problem-solving tasks and Working with Others.
2.10 The schools involved in the practical assessments were not selected entirely at random. They were selected for their convenient location (ease of field officer access) and their relatively large size (at least 20 survey pupils available at one stage). In consequence, the resulting pupil samples for practical tasks were not fully representative of their populations. The results of practical assessments are nevertheless indicative and of interest.
2.11 The 5-14 national bank of assessments provided most of the numeracy test items and some of the reading assessments used in the survey. All the practical tasks, including Writer's Craft tasks, were newly developed. New reading and numeracy tests were pre-tested in advance of the survey and in addition practising teachers were asked to check that the materials were suitable for pupils at the relevant levels and stages to be assessed.
2.12 To provide a context for the survey findings, teachers and pupils in the participating schools were invited to complete questionnaires inviting information and views about teaching, learning and assessment in English and mathematics.
2.13 In addition, for each pupil sampled in the survey, the school was asked to provide an estimate of the level that the pupil had attained most recently in reading, writing and mathematics, on the basis of evidence from classwork and national assessments.
2.14 The survey was managed by the Scottish Executive Education Department ( SEED) and designed and carried out in partnership by a team from the Scottish Qualifications Authority ( SQA) and Learning and Teaching Scotland, with support from specialist survey, sampling and statistics consultants.
2.15 SEED convened a survey planning group with membership from partner organisations and external consultants, local authorities, and HMIE. Assessment Europe was responsible for the survey design and for school sampling; pupil sampling was undertaken by SEED. Learning and Teaching Scotland convened an English language subject reference group with membership from schools, local authorities, teacher education, SQA and HMIE. The Scottish Qualifications Authority was responsible for new task development, for producing test booklets and distributing these to schools, for receiving test booklets back from the schools, and for preparing coding sheets for data processing. School and pupil questionnaires were developed by the Scottish Executive Education Department in collaboration with SQA. Pupil response data were keyed by Datapro Data Preparation Ltd. and Lindata Services, and analysed by Assessment Europe.
« Previous | Contents | Next »