« Previous | Contents | Next »
Listen
Survey objectives and design
The third listed objective for the 2007 Scottish Survey of Achievement, as described in the survey design document, was:
To assess and report pupils' Science Knowledge and Understanding attainment by 5 to 14 levels at P5 and P7 for pupils taught science through the medium of Gaelic.
Given the very small size of the Gaelic population, it was clear that this objective could only sensibly be addressed by assessing every pupil at P5 and P7 who was studying science through the medium of Gaelic that year. In other words the 'Gaelic-medium study' would take the form of a census survey. Identifying the relevant pupil population at each stage emerged as a major challenge. The first difficulty was identifying schools in which science was being taught through the medium of Gaelic. After consultation with local authorities and the Gaelic community (Bord na Gaidhlig), 44 potentially relevant schools and separate or integrated Gaelic medium units were identified. The schools were located in a total of 11 different authorities: Aberdeen City, Argyll & Bute, East Ayrshire, East Dunbartonshire, Edinburgh City, Eilean Siar, Glasgow City, Highland, Inverclyde, South Lanarkshire and Stirling, with most schools in Highland and Eilean Siar. When the schools were contacted, however, it was discovered either that there were no pupils being taught science through the medium of Gaelic in 2006-07, or that science education was being delivered only partially through the medium of Gaelic. It was decided, therefore, to include in the Gaelic-medium survey all the pupils that could be considered to be studying science partially or wholly through the medium of Gaelic.
Any assessment materials put in front of the 'Gaelic-medium pupils' would need to be in the Gaelic language. Moreover, they would need to be translations of those materials, or of a subset of the materials, that would be used in the main survey, if the assessment results were to have any comparative value. But in the main survey 12 different science knowledge and understanding ( KU) test booklets were administered to the pupils at any one stage (48 booklets in total, across the four stages P3, P5, P7 and S2), in six booklet pairs. Given that the size of the Gaelic population at P5 and P7 was expected to be between 200 and 250 pupils, it seemed inadvisable to administer the full set of different KU booklets to the Gaelic pupils. This is because it was highly likely that the individual task results would be at least as interesting as the level-based test results, and with such small populations six different booklet pairs at a stage would result in just 30-40 pupils per booklet, and therefore per task - far too few to be meaningfully reported in their own right. Moreover, the translation burden that the use of 24 different booklets would imply was considered cost-inefficient, and logistically challenging. It was therefore decided to opt for a single randomly chosen booklet pair for translation from the six used in the main survey at each stage, i.e. four different test booklets in total, two per stage. This does mean, however, that the attainment results produced for the Gaelic-medium pupils through use of this subset of test booklets will not be directly comparable with those produced for Scotland as a whole as recorded in the main survey report 1.
Given that some pupils would have been studying science only partly through the medium of Gaelic, test booklets were sent to the schools in both English and Gaelic. Pupils (or their teachers) were at liberty to choose whichever language they felt most comfortable working with in this testing context, both in terms of the language of the booklet and also in terms of their responses to test questions.
Since the assessment focus was specifically science knowledge and understanding, the Gaelic pupils were not assessed for science literacy or practical skills. It was agreed, though, that teachers would be invited to submit level judgements for the pupils as usual, for mathematics and science, and also for reading and writing, both in English and in Gaelic. In addition, the pupils would, like those in the main survey, be invited to complete a questionnaire about their science learning experiences. Again to limit translation costs and to maximise the number of respondents to each question, just one of the four different questionnaire forms used in the main survey at each stage was translated for administration in the Gaelic study. Again, pupils were given the questionnaire in both languages, so that they could make a language choice.
« Previous | Contents | Next »