The Scottish Government

« Previous | Contents | Next »

Listen

Assessment of Achievement Programme: Report of the Sixth AAP Survey of Science (2003)

7. Summary and Issues

7.1 Pupils' science attainment in 2003

The 2003 AAP survey assessed pupil attainment in terms of the outcomes and strands for Knowledge and understanding as set out in the Environmental Studies 5-14 National Guidelines (2000), and in the three skill strands within the attainment outcome Enquiry skills: investigating in science, those being Preparing for tasks, Carrying out tasks and Reviewing and reporting on tasks. Pupil performance in the core skills of communication (reading and writing), numeracy, and using information technology were also explored. For the first time in an AAP survey an attempt was made to explore pupils' informed attitudes in science. In addition, schools were invited to provide information about resources available for science and their science programmes. Pupils also completed questionnaires about their learning experiences in science.

7.1.1 Knowledge and understanding

The written assessments

Just under 6000 pupils in around 600 schools participated in the written science assessment, that is around 1300-1500 pupils at each stage. In total, 360 Knowledge and understanding tasks were administered to these pupils, 60 per level (A to F) and 120 from each of the three outcomes. The majority of pupils attempted two different test booklets, between them containing 12 tasks from each of two or three levels. On the basis of their assessment results on the 12 tasks at a level, pupils were classified as being 'secure' at the level (using the criterion of 65% or more of the marks achieved on tasks at the same level), or as having shown 'basic' knowledge and understanding at the level (at least 50% of marks achieved, but not as many as 65%), or as having shown 'considerable strengths' at the level (80% or more of the marks achieved). The main findings are as follows:

Attainment overall

• The proportion of pupils demonstrating 'secure' knowledge or 'considerable strengths' at the 'target level' for their stage (Level B at P3, Level C at P5, Level D at P7, Level E at S2) decreased from 54% at P3 through 26% at P5 to 7%-10% at P7/S2.
• Three-quarters of the P3 pupils were classified as being secure or better at Level A, and over half were secure or better at Level B. Similarly, three-quarters of the P5 pupils were secure or better at Level B, and around a quarter were secure or better at Level C.
• The majority (75-80%) of P7/S2 pupils failed to show evidence of 'basic' attainment (more than half marks scored) at their target levels: Levels D and E, respectively. In P3 and P5 the results were more encouraging with only a quarter and a half of the pupils, respectively, achieving fewer than 50% of the marks at Levels B and C.
• Just over a third of the P7 pupils were classified as secure or better at Level C and just under a fifth of the S2 pupils were similarly classified at Level D. Virtually no S2 pupils were secure at Level F, and virtually no P7 pupils were secure at Level E.
• There is no evidence in the survey data of any difference in the Knowledge and understanding attainment of boys and girls at any stage.
• On the basis of the small set of 'common' tasks, i.e. tasks used in the same form and marked in the same way in 1999 and 2003, the survey has produced no evidence of any change in P7 or S2 attainment since 1999.

P3 survey results

The survey results for this group of pupils were encouraging, and can perhaps be explained in part at least by the developments taking place in schools since the revised 5-14 guidelines were published in 2000, and implemented in many schools during the session 2001-2. A high proportion of primary head teachers indicated in the teacher questionnaire that their current science programmes had been introduced or revised since 2000. The P3 pupils tested in the 2003 AAP survey would have been P1 pupils in 2001 and may have benefited from the coherent approach to planning and delivery of the science curriculum taken at that time. Science programmes in schools frequently exhibited thematic groupings of attainment targets, which might extend across one, two or sometimes three adjacent levels. Pupils in P3 were most likely to have engaged in thematic studies involving attainment targets from Levels A and B.

P5 survey results

Results for P5 were also good, although less encouraging than for P3. The performance of pupils might have been affected by a number of factors. These pupils would have been in P3 when the revised 5-14 guidelines for science were introduced, might not have experienced extensive access to the science curriculum prior to that time, and might therefore have had insecure knowledge of Level A on entry to P3. Level C was defined as being attainable in the course of P4 to P6 by most pupils. In P5 it was unlikely that pupils would have covered all the Level C attainment targets tested in the survey. Thematic studies containing Level B and C attainment targets were common, and it was encouraging to see the strong performance at Level B, with 48% showing considerable strengths, 28% secure knowledge, 14% basic knowledge and only 10% achieving fewer than 50% of the marks. The lower figures for Level C of 25% showing basic knowledge, 19% secure knowledge and only 7% considerable strengths, might therefore be partially explained by less than full access to Level C attainment targets in P5.

P7 survey results

The target level for P7 pupils was Level D, described as being attainable by some pupils in P5-P6 or even earlier, but certainly by most in P7. This survey did not attempt to find out if any pupils were achieving Level D in P5.

The results for the P7 pupils were disappointing, with 78% showing less than basic knowledge of the Level D attainment targets. However, 64% of the P7 pupils showed at least basic knowledge at Level C. Thematic studies, introduced since the revised 5-14 guidelines were published in 2000, when these pupils would have been in P5, would have covered attainment targets at Levels C and D. Many Level D (or Level E) attainment targets would not be covered in P7 as local agreements between primary and secondary schools frequently planned for leaving many Level D and E targets until pupils were in S1. It is therefore possible that pupils in the survey might not have had access to many of the attainment targets being tested. This may be one explanation for the relatively high numbers of pupils who did not attempt to answer a significant proportion of the questions.

S2 survey results

The factors contributing to disappointing results for P7 pupils also apply to performance at S2, where performance gave the most cause for concern. The survey results for S2 pupils at Level E will have been subject to several of the factors described in the earlier sections. The target level for S2 was Level E, described in the 5-14 guidelines as attainable by some pupils in P7-S1, but certainly by most in S2. The S2 pupils tested in the survey entered secondary school in 2001, and may have experienced science topics based on Level C and D attainment targets in P6/P7. Secondary schools had revised their programmes of study since 2000, and along with primary schools had been involved in national responses to the report by HM Inspectors of Schools, Improving Science Education 5-14, (1999). However, the period 2000-2002 presented considerable and competing demands for secondary schools, with the introduction of new National Qualifications courses and difficulties associated with national assessment of these courses. Development work on 5-14 Science, particularly at Level F, was given low priority in many schools.

The survey did nevertheless attempt to identify the proportion of pupils attaining Level F, where Level F was defined as attainable in part by some pupils, and completed by a few pupils, in the course of P7-S2. Given that it was expected that a minority of pupils would be introduced to Level F targets during S1/2, it is not surprising that the 2003 survey identified a very small percentage (3% of pupils or 39 individuals) of pupils achieving a basic knowledge at Level F.

The relationship between question type and attainment

The nature of the questions provided at each level seems to have had a direct bearing on the results achieved. At Levels A and B the great majority of questions (97% at Level A and 85% at Level B) were of closed format. The demands in terms of writing were therefore set at the lowest possible level, giving these pupils the best chance of demonstrating what they knew and understood. At each subsequent level there was a higher proportion of open format questions, requiring pupils to convey their knowledge and understanding through written responses.

At Level C the proportion between open and closed formats was 1:2. The short written responses sometimes required pupils to demonstrate not only factual knowledge but also conceptual understanding. In the assessment of writing, also carried out in this survey, only 14% of the P5 pupils were classified as achieving Level C in writing, with all other pupils achieving lower than the target level. Pupils' capacity to produce open-ended written responses was likely therefore to be a factor in the levels achieved for Knowledge and understanding in science.

At Levels D and E the ratios of open to closed format questions were 3:4 and 1:1 respectively. These ratios are comparable to those found in other large-scale surveys, including international surveys. Nevertheless, at these and other levels, a lack of familiarity with tasks requiring a written explanation may have affected pupils' ability to demonstrate their knowledge and understanding. Only 8% of the P7 pupils whose writing was assessed in this survey were classified as achieving Level D in writing, with 25% achieving Level C, 40% Level B and 27% Level A. The relatively high non-response rates for some open-ended Knowledge and understanding questions could therefore be attributed to a combination of lack of access to the Level D and E science curriculum and the requirement to respond in writing to these tasks.

The ratio of open format to closed format questions was 3:1 at Level F. This would have posed considerable additional challenges to many pupils. Writing attainment in S2 was also disappointing with only 23% of pupils achieving Level D or above.

Changes in attainment over time

Although the number of tasks common to the 1999 survey and this survey was small, performance on these tasks does indicate that there has been no change in attainment at P7 or S2 over the period. This perhaps indicates that access to Level D, E and F targets, the nature of the question types and writing skills have all contributed to the results reported. The links between these factors and opinions found in both pupil and teacher questionnaires will be explored in a later section.

7.1.2 Investigating skills

Investigating tasks

Just over half the survey schools at the primary stages and around two-thirds of the survey schools at S2 participated in the practical assessment of pupils' investigation skills in science, that is 80-90 schools at each stage. Nine investigation tasks covering all three investigating strands were newly developed for use in the survey; some of these were used at one stage only while others were used at two stages. The tasks were administered in the schools by 148 field officers, who were practising primary teachers and secondary science teachers released by their authorities for survey participation. In most schools two groups of four pupils were involved, the two groups working on different assigned investigations. Pupil performance was observed and rated by the field officers, using checklists to record level judgments attainment target by attainment target. The general findings are as follows:

Attainment overall

• Across the stages, and averaging over attainment targets and tasks, a quarter to a third of the pupils had to be given assistance by the field officers before they could show enough evidence of relevant skills for them to be judged at one level of the other for one or other of the attainment targets included in the checklists.
• Over all pupils, and over all attainment targets at the same level across all tasks, on 10-25% of occasions no level judgment could be made, because pupils did not show evidence of the behaviour concerned, with or without prompting from the field officer.
• The attainment data provided by those pupils who could be judged showed clear evidence of stage progression in investigation skills.
• Averaging over attainment targets at a level within and across tasks, over half of the P3 pupils were deemed to have shown appropriate evidence of target skills attainment at Level B, with 11% also achieving at Level C.
• On average over pupils and attainment targets, 40% of the P5 pupils achieved the attainment targets at Level C, with 34% achieving Level B.
• Again on average, around one-third of the P7 pupils showed target skills attainment at Level D, with about one-quarter demonstrating attainment at Level C.
• Around one-third of the S2 pupils were rated as having demonstrated the attainment target skills at Level E, the 'target' level for this stage. In addition just under half of the S2 pupils showed evidence of skills attainment at Level D.
• There was no evidence in the attainment data of any gender difference in investigation skills.
• No comment could be offered on change since 1999, since the approach to practical investigation adopted in that previous survey differed considerably from that adopted here.

The effect of the assessment context

The investigation tasks were highly structured and were designed for completion in a relatively short period of time. Pupils carried out the tasks individually. The nature of the tasks and the conditions in which they were undertaken were probably unfamiliar to the pupils. In 1999 HMI reported that practical work in science was not a regular feature of most primary classrooms. By 2003 some progress had been made, and many pupils would have been familiar with whole-class practical activity, decisions being taken after considerable group discussion and with the support of the teacher or written instructions. The different nature of the investigating contexts suitable for a survey may have affected pupils' performance. Although the assessment was carried out as supportively and sensitively as possible, pupils came 'cold' to the problem to be solved, the time to carry out the investigation was necessarily more restricted than in class, the equipment might have been unfamiliar, and they were questioned by the field officer rather than their own (familiar) teacher.

The highly structured context for assessment, with tasks covering only two levels, and the calculation of averages across the attainment targets, might also have masked differences in achievement for each of the three strands, for which the data could not be reported reliably. For example, the skills attainment targets in the C arrying out tasks strand link with those in the 5-14 Mathematics guidelines. The core skills assessment in the survey showed pupils' numeracy skills to be well established. The skills embedded in the Preparing for tasks strand might have been less well developed as pupils may not have been given opportunities to exercise them independently. Many of the skills found in the Reporting and reviewing strand involved writing skills and linked with the skill descriptions found in the 5-14 English language guidelines. The survey showed that skills in the functional writing required for science were not well established and that attainment levels in writing were consistently below target levels for each stage.

However, despite the survey's constraints, the pupils provided evidence of progression from P3 to S2 when the results were averaged over targets and tasks. Pupils in P3 again performed better relatively speaking than pupils at other stages, with 53% demonstrating achievement of the attainment targets for investigating at Level B.

7.2 Core skills attainment

In addition to problem solving, which featured in the survey as a full exploration of pupils' attainment in the outcome Skills in science: investigating, the core skills of reading, writing, numeracy and using ICT were assessed in this 2003 science survey. Reading, writing and numeracy were assessed using pencil and paper tasks, whilst ICT skills were assessed practically, with pupils working on laptop computers supplied by the trained field officers. All the tasks used were carefully set in 'scientific' contexts.

Pupils' reading attainment was assessed using relatively extensive reading tasks, each comprising a stimulus text followed by sections of related questions, most focusing on information retrieval skills. Each task was classified at one particular 5-14 level (Level A to E), featured a 'scientific' topic, but did not require prior science knowledge or understanding. The main findings are as follows:

• Reading attainment was good at all stages, particularly at P3 and P5.
• Over 80% of the P3 pupils were 'secure' at Level A and over half were secure at Level B.
• Around 70% of the P5 pupils were secure at Level B and around 40% were secure at Level C.
• Roughly 60% of the pupils at P7 and at S2 were secure at Levels C and D, respectively, and around 40% were secure at Levels D and E, respectively.
• The results at P7 and S2 were in line with those reported for reading attainment in the 2001 English Language survey.
• In contrast with expectations, there were no apparent gender differences in reading in this science survey.

The results for reading suggest that for pupils in P3 and P5, where reading skills were particularly good, reading was not acting as an additional barrier to performance in Knowledge and understanding or Investigating skills assessment tasks. The same would be true for pupils in P7 and S2 where the results are in line with those reported in the 2001 English Language survey.

7.2.2 Writing

For the assessment of writing skills, the reading tasks were used as a stimulus for the writing, and to this end a writing activity was included in each Level C, D and E reading booklet. The performance of pupils on these tasks was disappointing, in terms particularly of the volume of writing that they produced, often none at all (20% of pupils). Pupils may have run out of time before starting their writing task, or may not have been motivated to start on this final task at the end of the booklet after having answered questions about the text, or were simply unable to say much about the topic in question. That said, the main findings were as follows:

• The pupils demonstrated relatively poor writing skills in the science context at all stages.
• Fewer than 15% of the P5 pupils produced writing deemed to be at Level C or higher.
• Fewer than 10% of the P7 and S2 pupils produced writing that was rated as being at the target levels for these stages (Levels D and E, respectively).
• For most of the pupils involved in the reading and writing assessment, writing attainment was lower than reading attainment; just 25% of the pupils assessed in reading and writing were classified at the same level in both.
• While proportionally more of the boys than of the girls failed to produce any writing in the survey (20% versus 14%), among those pupils that did produce writing there were no gender differences in quality.

Writing and overall attainment in science

The results for writing as a core skill suggest that writing ability might negatively have affected the results for Knowledge and understanding and the Reporting and reviewing strand for Skills in Science: investigating at P7 and S2 in particular. Pupils showed a marked reluctance to express their ideas and understanding in writing. At P3 the assessment tasks for Knowledge and understanding were mainly closed format, and writing was only rarely required when responding to tasks. Higher proportions of open ended tasks were administered to pupils at P5, P7 and S2, with correspondingly higher demands on pupils to convey their knowledge and express their understanding through written responses, sometimes quite extended written responses at S2. The incidence of non-response to tasks increased at each stage, in line with the increase in open formats.

The pupil questionnaire eliciting information about learning circumstances and pupils' views about their learning yielded some points related to writing. Pupils reported that in science whole class teaching predominated and the most frequent individual activities were writing in a jotter or file and completing worksheets. Those teachers who completed questionnaires corroborated these views and stated that when they were working on their own, pupils in S2 spent most time writing in jotters or files. Science lessons were also reported by pupils as being interactive, with a high level of questioning, answering and explaining to the class. This description matches the advice about direct teaching given in the HMI report Achieving Success in S1/2 and one of the key recommendations of the Improving Science Education 5-14 report, "…improve the effectiveness of teaching and learning, including making more effective use of direct teaching, questioning and discussion, practical work and assessment."

However, the time spent by pupils on writing in files and jotters did not appear to be contributing effectively to pupils' ability to respond to assessment tasks which required more extended responses and their lack of writing skills was apparently impeding their ability to demonstrate knowledge and understanding. Many of the science tasks required pupils to demonstrate conceptual understanding and not simple recall of facts. The nature of the interaction and writing undertaken in science classes might therefore prove to have been more directed to setting down factual knowledge, reporting on work done or completion of worksheets, rather than to the expression of understanding of concepts.

7.2.3 Numeracy

The numeracy tasks administered to pupils in this survey were derived from those used in the previous social subjects survey, and were used in the science survey in identical form or with simple changes in wording to render the contexts more 'scientific'. The main findings were as follows:

• On average, more than half the pupils at every stage responded successfully to numeracy items.
• A comparison across surveys revealed slightly better performances across all stages and tasks in the 2002 social subjects survey than in the science survey, the lower achievements in the science survey reflecting a generally higher non-response rate for all tasks in the science test booklets (in which the numeracy tasks were embedded).
• There were no gender differences in numeracy attainment.

It would appear that pupils were generally able to transfer numeracy skills successfully into scientific contexts. P5 and P7 pupils demonstrated good numeracy skills at Levels B and C, respectively, and these may have contributed to the Carrying out tasks strand of Skills in science: investigating. There is certainly no evidence to suggest that numeracy skills were any barrier to the demonstration of investigating skills in science in the way that writing appeared to be a barrier for the demonstration of knowledge and understanding.

7.2.4 Using ICT

Practical ICT assessment produced evidence of well-established skills in some areas and of clear progression elsewhere from one stage to another:

• Over 90% of the pupils at all stages, even P3, were skilled at opening documents and navigating through web pages.
• Around a third of the P3 pupils were also able unaided to add information to existing spreadsheet tables, compared with almost 60-70% of the pupils at other stages. At all stages lower proportions of pupils were able to centre information within columns.
• At P5, P7 and S2, where these skills were tested, rather low proportions of pupils managed within their spreadsheets to produce graphs to illustrate their tables of data, and none, or almost none, knew how to use the formula facility in Excel to carry out simple calculations on their data.
• Around a third of the P3 pupils were at ease simulating web searches while a quarter were able unaided to prepare to send an email. At the other stages the proportions were higher, ranging from 50% to 80%, depending on the activity and the stage.
• As was the case for the other core skills, there were no gender differences evident in the ICT attainment data.

Again, there was evidence of pupils being able to transfer skills developed in another curriculum area into science contexts. The pupil questionnaire revealed that high proportions of pupils had access to a computer and the internet at home. Primary head teachers gave positive responses for resource quality in relation to computer and internet access for teachers and pupils. Nevertheless, head teachers' responses suggested that using a computer in class for science was not common, particularly at S2. Working with a computer, alone or with a partner, was not a regular part of science learning according to pupils' responses. Computer use in science classes also declined with age. These findings may be contributing to the pattern of skills found in this survey. Pupils' performance was lower in the tasks associated with handling data and plotting graphs than in the other tasks undertaken. This could well be associated with unfamiliarity with the tasks for the reasons indicated above or pupils having difficulty transferring into science contexts skills acquired in ICT classes.

7.3 Informed attitudes

Over 1200 pupils at P5, P7 and S2 completed questionnaires inviting information about their science lessons, and exploring their familiarity and views about topical issues in science and technology. The enquiry revealed the following:

• There was a high degree of awareness of current issues among pupils, particularly at S2.
• Among twelve targeted issues, the internet, cancer research and weapons of mass destruction were those that most interested the pupils, while designer babies, cosmetic surgery and genetically modified foods held little interest at all for them.
• There were no gender differences in awareness, but quite strong and expected gender differences in topic interests: the boys were more interested than the girls in nuclear power, space exploration, robotics and weapons of mass destruction, while the girls' preferred issues were cancer research, designer babies and cosmetic surgery.
• When asked to offer an opinion on whether the issue concerned was 'good for society' or 'bad for society', the internet, cancer research and organ donation emerged as those most positively viewed in this sense, while weapons of mass destruction and global warming were the most negatively viewed. The majority of pupils declared themselves unsure of the value to society of designer babies, cosmetic surgery and genetically modified foods.
• Pupils' reasons for their opinions were generally valid and relevant, showing evidence of informed attitudes.

In an innovative complementary enquiry, over 2500 pupils participated in focus group discussions about given scientific issues. Almost 650 focus groups were observed and rated for informed attitudes by the field officers. The salient findings were that:

• A quarter of the groups, rising to a third at S2, were reluctant to engage in discussion and were difficult to animate and hence to assess.
• Pupil groups were generally more highly rated for showing evidence of Respect and care for self and others than for Commitment to learning and Social and environmental responsibility.
• The least well demonstrated attributes, within Commitment to learning, were Recognising the need to find out more facts and/or ask questions about the issue and Asking about evidence.

Of interest in these two aspects of the survey is the willingness of pupils to indicate in the questionnaire their awareness of and interest in a number of scientific issues by not only responding to closed format questions, but also offering reasons for their views in writing. Space was provided for the pupils to express an opinion, and these opinions were relevant and valid. It is interesting that many pupils seemed willing to write a phrase or sentence to express an opinion on a topical issue, whereas they were often unwilling to write similar amounts to demonstrate their knowledge and understanding in the science tests.

Responses to the focus group discussions were mixed, with many of the older pupils showing their reluctance to engage such discussions. This may or not be related to interest and motivation for science. Primary teachers reported that pupils were highly motivated towards learning in science but secondary principal teachers did not feel that pupils were so motivated to learn science. The reluctance to engage in discussion was most apparent with S2 pupils.

7.4 The pupil and teacher questionnaires

Pupils were asked about themselves and their families, and then asked about their experiences in science lessons and their perceptions about their learning experiences. Teachers were asked to report on a range of issues relating to science teaching in their schools and classes.

In this summary the results of pupil and teacher surveys are combined and discussed in relation to the findings of attainment in Knowledge and understanding and in Skills in science: investigating, where the majority of pupils are not meeting the target 5-14 attainment levels. Issues relating to transition and comparisons between pupil experiences in primary and secondary schools can also be made if the results of these questionnaires are combined.

The home context

Home resources for learning were impressive, with high proportions of pupils having access to a quiet place to study and facilities such as TV/video, computer with internet access, calculator and dictionary. For the majority of pupils their learning in school was clearly well supported by the resources provided at home. The information about family activities does not provide many insights which can be related to performance in science in school. However, talking about schoolwork and progress occurred on most days according to over half the pupils. Parental support for children's learning was significantly more highly rated by primary head teachers compared with S2 principal teachers.

The classroom context

Pupils were asked about activities within science lessons and the picture which emerges is one where they were most frequently taught as a class, they wrote in jotters or files and completed worksheets in most lessons. They worked quietly alone more often than they worked in pairs or groups, but they did talk to others about the topic in most lessons. They were not taught in a small group very often and neither did they have the opportunity to talk one-to-one with a teacher very frequently. The teachers' reports about learning activities were similar to those provided by the pupils. The questions about homework in science focused on the regularity with which it was undertaken, the time spent on it and if a computer was used. The survey did not explore the nature of the tasks undertaken by pupils at home to support their science learning, or their views about these tasks. It is therefore not possible to comment on the effectiveness of homework in relation to learning in science.

Learning and teaching in science

The overall picture of science lessons emerging from the pupils' questionnaire section relating to the nature of teaching and learning is not entirely consistent with that described above for science activities; in many ways a more positive situation is revealed. Pupils were expected to work hard and encouraged to say what they thought and were given help when they needed it. Nearly 80% of the pupils stated that they found it easy to concentrate and work hard in class. Pupils saw science as fact-based but also appreciated that learning in science is about asking 'Why?' and 'What if?'. P5 pupils particularly reported that science is about asking and seeking answers to such questions. Pupils reported that they were expected to explain their answers to questions, with 85% saying that this is either 'always' or 'mostly'. Higher proportions of pupils in P5 reported these aspects of science lessons than pupils in P7 or S2. These findings are particularly interesting given the results for Knowledge and understanding and pupils' reluctance to provide extended responses to questions probing conceptual understanding. P5 pupils were not required to answer as many extended response questions as pupils in P7 or S2. Perhaps this is providing more evidence for writing being the barrier to demonstration of knowledge and understanding. Pupils appear to have been thinking and talking about science ideas and explaining their understanding in their classes. They also appear to have been undertaking a great deal of writing, but perhaps this writing was not about ideas and personal understandings and explanations, i.e. the kind of writing demanded by the extended response questions in the survey.

The motivation to learn science

The findings about pupils' motivation to learn science are interesting, as there seems to be a trend showing decreasing motivation with age. Pupils were well-motivated to learn in science in P5, as shown by the higher proportions responding positively to questions about wanting to do well and working hard in science, being given opportunities to discuss interesting topics and to do investigating. Pupils' perceptions about the importance of science for later learning in other subjects and for jobs were also age related, with importance declining with age. Far more P5 pupils looked forward to science than did pupils in P7 or S2.

Principal teachers and S2 subject teachers gave significantly less positive ratings to their pupils' motivation to learn than did primary teachers. Primary head teachers also reported high ratings for pupil attendance at and behaviour in science classes than did secondary principal teachers.

Assessment in Science

Pupils' responses to questions about assessment and feedback are also revealing and again a trend in relation to age emerges. Significantly more pupils in P5 reported that they were shown how to improve their work and that their corrected work showed them where they had gone wrong; less of their assessment was done in short tests and the information they got about progress was not given as levels, grades or test scores. In contrast, S2 pupils reported that they got information about progress mainly in the form of levels, grades or test scores and that most of the assessment was done in short tests. They did report that their corrected work showed them where they went wrong and that they were shown how to improve. Principal teachers mentioned giving 'marks or grades for effort' but only one primary head teacher did so. This is possibly because effort would be discussed with pupils in the normal personal feedback they would receive instead of marks, grades or scores. These findings have resonance with other studies that show a relationship between the nature of feedback and pupil motivation; where the giving of marks is less effective than personal feedback against shared objectives.

Science programme and resources

A number of issues relating to science teaching were explored only in the teacher questionnaire, answered by primary head teachers, secondary principal teachers, primary class teachers and secondary subject teachers.

The time scale in which schools have responded to the revised guidelines published in 2000 differs between the two sectors. A very high proportion of primary head teachers reported that their science programmes had been introduced or revised since 1998, with the majority having done so since 2000. Revision of the science programme would appear to be ongoing in the majority of primary schools. In contrast, fewer secondary principal teachers reported that their science programmes had been revised since 2000 and just less than 50% reported that their programme was being revised in 2003.

The results for P3 and P5 pupils in this survey are more encouraging than those for P7 and S2, and perhaps indicate the progressive nature of developments in science programmes since the revised guidelines were published in 2000, and might also be used to predict tentatively that further progress might be detectable by 2007 when the next Science Survey will be conducted. By this time the P3 pupils of 2003 will be in P7, and the P5 pupils of 2003 will be in S2, and all schools should be implementing programmes based more firmly on the 2000 guidelines.

Revision of science programmes since 2000 has been accompanied by the introduction of a wide range of resources developed for 5-14 Science. In the 2003 survey the resources used most commonly in secondary schools were materials produced by the schools themselves. The national 5-14 guidelines were being used by most teachers, with around half also having access to additional guides produced by their local authority. Commercial textbooks and resource packs were frequently used in both primary and secondary schools. Around a quarter of teachers in both sectors produced their own materials for classroom use. In 2003 materials produced by teachers' groups or associations, other schools or other authorities were not much used. In 2003 teachers did not have access to many resources designed specifically for 5-14 Science.

Primary teachers had more concerns about resource provision than did S2 teachers, and also felt less confident and less well prepared to teach all the science outcomes. However, more primary teachers felt well prepared for developing informed attitudes than did secondary subject teachers. S2 subject teachers felt themselves to be very well prepared to teach all the science outcomes and investigating skills. As these are specialist teachers with science degrees this finding is not unexpected. However, the teacher knowledge and confidence in each sector does not correspond to the results found in the survey for their pupils' knowledge and understanding, investigating skills, core skills and significantly their motivation for science. Perhaps this relates to the extent to which programmes of study and methodology have been revised since 2000, with primary schools having engaged in this process ahead of the majority of secondary schools.

Since 2003 a greater variety of resources have been produced locally and nationally to support the delivery of 5-14 Science. Most of these resources promote effective learning and teaching and are based on current research on encouraging critical thinking skills, eliciting children's ideas, stimulating investigating, learning styles, modalities and intelligences. The national project Assessment is for Learning has involved large numbers of teachers and has promoted the benefits of formative assessment strategies on pupil learning and achievement. Resources and case studies from this project are also now available to support teachers. Efforts have been made, both nationally and in local authorities, to provide CPD opportunities for both primary and secondary teachers. These CPD events and resources have aimed to improve the knowledge and confidence of primary teachers, and to give secondary teachers insights into the experiences that pupils are likely to have had in the primary school and the learning methods they are most likely to be familiar with. The 2003 AAP survey in science did not elicit information about transition arrangements and the extent to which schools achieve continuity, coherence and progression for pupils, but the survey findings suggest that this is an area which will become increasingly important.

7.5 Key issues and conclusions

The findings from the survey raised a number of key issues relevant to future developments in learning and teaching science in the primary and lower secondary school:

• attainment in knowledge and understanding
• attainment in writing
• pupil interest in and motivation for science
• learning and teaching strategies
• assessment and feedback to pupils.

This survey provides evidence, from a study of a limited number of common questions, that there has been little change in attainment in Knowledge and understanding since the 1999 survey. Attainment in writing is disappointing at all levels and stages, and pupils' inability to express their ideas and explanations in writing may have hindered their ability to demonstrate knowledge and understanding in science. Learning and teaching strategies are varied but are not supporting pupils in the development of the kinds of writing skills needed to demonstrate conceptual understanding. Pupil motivation for science is strong in the primary school but diminishes as they reach P7 and continues to decline in S2. The nature of assessment and feedback changes as pupils move from primary to secondary school and this may be having an effect on motivation.

The concerns expressed by teachers in 2003 about the availability of resources, and confidence and preparation for delivering the 5-14 science curriculum, have been addressed nationally and locally by the funding of resource development and CPD opportunities.

This is the first AAP science survey to report since the revised 5-14 guidelines were published in 2000 and the first to report pupils' achievements in Knowledge and understanding with reference to 5-14 levels. Pupils as young as P3 were involved in this survey for the first time. In some ways the findings reported here should be regarded as a benchmark against which future cohorts might be measured. These benchmarks may also be used to make judgements about the effectiveness of those national projects and local initiatives included in the Improving Science Education 5-14 programme which has received central funding in recent years.

« Previous | Contents | Next »