Scottish Government

Evaluation of Project 1 of the Assessment is for Learning Development Programme:
Support for Professional Practice in Formative Assessment
Final Report

10. Phase 2 of the evaluation

10.1 Scope of Phase 2 of the evaluation

In phase 2 of the evaluation visits were made to a further sample of schools including an independent school and a special school. The activities undertaken in these schools were similar to those visited in Phase 1 of the evaluation and the findings reinforced those from Phase 1. The key areas of interest in Phase 2 of the evaluation were the way that dissemination had been undertaken and the extent to which the intervention had continued to be implemented in the longer term.

10.2 Primary schools

10.2.1 Implementation of the project

In the primary schools visited, as in Phase 1 of the evaluation, the project was implemented by two teachers with strong support from the head teacher. A range of strategies were adopted including think time, talk time, higher order questioning, target setting, peer assessment, self-assessment and giving children more responsibility for making choices about their learning. Peer observation of teaching was the starting point of the process and led to teachers elaborating on their plans.

In one school the project had broadened out from its start in mathematics to include writing, reading, listening, and talking. The key activity was a programme of systematic interviews in which pupils talked about their learning and any difficulties they were experiencing. The funding had been used for supply cover so that interviews could be scheduled in school time. Since the project had formally ended the work had been adapted with termly interviews held the outcomes of which were shared with parents. Another development was the use that the school was making of PICASSO software, systematically laying out the curriculum by levels and enabling the pupils to use it to discuss strengths and difficulties in relation to the next step in the curriculum. This was increasing their engagement with learning.

Teachers made similar comments to those in phase 1 of the evaluation. Initially, they felt that there was a lack of clarity about what was expected of the project schools by the LTS Project Team. They were concerned that HMI might not be fully aware of the work of the project. They reported that the project had an adverse impact on the amount of time that they spent in the classroom because of the time they spent elsewhere for planning and discussing the project. They were also concerned about the amount of time it took to teach a particular topic. For instance, a lesson that had previously taken twenty minutes now took an hour. This created tensions between the need to cover the curriculum for summative assessment and the focus in the project on formative assessment.

On the positive side, teachers and head teachers perceived that what was being implemented had been seen as good practice in the past. They were enthusiastic that this had been revived and brought up to date. The formative assessment practices introduced were permeating to all other areas of their teaching. Another positive aspect of the implementation related to record keeping. Schools became more aware of the need to record evidence. Overall, the project was seen as offering professional development opportunities for everyone.

Box 25

10.2.2 Impact on teachers

In the primary schools teachers had enjoyed and benefited from participation in the project.

'I love think-time; I think it takes the pressure off some children. That's why I think think-time is entirely beneficial. I like talk time. Again, when I started talk-time with my primary 2s they weren't sure about it. And now they can very sensibly whisper to each other, make plans, share a mental maths answer. I find it working very well- I got to know the children as learners much more quickly than I would have done otherwise.' (Primary 2 teacher)

Some of the teachers reported that at this stage in the project with no funding, while they had continued to apply the lessons learnt and the techniques developed 'keeping up the momentum was hard'. Some teachers reported that having a new class in the new academic year had not allowed them to continue development of the project with the same pupils. They had to develop relationships with the new pupils. Initially, the teachers indicated that the school's participation in the project was key in giving them the time, resources, motivation and interest to develop the work on formative assessment. This seemed to have tailed off as a result of 'no longer being in the project, as such'. However, they recalled the impact of the project on their previous year's pupils and indicated that they were more conscious of the applications of learning to what went on outside of school and that this had continued.

One teacher reported that:

It gave me a boost in terms of enthusiasm. And I have to say career-wise potentially it was good; I've got a job application in at the moment so we'll see what happens. (Primary School Teacher)

Overall, there had been an impact on pedagogy and teachers had enjoyed their engagement with the project. Whether this would continue in the long term in these schools is debateable.

10.2.3 Support of the head teacher

In all of the schools visited the head teacher had a very clear and detailed idea of developments during the project and the current post-project phase and had supported the staff. The head teachers were enthusiastic and felt that the impact had been substantial. In some cases this had led to a review of assessment practices.

'I think it has helped us as schools to accept and know we're right in what we always believed i.e. that assessment doesn't have to be marks. I think we are gradually using individual teachers more as role models who've been through longer experiences of learning, where they'll go and share the classrooms and they team-teach. And they observe each other.' (Head Teacher)

10.2.4 The impact on pupils

As in phase 1 of the evaluation the impact on pupils was substantial. Pupils were reported to have raised self-esteem and to feel grown up about taking their own decisions about their learning:

'Children in both of my classes now can say, "I'm good at that, I'm not so sure about that." And they very frequently had or do now have the opportunity to say, "I'm not so sure." And I think that's very valuable for them.' (Teacher)

One teacher described how teaching using open questions, getting pupils to think, question and justify their answers had been really successful. She described how:

'Some of the less-able pupils really took off, and they were the ones that seemed to be- they were slow at first and then all of a sudden they just seemed to run with it. And they seemed to be able to learn much more clearly, because they were doing and they were discussing it and thinking; they were learning from others. And the others were- the more brighter- the brighter people, the more able, were actually listening to the less able and saying, "Oh, that's a good idea."' (Teacher)

Confidence also improved:

'The less able suddenly realised that they weren't having as many problems as they thought they were. They were suddenly able to come up with the answers that really were stunning that maybe they hadn't thought they were able to do. So it gave them confidence.' (Teacher)

There was much greater engagement in learning:

'I think in the past where you've said, "Right, read this, make a note on it, let's do this experiment." And I would wander round. Now they're trying to plan how they're doing the experiment, and they're having much more fun. So, of course, the teacher enjoys it too.' (Teacher)

The reluctant learners were more engaged as were those who had previously found learning difficult. Not only did implementing the strategies improve motivation, it also improved attainment. In one school test scores improved overall by 30 % in the trial class.

Pupils themselves reported that sharing ideas was beneficial:

'We talk to partners. And it really helps because you hear your own ideas, and also hear your friend's idea at the same time, so then you mix them together. What happens is that you talk to your partner, write it down, try to find a couple of ideas, then it's a big class discussion. And she goes around asking what you think. But it's really good.' (Primary Pupil)

'Two minds are better than one, and you come to a more probable answer. And one person could think this and the other person could think that, and together they could make a right answer.' (Primary Pupil)

'What she, sort of, does is she, sort of, asks us to think about the question; she'll, sort of, give us hints while we're thinking.' (Primary Pupil)

'Well, I think that practical work followed up by a class discussion works a lot better, because you get to do the fun stuff, which makes you learn because you like it.' (Primary Pupil)

In one school the impact on one particular pupil was noted by staff. He was notorious for his lack of participation in lessons and the difficulties over the years to get him to participate fully. The project's emphasis on lengthening wait times led to this pupil's increasing participation and 'finally' to his 'inclusion' in the learning process. At the same time the teachers involved also reported the quality of the pupil's writing had improved. This very visible success was replicated at the level of the whole of group (P7). The teachers reported that 'relationships with the children blossomed … trust developed. It became possible to bring forward difficulties without any negative impact.'

10.2.5 Impact on the whole school

Teachers were reported as communicating more than in the past about their teaching and other professional matters related to the project. The involvement of colleagues in the project meant that all staff in the school felt that formative assessment was important. Overall, the project had a positive impact for teachers and students, those involved in the project and across the whole school. Relations improved between individual staff. In all the primary schools there was a noticeable improvement in the attitude to teachers and to learning by some pupils who had previously had a less good relationship; as well as an overall more positive attitude by individuals and the pupil group as a whole.

10.2.6 Parental Involvement

As in the first phase of the evaluation, parents generally had not had any input into the project, although head teachers indicated that they had plans to explain the project to parents as part of the dissemination process. One school had introduced the project at a meeting for parents of children new to the school:

' At that meeting we said, "We're carrying out research with the children, if you have any great concerns of course contact us." And a few parents asked a few questions, but very little in actual fact. They accepted what we're doing, which was lovely. Again, I spoke with parents during consultation times and they were comfortable. They noticed certain things, some commented on awareness of technical skills within writing, others commented on maths development and the children's ability to justify their answer.' (Primary 2 teacher)

10.2.7 Difficulties experienced

The difficulties reported in the Phase 1 of the evaluation were re-iterated in Phase 2: the time taken to teach particular topics using formative assessment procedures; and the need to meet the demands of summative assessment. Ultimately the time invested in helping the pupils to learn to learn was perceived as productive. As one teacher reported:

' That's the difference between summative and formative. The two don't mesh easily. And they were being tested summatively and I was teaching them formative assessment, so they fell down in the first tests. But at the end they did so much better than any of the other pupils. It shows that they then had the maturity and the understanding and the ability to think. And so they were able to do the tests.' (Primary 7 Science Teacher)

10.2.8 Strengths of the project

The strength of the project was the way that teachers refocused their attention on 'teaching' and 'learning' and on the need to empathise with the children in their learning role.

10.2.9 Support offered for the project

The participating teachers would have liked more access to the experiences of other teachers involved in the projects. They wanted to be able to understand and 'make more sense of the big picture in Scotland as a whole.' They acknowledged that this information was probably on the LTS website but they had experienced some difficulty in accessing the web-site and had not used it to satisfy these goals.

Teachers praised the support given to their work by the recall meetings and described how it gave them the confidence to continue when they were experiencing difficulties:

'The first recall day I think was about five weeks into my project, and it was perfect timing, because that was the point where I was feeling a bit down and a bit concerned about the whole thing. And I went along and there was someone speaking who'd already completed the project. They stood there and said they had had the experience, or they'd seen in the past, that people didn't perform very well, and they became very disillusioned, the staff became disillusioned, at about five/six weeks, and then things started to happen. And it actually gave me a bit of a boost. And I talked to the person afterwards, and I thanked him, and said, "That's helped because I was ready to pack it in." And they just said, "No, no, don't pack it in, just keep going, you'll find out it does work." And sure enough it really started to take off after that. So I needed- I needed that first recall badly.' (Primary 7 Science Teacher)

Teachers indicated that they would like some feedback on their reports.

10.2.10 Sustainability

The teachers who had been involved in the project had continued to retain their enthusiasm and commitment to formative assessment and continued to apply the principles and practices developed during the funded phase but in a less intensive way than the previous year. In some schools there were clear concerns that the way they had developed the project could not be sustained without some funding to provide cover for teachers to undertaken the work with individual children.

10.2.11 Dissemination

At primary level, dissemination had taken place within the school both formally through an INSET day and informally by teachers observing each other and discussing their strategies in their departments.

Participating teachers were keen to share their methods nationally by visiting other schools but there was the problem of providing cover for their own teaching. An alternative to visiting other schools which was suggested was to provide video examples of the techniques which could be used throughout Scotland.

'I don't know whether the answer is video lessons, let people come in and watch teaching just as you've come in. If people want to come in I know I'm more than happy to have anyone come into my class. I think that is probably the way forward, that if people see how you teach and see the effect it has on the children, they're more likely to use it themselves.' (Primary 7 Teacher)

Head teachers and staff involved in the project had taken part in school wide dissemination programmes, dissemination activities in clusters of local primary schools, in authority-based INSET and in Learning and Teaching in Scotland (LTS) driven dissemination. Staff in one school had been involved in an LTS dissemination of several projects to a national conference. They had felt a little nervous and threatened by the experience but enjoyed and benefited from getting to know about another project related to their work. In one Education Authority a secondee had been appointed to act as the Development Officer for formative assessment in the Authority. Within the authority there were new 'project schools' working with the Development Officer. One head teacher had given input on the progress of the project and the future plans for it, to a "cluster group" meeting of ten local heads. She believed that the key to successful dissemination was a partnership between a regional/local adviser offering INSET to teachers and practitioners involved in the project and their experiences.

In one school there had been school wide dissemination with all staff having two INSET sessions led by the teachers who had worked on the project the previous year. The teachers aimed to offer colleagues a taster of all the techniques of formative assessment used in the school during the project. There followed a period when all the staff tried for themselves an aspect of the project. In the light of these 'tasters' the staff decided on a whole-school implementation of the project in all classes. It was decided to focus on language and in particular on the process of giving feedback on written work. The whole school staff were supporting each other in this development formally and informally. They felt that clear guidelines given at the outset of the activity would prove to be one of the most helpful aspects of the whole dissemination.

10.3 Secondary schools

10.3.1 Implementation of the project

In the secondary and junior high schools visited, as in the first phase of the evaluation, two teachers in each school implemented the project. In one school the specific teacher strategies explored related to the use of questioning techniques to encourage interaction and peer assessment during class discussion. The school was also interested in investigating the most effective forms and use of formative assessment. In another school the two foci were: how language could be a barrier to pupils expressing their knowledge and understanding; and developing checklists for pupils which encouraged self-assessment, pupil independence and an analytical approach to study. The project was implemented in English and aimed to get the pupils 'to achieve more independence in their work and give them a sense of ownership. Checklists of criteria were used with pupils so that when they produced their first draft of written work they could check off each of the criteria before they handed it in. Other strategies employed included longer wait times, no hands up and traffic lighting for providing feedback to pupils.

10.3.2 Impact on teachers

There was a mixed response from teachers. Those participating in the project demonstrated a positive and open attitude, with a willingness to embrace new ideas. This was judged as an important contribution to the success of the project in the school. Most teachers who had been involved in the project were very committed to the principles of formative assessment and to the work that they had done, indicating that they had become 'reflective practitioners' as a result of the project and that they would continue to put into practice the techniques and lessons learnt. They had engaged in a careful self-examination of their approaches to teaching and learning and cross-curricular insights had been heightened. One head teacher indicated that the project had engaged the attention of those who were interested in 'new' developments but that the message was diluted as it progressed to those who had not had first hand involvement. However, it was effective in allowing teachers to generate change 'bottom up.'

Some teachers reported that they felt that they no longer needed to control the class as rigidly as before and in return, the students took a lot of the responsibility for their learning. Students had been given a greater degree of autonomy and had responded very positively to this. Pupils were more engaged in the learning process and appreciative that their teachers were listening to and thinking about their ideas and opinions.

The two key issues raised by the teachers in junior high and secondary schools were the relationship between the formative assessment project and the summative assessment built into the system and the amount of time taken to implement the strategies leading to difficulties in covering the curriculum.

'The actual time which was given over, say, to primary P4-7 composite to work on analysis of writing skills was such that to apply similar approaches over many curriculum areas would be impossible. However, if principles are learned and internalised by pupils the 'training' time can be reduced.' (Head teacher)

There was a general feeling of disappointment that the policy changes that they felt were being signalled by the Assessment is for Learning project did not appear to have been followed through in subsequent government policies. The impasse on Personal Learning Plans and the continuation of National Tests were cited as evidence for this. This signalled a less clear vision than they felt was being offered at the outset of the project.

10.3.3. Impact on pupils

Teachers reported a better and more positive attitude to work from the students. They also indicated that the project had demonstrated that formative assessment could be used as a tool to raise the expectations, self-esteem, and attainment of students. Students in one school from both Y7 and Y10 were very clear and consistent about what constituted good teaching: participative methods, with a lot discussion, problem based, with limited teacher input and strictly no worksheets. They were able to define accurately, self-assessment, peer assessment and teacher assessment. They were clear that they preferred teacher led assessment with some limited written feedback but with as much oral feedback as possible from the teacher. Overall, the project was judged by participating teachers as having been well received and appreciated by students. One teacher described how useful he found the strategies especially the use of checklists:

' So before pupils give in a piece of work they had to tick the checklist to say that they'd got at least two paragraphs, used capitals letters, checked spellings etc., etc. ….. I think in terms of the kids' actual work, the work that they're producing, it's- they're actually- a lot of them are producing very good work at the first draft stage, which has never happened before. And the result of that is that it actually cuts down on so much time in the class just on having to redraft, because obviously we have to redraft a neat copy.' (English Teacher)

He also explained how this led to pupils target setting for themselves as they checked their work and saw what they needed to do in the future. As the check list followed the piece of work from first to final draft a huge amount of administrative time was saved which made the teachers much more efficient assessors. The pupils were reported to have really improved their level of work as a consequence of using the checklists:

I mean, there was a couple, two of them, that came in at level C from primary school, and it was really clear to me from using it that they were much better than that. They were, kind of, quietly just, kind of, moving along and not really thinking about it. You know, they almost jumped two levels in a year. (English Teacher)

Some pupils made no progress at all as a result of the checklists but the majority had made some progress in either reading or writing. Pupils had also become much more aware of the assessment criteria and many had become more positive in their attitudes towards about their work:

'It is more positive. They don't moan, as much because, you know, they don't necessarily need me to reassure them about whatever they're doing. They are much clearer about what is required.' (English Teacher)

The pupils were reluctant to assess each other's work, especially the lower attainers, because they did not like other people looking at their work. The process was more successful with the able pupils. The pupils understood how the checklists helped them to produce better work by a process of self-assessment and through making the criteria clear. One pupil mentioned that she liked the section on the checklist for her to put a comment on her work. Another indicated that it made him a more independent learner. A range of other comments were made:

'It helps you do your work quicker because instead of everyone waiting to see the teacher, you can just look at the sheet and it helps you with what you've got to put in your essay.' (P5 pupil)

'When you're doing your essay or something, when you're finished, you just check over, and if you've missed something you can go back and put it in. And it makes it better.' (P5 pupil)

One pupil liked the way the sheets related to the different levels of attainment -

'Sort of it improves your grade because you get different sheets depending on what grade you're at. So if you're at a level and you get a checklist for As, then you can include other things and then you can get a better level. (P6 pupil)

Another pupil described how the practice of using a checklist had spilled over

into other lessons:

' Well, it doesn't just help your English, it helps you in all classes. It makes you think about the punctuation; and it's quite annoying when you need to keep going back and changing it so if you do it right then its great.' (P6 pupil)

Overall, pupils had a good grasp of the usefulness of the formative assessment strategies and how these had contributed to them becoming more confident, independent learners, which was one of the aims in the action plan. Their teacher reported that if a piece of work was given out without a checklist the pupils complained.

The Head of Department indicated:

'I would say it is encouraging them to take more responsibility and have a better sense of ownership of their work. , I hate it when kids say, "Mark that," and they haven't checked it. This system does encourage them to reflect and to take that wee bit of time. And if it's only just a bit of proof reading and editing it's still more than someone, "Oh, I've got the work finished, I must - it's no longer my responsibility." So I do think it has advantages as far as that's concerned. And I would say my impression is that it has encouraged the kids to be more engaged in their own work, to take, as I say, take more sense of pride in it. More ownership in terms of responsibility.' (Head of English)

This was reinforced by the head teacher:

'Because a lot of our students are quite unsure. And it's to do with self-esteem as well. And therefore if they're unsure they won't put pen to paper, they won't try things out, they get kind of stuck. They're always looking for reassurance from the teacher. So it put the teacher in a different role - much more of a sort of facilitator role rather than a teacher-led, … if you've got a large class, trying to get round every student. Whereas now they're working much more independently. That actually was the key point for him; he wanted students to be independent learners rather than being spoon-fed.' (Head teacher)

There were also indications of improvement in attainment:

'Attainment I think is improving, definitely. We are about to assess summatively - that would be the proof really - but in terms of the work I'm getting, more and more children are getting to the higher levels from that particular group.' (Head of English)

10.3.4. Whole school impact

In one school, although the teacher had moved on, the formative assessment practices were sufficiently well-embedded in the department that they will continue to be used. This may be in part because the checklists have been prepared and are easy to use. The current head of department explained that they had benefited all of the teaching:

' These checklists are making the children clearer about where they should be; and it makes you clearer about where you're aiming. You know, that this is what is required in a level 8 piece, or whatever it might be, in this particular genre. So that's a benefit: it can inform your teaching and learning. Also, I think it's interesting when you look at what they're achieving, if you're looking at the class you can - it will help you. 'Well, okay, this is maybe an area that I need to work on and perhaps analysing the style and structure in a literary text.' (Head of English).

In most of the secondary schools the impact had largely been restricted to those departments who had piloted the work. Some head teachers were very supportive of the project, others saw only the difficulties.

10.3.5 Dissemination

The extent to which the project had been disseminated within the school varied. In one school, the project manager was not aware of the details of the funded phase of the project, of the impact on staff and students, of the LEA or the LTS led dissemination programme or of the outcomes of the project. There was a 'hands off' approach. As a result of this no school based dissemination had taken place although there were plans for the future. Dissemination within the school had occurred through informal meetings between involved staff and their colleagues and through more formal department based dissemination. The LEA and the LTS had used the experiences of the staff involved in the project in both authority based and nationally based dissemination programmes. The authority dissemination had occurred through the LEA Principal Teacher networks in the subject specialisms of the teachers who had been involved in the project. The EA Officer was seen as the focus of the dissemination strategy in the EA and as the person responsible for getting the Formative Assessment Development Officer seconded to work as an Advisory Teacher for Formative Assessment and the related sphere of Personal Action Planning. The LTS' dissemination activities in which the teachers had been involved were acknowledged as helping to raise the profile of Formative Assessment in the Authority substantially.

In another school, there had been an INSET session after school for about 25 interested staff which had been very successful. Other departments had decided to adopt the checklist system and a member of the Senior Management Team had taken on the task of developing a formative assessment policy for the school. Currently an audit was being undertaken to establish which departments were adopting formative assessment strategies.

There was a clear indication that what was important in the dissemination process was enabling teachers to communicate with each other:

'I think similar In-Service days to the ones that we had would make a huge difference, I think. People actually talking. And I think that was the big thing about those actually because it was ordinary teachers talking, and I think that's really, really important because we often have someone from on high and people just think, 'oh, they don't have a clue.' But if someone's actually standing in front of them you who still teaches, you know, I mean, I think that makes a huge difference.' (English Teacher)

10.3.6 Difficulties

The main difficulty experienced was getting the project underway as the initial presentation by the LTS team was reported as lacking clarity. Once this difficulty was overcome, the biggest problem was in relation to changing practice within the existing structure of the curriculum. This was felt to be time consuming both within (under the new approach to questioning brought by the project) and outside the classroom (in terms of preparation for lessons and reflection about them)

The participating teachers reported that the introduction of peer assessment needed to be carefully thought through as it could create difficulties. They suggested that guidance was required to ensure its maximum effectiveness.

The Senior management teams in the schools felt that involvement in the project had been worthwhile but indicated that the process had been very time consuming and had only been directed at two classes. The implications for adopting the approach throughout the school were considerable and related to class sizes, the pressures of the curriculum, the pressures of reporting to parents, and the impact on attainment as measured through statistical analysis of raw data e.g. national test results. The pressure within secondary schools on summative assessment was a major concern. The major impact of performance in examinations on the life choices of the pupils in the care of schools meant that senior staff had to be confident that the project would not interfere with this success. This could only be assessed in the long term.

10.3.7 Parental involvement

Generally speaking parents had not been involved in the project. This was more marked at secondary level than primary.

10.3.8 Sustainability

In the second year of the project there was some evidence that formative assessment was still being used effectively in the departments where it was developed. In some schools formative assessment practices had been effectively disseminated within the school and were likely to be sustained because they had been taken up by the senior management team and whole school policies were evolving. The school which reported the most success was one where the pupils were amongst the most disadvantaged as assessed by entitlement to free school meals .