« Previous | Contents | Next »
Listen
4.4 Impact on policy and practice
One of the key aims of the evaluation was to determine the impact on policy and practice as a measure of the effectiveness of the programme.
Policy and planning issues
At this stage of the evaluation (May - September 2003) it was recognised that AifL might only be beginning to affect policy at authority and school levels although, as pointed out by one co-ordinator, 'many local authority service plans and school development plans would already have a focus on aspects of teaching and learning, especially assessment', and therefore the impact of the AifL projects could not be easily isolated.
The assessment co-ordinators reported a wide range of assessment issues and initiatives on which they were already working and which inter-related with AifL developments:
- 8 reported that AifL developments were relevant within their teaching, learning and assessment policies and plans
- 7 reported uses of baseline or standardised testing
- 5 mentioned existing developments of PLPs, either as part of new community schools, or as an authority-wide development
- 4 mentioned pupil tracking and target setting
- 2 mentioned developing reporting procedures
- 2 mentioned early intervention strategies.
Other developments mentioned were transition arrangements, involvement in AAP, agreeing standards within National Qualifications, moderation on levels of the 5-14 curriculum, co-operative teaching and learning, self-evaluation, and recording and collection of evidence.
Eleven co-ordinators reported that AifL Programme issues had not been included in teacher induction programmes, while 6 indicated that this was planned for the following academic year. Twelve stated that it had been included in some way, from 'being mentioned', 'referred to briefly' and 'information passed on' to half-day inputs on specific aspects such as formative assessment or National Assessments. One simply reported that it was part of in-service to probationers.
Seven assessment co-ordinators reported that AifL-related staff development had been provided in a variety of ways: for example, in twilight sessions, courses, conferences and seminars - some for management only, but others for all teachers; 4 indicated that development had not yet gone beyond the pilot schools; in 12 cases it was planned to introduce opportunities in 2003 to 2004 (see section 5.3 for a report on these developments).
Headteachers were asked specifically about school policies for assessment. Seventy-nine (86%) had a policy for assessment in place prior to involvement in AifL. Out of the 79, 46 (58%) had modified that policy in the light of their involvement. Of the 13 who did not have a policy in place, only 3 had subsequently developed one. Some 66% of primary schools with policies had made changes because of their involvement while 40% of secondary schools had revised their policy.
Views on how well AifL projects had linked to development planning were sought. For 44 schools (48%) the AifL project had linked very well, with 18 (20%) of the headteachers reporting good links. Nineteen headteachers (21%) felt it had linked 'only a little', 3 (3%) said 'not at all' and eight (9%) did not respond. Out of the 57 primary respondents, 13 (23%) responded negatively, while out of the 21 secondary respondents, 9 (43%) responded negatively.
Case study schools
The issue of linking initiatives such as AifL to development planning was more important for secondary schools who found it more difficult to take on board additional issues or activities which were not on the plan. Many primary schools welcomed the opportunity to become involved precisely because it was on their plans, with teachers commenting that it 'dovetailed with plans'. Some schools have incorporated assessment-related activities into the development plan as a result of their involvement in AifL.
Headteachers were also asked to indicate if they felt that project developments were contributing to meeting their National Priority targets. Their responses are given in Table 4.9.
Table 4.9: Meeting national priority targets
National Priorities | very much | much | only a little | not at all | no response |
|---|
% | % | % | % | % |
|---|
Achievement and Attainment | 20 | 26 | 37 | 2 | 15 |
|---|
Framework for Learning | 18 | 40 | 21 | 8 | 13 |
|---|
Inclusion and Equality | 24 | 24 | 26 | 9 | 17 |
|---|
Values and Citizenship | 6 | 28 | 27 | 22 | 16 |
|---|
Learning for Life | 14 | 37 | 28 | 6 | 14 |
|---|
If part of the aim of providing a coherent framework of assessment is to integrate developments with other aspects of SEED policy, then the majority of teachers and headteachers appear to see AifL making at least some contribution to meeting their National Priority targets.
Impact on practice
All key groups were asked to give their views on the extent to which involvement in the AifL Programme might impact on the practice of teachers in schools.
As a starting point, both headteachers and teachers were asked to what extent the developments introduced through AifL related to existing practice. The questions for the teachers were framed in relation to their personal practice, whereas the headteachers' questions referred to practice within the school. The questions have been combined in Table 4.10.
Table 4.10: Relationship between projects and existing practice
| HT | Teacher |
|---|
n | % | n | % |
|---|
The ideas introduced were completely new to the school and so we were/I was starting from the very beginning | 14 | 15 | 41 | 21 |
|---|
We/I had been thinking about introducing new practices and so the ideas were already in our development plan/had been part of my planning | 16 | 17 | 17 | 9 |
|---|
We were/I was already developing approaches in line with the project and therefore the project provided the opportunity to make further progress | 44 | 48 | 65 | 35 |
|---|
Aspects of the project were well established (as part of my practice) but the project provided us with the opportunity/allowed me to develop new dimensions | 14 | 15 | 53 | 28 |
|---|
The majority of respondents reported that ideas were already developing within the school or were already well established.
Further statistical analysis 2 identified no significant difference between sectors or projects with respect to the relationship of developments with existing practice. Although not statistically significant, it is interesting to note, however, that Reporting to parents and others was the only project in which both headteachers and teachers did not indicate that ideas were new - indeed the majority of headteachers reported that they were already developing approaches in line with the project and the majority of teachers thought aspects were already well established. The small number of Project 8 participants were more likely to indicate that ideas were new or part of planning, but not yet developing. Otherwise, the experience across the projects ranged from completely new to well-established in practice. Therefore schools and teachers had different starting points, and so it might be anticipated that different degrees of change might be experienced.
The local authority assessment co-ordinators and the project DOs were asked to indicate their agreement with three broad statements related to the aims of AifL. They were also given the option of indicating that it was too early to make any comment on progress. Table 4.11 summarises agreement and disagreement.
Table 4.11: Views on the impact of AifL projects ( LA co-ordinators and LTS development officers)
The projects are contributing to … | LA co-ordinators | Development Officers |
|---|
Agree | Disagree | Too early | Agree | Disagree | Too early |
|---|
n | n | n | n | n | n |
|---|
… developing understanding of approaches to assessment in schools | 20 | 0 | 9 | 9 | 0 | 0 |
|---|
…. improved practice in assessment in schools | 21 | 0 | 8 | 6 | 0 | 3 |
|---|
… improved recording and reporting of achievement | 11 | 4 | 13 | 3 | 0 | 6 |
|---|
Comments from local authority co-ordinators indicated that it was difficult for them to comment at this point on anything other than the projects in which they were actually involved, and while 'a good start had been made in project schools with some staff' … there was still a long way to go'. Five of the co-ordinators with schools involved in Project 8 expressed concerns over the lack of clarity and 'shifting goal posts' in this project.
The DOs also emphasised that their view of progress related to the schools who were actively involved in the projects. One indicated that progress 'was in the hands of the schools and sometimes they have just too much to do'. The development officer for Project 3 reported that 'classroom practice has most definitely been enhanced. There is clear evidence in many cases that schools are now more focused in their planning and assessment which has resulted in improved learning and teaching and provided staff with clearer evidence of achievement and of pupils' needs, so as to compile more accurate and succinct records and reports'. Overall, there was greater reservation about progress towards developing a more coherent system of assessment and a unified system of recording and reporting:
- 6 of the 9 DOs thought that the AifL projects would be effective in developing a more coherent system of assessment, with 3 saying it was too early to say; 20 local authority co-ordinators thought they would be effective and 9 thought it was too early.
- 4 of the DOs thought that the AifL projects would be effective in developing a unified system of reporting, while 5 thought it was too early; 13 authority co-ordinators thought they would be effective, 2 disagreed and 14 thought it was too early.
The HE representatives gave lengthy responses concerning progress towards achieving understanding assessment for learning and improving assessment practice in schools. They addressed theoretical issues as opposed to observations of practice - or possibly theoretical underpinnings as to why improved practice might be inhibited - and issues of understanding the process of embedding change in pedagogical practice. The tensions between formative and summative assessment were raised and the views expressed that while it might be possible to produce a unified system on paper, getting to the position where principles were widely embraced (or even understood) was very ambitious. This could not be achieved quickly. A recurring theme was the need to break down barriers, change teachers' attitudes and address the culture of assessment for accountability rather than learning. These views related in particular to the potential for formative assessment to radically change practice. Teachers needed to understand the constructivist principles underpinning a formative assessment approach before they experimented with formative assessment techniques in the classroom. There was the possibility that, if this does not happen, teachers will revert to mechanical, output driven approaches to assessment rather than engaging with change.
All agreed that achievement of the programme aims was a 'massive task' and that it would be a long term undertaking, requiring '5 to 10 years of commitment, enthusiasm and ongoing support' ( LA) - and ongoing funding, as 'without funding it may fade' ( HE).
The issue of impact on practice was explored in much greater detail in the headteacher and teacher surveys, covering impact on teacher practice, benefits to pupils, involvement of parents and impact on information needs, both generally and in relation to PLPs.
Benefits in relation to staff development and changes in classroom practice
Participants were asked to indicate the extent to which they agreed or disagreed with a series of statements on the various ways in which involvement might impact on the practice of teachers in schools. The statements were derived from aims of the projects and from comments noted in discussions with participants. The statements for headteachers were framed to allow them to comment on observed impact on teachers and for the teachers they were framed for them to comment from their own experience. The questions have been combined in Table 4.12. Respondents were given instructions that if they felt they could not express a view they should make a note to this effect or leave blank and explain if possible in an open section. The missing responses have been included in the table as NR.
Missing responses in the headteacher survey occurred mainly in relation to Project 8, with approximately half indicating that the implementation of the project had not involved any changes to classroom practice as it had been mainly a consultative exercise. The others had been actively developing PLPs and recording and reporting systems prior to involvement in Project 8 and linked this with their Project 8 work, and they responded to this question presumably in the light of the ongoing practical developments. One respondent from Project 8 strongly disagreed throughout, adding the comment: 'the project has not progressed at all. Our initial aims/plans were changed and we still await guidance on next steps. There is no communication'. Likewise, a number of teachers involved in Project 8 did not respond to the question, for similar reasons.
Two schools from Project 3 and 2 from Project 9 did not respond to this question, with the Project 9 schools explaining that they had been focusing on improving reporting to parents and so there had been little change to classroom practice. One Project 3 school explained that their PLP was well established as part of New Community School developments and they had focused on involving parents in the process. Others indicated that where they had not responded it was because they felt it was too early to form a view. Similar reasons for non-responses for Projects 3 and 9 occurred more generally in the teacher survey.
Table 4.12: Views on benefits of involvement of teachers (Statements combined from the headteacher and teacher questionnaires)
| Headteachers | Teachers |
|---|
SA | A | D | SD | NR | SA | A | D | SD | NR |
|---|
% | % | % | % | % | % | % | % | % | % |
|---|
a) The teachers involved in the project/I have gained a clearer understanding of different purposes of assessment | 52 | 33 | 4 | 1 | 10 | 39 | 43 | 8.5 | 0.5 | 9 |
|---|
b) The teachers involved in the project/I have changed their/my classroom practice | 33 | 44 | 9 | 1 | 14 | 23 | 48.5 | 17 | 0.5 | 10 |
|---|
c) The teachers involved are/I am using more varied approaches to assessment | 32 | 40 | 12 | 1 | 15 | 26 | 53 | 12 | 0 | 9 |
|---|
d) The teacher are/I am now more aware of the individual needs of the pupils | 24 | 47 | 13 | 1 | 15 | 22 | 51.5 | 18 | 0.5 | 8 |
|---|
e) The nature and quality of feedback to pupils has improved/The way in which I give feedback has improved | 31 | 45 | 7 | 1 | 17 | 30 | 53 | 9.5 | 0.5 | 7 |
|---|
f) Some teachers/I have found it difficult to adopt new practices | 6 | 12 | 38 | 24 | 20 | 0.5 | 7 | 52 | 29.5 | 11 |
|---|
g) Teachers not directly involved in the project have shown an interest in adopting different approaches to assessment | 8 | 52 | 11 | 2 | 27 | 12 | 44 | 13 | 1 | 30 |
|---|
h) Some teachers have shown resistance to the developments | 1 | 13 | 44 | 26 | 16 | 3 | 20 | 41 | 16 | 20 |
|---|
i) Overall, involvement in the project has provided valuable staff development opportunities | 46 | 39 | 4 | 1 | 10 | 30 | 47 | 10 | 1 | 12 |
|---|
Some further cautionary notes were added by headteachers that only a few staff had been involved and wider impact on the school had still to be developed. It was also noted that some staff had benefited more than others due to their willingness to change, for example: 'Not all changed their practice - although twelve teachers took part, in only one or two departments has practice changed' (Secondary - Project 3); and 'Some staff have improved feedback to pupils. Others less so' (Primary - Project 4).
Overall, however, the majority (over 70%) of both headteachers and teachers were in agreement or strong agreement regarding the benefits to teachers in terms of understanding assessment, changing practice, using more varied approaches, greater awareness of individual pupil needs, improved feedback and ongoing professional development.
The slight variations in responses between headteachers and teachers are illustrated in Charts 4.1 to 4.5.
Chart 4.1: Benefits to teachers - clearer understanding of assessment (percentages)
Chart 4.2: Benefits to teachers - change to classroom practice (percentages)
Chart 4.3: Benefits to teachers - more varied approaches to assessment (percentages)
Chart 4.4: Benefits to teachers - improved feedback (percentages)
Chart 4.5: Benefits to teachers - valuable staff development (percentages)
Case study schools
In some of the case study schools, teachers reported that involvement had helped clarify what they were teaching and had led to better planning. In one school teachers spoke of previously depending on national tests and unit checks to determine pupils' levels of ability; they now felt they knew more about the children's levels of learning because of applying formative assessment strategies. Primary teachers involved in Project 4, having been introduced to clarifying criteria in ICT and PE, were transferring their thinking to other areas of the curriculum. They were discussing criteria and information about levels with colleagues
Further statistical analysis of the headteacher data with respect to difference between projects (using Kruskal-Wallis and Mann-Whitney non-parametric tests because of low project-group numbers; p<0.05) indicated that there was a significant difference in relation to improved understanding of purposes of assessment, changing classroom practice and using more varied approaches to assessment. This, however, occurred because the Project 8 participants who responded rated items more negatively than respondents from Projects 1 and 5, which is not surprising given the comments on Project 8 above.
Further statistical analysis of the teacher data (using analysis of variance) with respect to differences between projects identified significant differences (p<0.05) in relation to changing classroom practice, using more varied approaches to assessment, being more aware of pupil needs and improving feedback. The differences in changing classroom practice occurred because Project 1 (formative assessment) respondents rated this more highly than respondents from Projects 3, 8, 9 and 10. This is perhaps unsurprising given that formative assessment specifically focuses on classroom practice while the others were focusing on management and reporting issues, although the explanation for Project 10 is less clear. A number of the Project 10 schools were working on formative assessment, though they had not been involved in the project long when the questionnaire was completed. The difference in relation to using a wider variety of approaches to assessment was again because Project 1 respondents were more positive in their responses than other respondents, particularly those from Projects 3, 9 and 10. The differences with respect to being more aware of pupil needs and improving feedback arose because those involved in Project 10 were more negative in their responses than Project 1 respondents. Later discussions with Project 10 participants revealed that, due to the nature of their work, they are already very much aware of their pupil needs and specialise in formative feedback, and participation in the project had not improved this.
Sector differences were considered using analysis of variance (p<s0.05) and the only difference that emerged in both headteacher and teacher data was showing resistance to new developments. The difference can be accounted for by stronger agreement from secondary respondents compared to primary respondents. Additionally, management respondents from secondary schools were more likely to agree that some teachers found it difficult to adopt new practices, though this difference was not found in the teacher data. These differences are displayed in Charts 4.6 to 4.9.
Chart 4.6: Sector differences - 'Some teachers have shown resistance' (Headteachers) (percentages)
Chart 4.7: Sector differences - 'Some teachers have shown resistance' (Teachers) (percentages)
Chart 4.8: Sector differences - 'New practice difficult for some' (Headteachers) (percentages)
Chart 4.9: Sector differences - 'New practice difficult for some' (Teachers ) (percentages)
Benefits to pupils
A similar question focused on the benefits to pupils as a result of their involvement in the project. The statements were derived from purported benefits of formative assessment stratgies, supplemented by comments noted from participants. The responses from both headteachers and teachers are given in Table 4.13.
Table 4.13: Views on benefits to pupils
| Headteachers | Teachers |
|---|
SA | A | D | SD | NR | SA | A | D | SD | NR |
|---|
% | % | % | % | % | % | % | % | % | % |
|---|
a) Pupils have become more actively involved in the learning process | 32 | 49 | 1 | 0 | 18 | 35 | 46 | 5 | 0 | 14 |
|---|
b) Pupils are better equipped to assess their own learning | 23 | 50 | 5 | 0 | 22 | 23.5 | 56 | 5 | 0.5 | 15 |
|---|
c) Pupils are able to set realistic targets | 9 | 56 | 8 | 0 | 27 | 10.5 | 50 | 16 | 1.5 | 22 |
|---|
d) Pupils are developing skills in peer assessment | 9 | 50 | 14 | 0 | 27 | 15 | 47 | 19 | 1 | 18 |
|---|
e) Pupils have shown improved behaviour in the classroom | 5 | 30 | 30 | 1 | 33 | 5 | 39 | 34.5 | 0.5 | 21 |
|---|
f) The developments enhance the learning of all pupils | 29 | 47 | 4 | 0 | 20 | 21 | 54 | 11 | 0 | 14 |
|---|
g) I have evidence that pupil attainment has improved through the project activities | Teacher question only | 9 | 23 | 24 | 0 | 44 |
|---|
h) Pupils themselves report positive views of project activities | 20 | 47 | 4 | 0 | 29 | 12 | 52 | 10.5 | 0.5 | 25 |
|---|
i) The approaches developed are not suitable for all pupils | 1 | 13 | 47 | 19 | 21 | 5 | 27 | 41 | 10 | 17 |
|---|
j) Pupils have increased in confidence and show greater self-esteem | 13 | 53 | 9 | 0 | 25 | 12 | 60.5 | 10 | 0.5 | 17 |
|---|
k) Pupils are more motivated towards learning | 17 | 57 | 3 | 0 | 23 | 13 | 58 | 12 | 0.5 | 16.5 |
|---|
l) The initiative has had little impact on classroom behaviour | 5 | 23 | 32 | 12 | 28 | 3 | 30 | 37 | 8 | 22 |
|---|
Overall, there was agreement that pupils benefited on all aspects, though there was a higher level of non-response compared to the question on teacher benefits. However, headteachers and teachers who did respond indicated that pupils were more involved in learning, more motivated, better equipped to assess their own learning, positive about the project activities, showing greater confidence and able to set realistic targets. Peer assessment was less likely to have been developed than self-assessment.
The majority of headteachers involved in Project 8 indicated that they could not respond to this question due to limited pupil involvement. Only 3 Project 8 respondents completed this question. Non-responses were also higher in Project 3. However, non-respondents were found across all projects, with more choosing not to answer questions about behaviour, target setting, peer assessment and pupils' views on the projects. Explanations were in many cases that some of the activities had not yet been tried: it was still early days in developments and only a few pupils had been involved. With regard to behaviour, many indicated that behaviour was not a problem - some respondents answered the questions but indicated little change, while others did not answer the questions. One commented: 'improvements in behaviour may be more noticeable in some classrooms than in others. The classroom dynamics vary greatly and so does the impact of the strategies on pupil development' (Primary - Project 1).
In the teacher responses, the majority of respondents from Project 8 and around half of Project 9 participants did not answer this question. A higher number of Project 3 participants also did not respond compared to the remaining projects. As with headteachers, the reasons were due to limited impact of the work of these projects on the issues referred to in the question. Teacher views of the impact on behaviour were similar to those of headteachers.
Of particular note is the high number of teachers who felt that it was too early or not appropriate to consider the potential of the projects to increase attainment, with only one-third indicating that they had evidence of this. This is illustrated in Charts 4.10 and 4.11.
Chart 4.10: 'I have evidence of increased attainment' (Teachers) (percentages)
Chart 4.11: Sector differences - 'I have evidence of increased attainment' (Teachers) (percentages)
Some teachers found it difficult to identify hard evidence: 'There is not hard, quantifiable evidence … since there was no significant body of directly comparable work' (Secondary - Project 1). However, as noted, one-third indicated that they had evidence of increased attainment. One teacher reported: 'I only projected about one-third of class to attain level A writing but one-half attained this level. I believe that the oral feedback was paramount in this' (Primary - Project 1) .
Both primary and secondary teachers referred to improved levels/scores compared with non-project comparator groups; this was mainly in relation to Project 1 but teachers involved in Projects 4 and 5 also reported improved test results, which they attributed to involvement in the project, for example, 'Many pupils attained level F writing in S1 as a result of project. Quality of imaginative writing definitely improved, particularly for those pupils who like a "checklist" to help them' (Secondary - Project 5).
Others spoke in terms of having evidence to send to parents, having observed increased participation and motivation of pupils, and receiving comments from both parents and pupils.
Other comments from headteachers regarding benefits to pupils included:
'Involving children with significant learning difficulties in their own self assessment has been challenging and exciting. It is so good to see their full potential' (Special - Project 4)
'Children know exactly what they are learning, how well they are doing and what they need to do next. They are greatly motivated by their success in achieving small steps' (Primary - Project 2).
Further comments from teachers included:
'Children have become more aware of themselves as learners and also are developing awareness of how their input affects others (either negatively or positively) in the context of group/class work. They are becoming much more effective in identifying areas for improvement in terms of their own learning' (Primary - Project 1)
'The class with whom I have worked are a top S1 English class, so attainment has always been high. However, I have seen increased confidence in the class through the new approach' (Secondary - Project 1)
'Children were apprehensive at first as they had not much experience of target setting. However as the project developed they gained confidence and were able to comment on the benfits of target setting and evaluating targets. They were keen to set targets in other subject areas' (Primary - Project 2).
Further statistical analysis of the headteacher data (using Kruskal-Wallis) identified no significant differences regarding benefits to pupils between projects or sectors. Further analysis of the teacher data was undertaken though Projects 8 and 9 were omitted due to the small numbers who responded from these projects. Analysis of variance was carried out on the remaining projects. This revealed that, on all items except b, c, f and i in Table 4.13, teachers involved in formative assessment were more likely to be positive about pupil outcomes than those involved in Managing PLPs. This is perhaps not surprising due to the difference in purpose of the projects. In relation to sectors, teachers from special schools were less likely to agree that pupils were able to set realistic targets and were more likely to agree that the approaches are not suitable for all pupils.
Case study schools
Teachers from some of the case study schools reported a wide range of benefits to pupils: pupils were more aware of what they need to do; they were more aware of their strengths and weakness; they were more focused on their skills; they were able to see their own progress; they were clearer on their targets and got a greater sense of achievement when they reached them; pupils were more independent of their teachers; there was greater learner autonomy. In one school the headteacher reported that pupils who had been introduced to target setting through Project 4 were going into other classes with the expectation of getting targets. This, she said, 'changes the ethos in the classroom'.
Involvement of parents/carers
One of the key aims of the AifL Programme is to involve parents and carers more actively in the progress of their children's learning. This is a particular feature of Personal Learning Plans and Reporting to Parents, but all projects had the opportunity to share with parents regarding their involvement in the AifL Programme. In both the headteacher and teacher responses, given below, where increased parental involvement is reported, schools are representative of all projects except Project 5. This project was still in its early stages and it was hoped that parental involvement would increase as it continued; plans for future involvement of parents was mentioned across all projects. However, higher proportions of respondents from Projects 2, 3 and 9 responded positively to the statements about parental involvement.
Some headteachers reported that involving parents had not been one of their aims. A number also reported that relationships with parents were strong and it was not easy to say that more had become involved, though the quality of communication may have improved.
Sixty-seven headteachers (73%) and 110 teachers (58%) reported that parents had been informed of the school's involvement in the project; 37 headteachers (36%) and 56 teachers (30%) reported that parents were also involved in the evaluation of the project. Thirty-three headteachers (36%) felt that the project had enabled parents/carers to become more actively involved in contributing to assessment and target setting for their children, while 56 teachers (30%) agreed.
Twenty-seven headteachers (29%) considered that 'many' parents of the children taking part in the project had become more actively involved in their children's learning, 6 (7%) thought it was 'about half' and 11 (12%) reported 'a few'. Teacher estimates varied slightly: 34 teachers (18%) considered that 'many' parents were now more involved, while for 20 (11%) it was 'about half' and for 13 (7%) it was 'a few'.
Thirty-four headteachers (38%) and 62 teachers (34%) reported that involvement in the project had resulted in changed practices in communicating with parents/carers. Examples provided included:
- Personal Learning Plans are given to parents and so parents are more informed, more involved and engage more in discussion with the teachers; PLPs have created a dialogue. (This was the most frequently mentioned change and occurred across Projects 1, 2, 3 and 9)
- parents are more willing to be involved in discussing assessment issues (Project 1)
- parents are asked to comment on pupil work on a regular basis (Project 1)
- workshops for parents have been organised (Projects 1, 3 and 9)
- a programme for sharing learning outcomes with parents has been initiated (Project 3)
- longer time given to PLP meetings than normal parent meeting (Project 3)
- clearer reports can be given to parents (Project 4)
- video-disc sent home to parents and used for discussion of progress (Project 4)
- consulting with parents on what they want reported (Project 9)
- parents have been involved in working groups (Project 9)
- arrangements for meetings with parents more informal (Project 9).
Case study schools
Two Project 9 schools had involved parents in focus groups, in one case to develop reporting policy and in the other to review parents' evenings. Parents valued being asked to take part and reported that it made them feel that they were being listened to, especially when they could see the changes discussed being put into practice. They felt it was important for the school because it gave them a cross-section of parental views before going to the wider parent group. They saw their involvement as a way of contributing to the school.
Others involved in Project 9 reported that it had made teachers more aware of parents' needs and enabled them to listen rather than assume they knew what parents wanted. For one primary school it had made them 'willing to adapt their strategies to reach out to the parent population'.
Impact on information needs (including PLPs)
A key element of the AifL Programme is to ensure that the information needs of pupils, parents, teachers and others are met. The headteachers were asked to indicate the extent to which they thought a range of information needs was being met by the AifL Programme, and the responses are reported in Table 4.14.
Table 4.14: Extent to which AifL has contributed to meeting information needs: (Headteachers)
Pupils | Very much | Much | Only a little | Not at all | No response |
|---|
% | % | % | % | % |
|---|
How well they are progressing through a level | 16 | 34 | 22 | 9 | 19 |
|---|
How well they are developing skills | 16 | 48 | 13 | 2 | 21 |
|---|
What progress they are making over time | 18 | 44 | 15 | 2 | 21 |
|---|
Identifying next steps in their own learning | 25 | 29 | 20 | 4 | 22 |
|---|
Confidence in their own judgements of their achievements | 21 | 29 | 22 | 2 | 26 |
|---|
Class teacher |
|---|
Knowledge of individual pupil progress | 23 | 37 | 14 | 6 | 20 |
|---|
Recording of individual pupil progress | 24 | 29 | 21 | 4 | 22 |
|---|
Planning next steps for individual pupils | 24 | 38 | 13 | 4 | 21 |
|---|
Planning next steps for groups | 21 | 34 | 15 | 5 | 25 |
|---|
Confidence that assessment judgements are valid and reliable | 22 | 35 | 11 | 8 | 24 |
|---|
Effectiveness of materials and resources used in class | 12 | 38 | 21 | 6 | 23 |
|---|
Effectiveness of teaching approaches | 17 | 39 | 13 | 8 | 23 |
|---|
Pupil performance against national standards | 10 | 26 | 27 | 11 | 26 |
|---|
Other teachers |
|---|
What a pupil has already achieved at an earlier stage | 7 | 23 | 25 | 12 | 33 |
|---|
Aspects in which pupils need specific support | 16 | 25 | 17 | 9 | 33 |
|---|
Effective collating and sharing of information for transition from nursery to primary school | 3 | 8 | 6 | 25 | 58 |
|---|
Effective collating and sharing of information for transition from primary to secondary school | 9 | 18 | 20 | 16 | 37 |
|---|
Parents and carers |
|---|
Understandable information about their child's progress | 17 | 23 | 24 | 9 | 27 |
|---|
Specific strengths of their child's performance | 22 | 25 | 18 | 8 | 27 |
|---|
Aspects which need support | 21 | 24 | 19 | 8 | 28 |
|---|
Local Authority |
|---|
Monitoring progress of schools against targets | 5 | 8 | 29 | 21 | 37 |
|---|
Achieving aims of authority's improvement plan | 10 | 19 | 22 | 14 | 35 |
|---|
The missing responses were mainly from the same schools throughout, with additional schools logging no response to some statements and, in particular, in respect of other teachers' and local authority needs. Those responding 'not at all' were spread across all projects, though from a limited number of schools. Many indicated that it was too early to know what impact projects might have on meeting many of the needs and some hoped that as the project progressed (for example PLPs) improvements would be made. Some suggested that they had in place good reporting systems and as such did not need to improve. One primary headteacher (Project 4) commented: 'As 4 teachers were involved, the above are difficult to answer. In one class the results are very positive but it varies after that to little or no impact'. A primary head from Project 5 commented : 'We are still in the early stages of the project, but our transition information from P7 to S1 was more specific and related to next steps in learning' .
At this stage the projects appeared to be contributing towards the needs of pupils and class teachers, with 50% or more of respondents agreeing they contributed much or very much to the items related to these groups (except for performance against national standards). Just under half thought that improvements were being made with regard to parents. Other aspects required further development.
These data are illustrated in Chart 4.12 in which the data from Table 4.14 has been summarised by calculating respondents' average ratings for each information user group and collapsed into agree, disagree and no view.
Chart 4.12: Progress towards meeting the information needs of various groups (Headteachers) (percentages)
Teachers were asked in more detail about the implementation of PLPs and were asked to give their views on the extent to which PLPs were contributing to meeting information needs.
Forty-five teachers had been using PLPs prior to involvement in the AifL projects: that is, 30 from 'non- PLP projects' and 15 from ' PLP projects'. Seven out of the 30 non- PLP project participants indicated that PLPs were used as a feature within their AifL projects: these were integrated into the developments of Projects 1, 4 and 9. PLP projects which already had them in place either redesigned their PLP, extended it to other stages, eg nursery, developed it for other aspects of the curriculum or had begun to look at electronic support as part of their involvement in AifL.
The questionnaire was designed on the assumption that all those participating in Projects 2, 3 and 8 would be using PLPs. However, in reality a number of teachers did not actually use PLPs during 2002 to 2003, which was the development period covered by the questionnaire. A number had focused on designing the PLP or undertaking curricular audits to prepare for PLPs. As indicated before, a number of Project 8 participants had acted only in advisory roles and so did not use PLPs either. Therefore, although there were 61 respondents who were linked to PLP projects, only 47 felt able to answer the questions about the extent to which PLPs were assisting in meeting information needs. Six participants from other projects also answered this question. The respondents represented 23 schools: 16 primary, 4 secondary and 3 special.
With respect to demands on teacher time, 28% thought that PLPs were easily managed, 68% thought they were manageable but demanding and 4% thought they were excessively demanding. With respect to pupil time, 6% thought PLPs were easily managed, 75% thought they were manageable but demanding and 19% thought they were excessively demanding.
Issues which made them demanding included the development of the PLP before putting it into practice and the time spent with each individual child discussing targets. It was noted that children 'spend quite a lot of time thinking about targets' and that checking and reviewing these were time-consuming. It was suggested that it could not have been done without additional support staff or cover being available; it was felt that meeting with pupils and target setting could not take place within normal class time. Where PLPs were being tried with some pupils, it was noted that the task could not have been done if all pupils had been involved.
One teacher commented: 'Well worth the input of time. As I become more accustomed to using them and the children more adept at self-assessment, time implications should be lighter' (Primary - Project 8). On the other hand, another teacher from the same school commented: ' PLPs require a great deal of teaching and discussion with the pupils in order to get worthwhile, achievable targets. The pupils dislike them and find doing them a dreary task'.
Only 6 teachers reported trialling PLP software and they reported it to be good or very good (Picasso, Filemaker Pro, and a local authority profiling system).
Teachers' perceptions of how much PLPs contributed to improving information needs are reported in Table 4.15.
At this stage of development (October 2003), teachers were more likely to agree that PLPs would contribute to meeting the needs of the class teacher, pupils and, to a lesser extent, parents, than to meeting the needs of other teachers and the local authority. The levels of non-response might be taken to indicate areas in which respondents had not yet used PLPs. It was unclear how the use of PLPs in schools could meet the information needs of local authorities.
As with the headteacher data, the data in Table 4.15 has been summarised and is illustrated in Chart 4.13.
Table 4.15: Extent to which PLPs have contributed to meeting information needs (Teachers)
Pupils | Very much | Much | Only a little | Not at all | No response |
|---|
n | n | n | n | n |
|---|
How well they are progressing through a level | 7 | 20 | 14 | 6 | 6 |
|---|
How well they are developing skills | 6 | 28 | 15 | 2 | 2 |
|---|
What progress they are making over time | 9 | 24 | 13 | 2 | 5 |
|---|
Identifying next steps in their own learning | 17 | 17 | 11 | 7 | 1 |
|---|
Confidence in their own judgements of their achievements | 13 | 18 | 12 | 4 | 6 |
|---|
Class teacher |
|---|
Knowledge of individual pupil progress | 17 | 18 | 11 | 5 | 2 |
|---|
Recording of individual pupil progress | 19 | 19 | 8 | 7 | 0 |
|---|
Planning next steps for individual pupils | 19 | 21 | 9 | 6 | 9 |
|---|
Planning next steps for groups | 18 | 20 | 9 | 6 | 0 |
|---|
Confidence that assessment judgements are valid and reliable | 8 | 21 | 9 | 6 | 9 |
|---|
Effectiveness of materials and resources used in class | 4 | 18 | 17 | 7 | 9 |
|---|
Effectiveness of teaching approaches | 8 | 23 | 8 | 7 | 9 |
|---|
Pupil performance against national standards | 0 | 14 | 13 | 11 | 15 |
|---|
Other teachers |
|---|
What a pupil has already achieved at an earlier stage | 7 | 17 | 5 | 8 | 16 |
|---|
Aspects in which pupils need specific support | 11 | 10 | 5 | 11 | 16 |
|---|
Effective collating and sharing of information for transition from nursery to primary school | 5 | 5 | 1 | 10 | 32 |
|---|
Effective collating and sharing of information for transition from primary to secondary school | 7 | 11 | 2 | 7 | 26 |
|---|
Parents and carers |
|---|
Understandable information about their child's progress | 12 | 10 | 11 | 5 | 15 |
|---|
Specific strengths of their child's performance | 10 | 10 | 14 | 4 | 15 |
|---|
Aspects which need support | 10 | 10 | 13 | 5 | 15 |
|---|
Local Authority |
|---|
Monitoring progress of schools against targets | 1 | 4 | 3 | 10 | 35 |
|---|
Achieving aims of authority's improvement plan | 5 | 4 | 3 | 8 | 33 |
|---|
Chart 4.13: PLPs' contribution towards progress in meeting the information needs of various groups (Teachers) (percentages)
Benefits and challenges of AifL Programme and what might be done differently
In a series of open-ended questions, participants were asked to identify benefits and, in retrospect, what they might have done differently. Headteachers and teachers were also invited to identify challenges.
Benefits
Local authority assessment co-ordinators, project DOs and HE representatives gave a wide range of responses and the views reported are those where there was most agreement.
All three groups were in agreement that a key benefit of the programme had been enhanced teacher motivation and enthusiasm - the descriptions were numerous: re-energised, satisfied, confident, renewed enjoyment. One co-ordinator stated that is was good to watch teachers enjoying what they were doing. This was related to other benefits for teachers such as teacher control and ownership of developments, increased emphasis on and opportunity for teachers to reflect on their work, and the opportunity for teachers to develop their knowledge, skills and understanding.
The benefit mentioned most frequently by local authority co-ordinators was the opportunity to network and share practice with other authority staff and DOs. The opportunity for teachers to work with those in other schools was also important. Local authorities mentioned the benefit of being involved in a national initiative as it gave direction and structure to authority planning and, in at least two cases, endorsed and lent support to what the authority was wanting to do: one co-ordinator reported that the high profile of a national project meant that non-AifL schools took note of what was happening in neighbouring AifL schools and so took the developments seriously. Local authorities also mentioned the benefits of receiving financial support.
Other benefits, though less frequently mentioned, were: greater pupil involvement in learning ( DOs and HE representatives); development of research and action research based in schools ( LA co-ordinators and HE representatives); the projects have produced evidence which will support change in practice in Scottish schools ( DO).
School respondents were also given the opportunity in open-ended questions to list up to 3 benefits and 3 challenges of being involved in the project. An overview across all projects is given here, while a more detailed analysis on a project-by-project basis is included in Appendix 4. Projects 5, 8, 9 and 10, as second phase projects, were in earlier stages of development and so fewer responses were given in relation to benefits and challenges for these projects than for Projects 1, 2, 3 and 4.
For headteachers the key benefits identified were:
- raised awareness of issues of teaching, learning and assessment and of research (62)
- increased involvement of the pupils in learning, assessment and in supporting each other (33)
- staff development (31)
- working with others, including teachers from other schools and non-teachers (23)
- better links with parents (19)
- opportunity to reflect on practice (17)
- using technology more effectively (9)
- financial support for developments (5)
- being part of a national initiative (5).
A small number of other benefits were identified by individual schools, including improved primary-secondary liaison, improved approaches to reporting, increased parental understanding of how what they do at home helps their children learn and influencing the direction of the school plans.
In terms of benefits to themselves as teachers participating in the project, the key themes raised by teachers were:
- opportunities for development for the individual teacher (by far the most common issue, with 237 mentions)
- benefits in terms of working with others (58)
- opportunities for pupil development (38)
- increased pupil involvement in the assessment process (39)
- better links with parents/carers (22)
- improved use of technology (16).
Challenges
Headteachers were asked to identify up to three significant challenges in taking part in the programme. All but 6 schools identified at least one, with most giving 2 or 3. Responses were categorised into themes as follows:
- Lack of time (33)
- Pressure on timetables and schedules in order to fit project into existing programmes and keeping the project on track (26)
- Ensuring supply cover for staff involved (23)
- Staff ownership, or reluctance of some staff to be involved (20)
- Lack of clarity in what was expected of the project (14)
- Technical problems (8)
- Lack of support (4).
Twenty-six headteachers identified other challenges. A number of those involved in Projects 2 and 3 highlighted the challenges of finding an effective system to manage PLPs and to ensure progress and continuity from year to year. Others found involving parents challenging. Other issues mentioned were keeping all staff involved, involving new staff, and writing up the project report or case study.
Similarly, teachers identified a range of challenges that they had experienced as a result of their involvement in the project:
- having to change one's own practice and do things differently (110)
- management of time, timetables and schedules (73)
- availability of time (45)
- resistance of pupils to new ideas/ways of working (42)
- resistance of colleagues to new ideas/developments (15)
- making a presentation at a conference or seminar (10)
- technical problems (7).
Other challenges mentioned included 'getting started', getting parents involved and, for those involved in developing PLPs, identifying an approach suitable for all pupils and for continuity between years.
What might have been done differently?
In relation to what might have been done differently, all three groups of key stakeholders suggested that a longer lead-in time would have been beneficial. All felt that they needed a clearer understanding of the overall programme from the outset. DOs and HE representatives both thought that more time was needed to understand the underlying theories and principles before starting to work in schools. DOs and local authority co-ordinators both felt that the details of each project needed to be thought through in greater detail: in particular there was need for clearer milestones and guidelines before schools became involved. In respect of greater forethought, all three groups spoke about the relationship between the three projects related to personal learning plans (Projects 2, 3 and 8). Each had a different suggestion but the message was that better advance planning would have led to a more satisfactory outcome.
Other suggestions for change included ensuring that initiatives coincide with the development planning cycle and allowing a longer timescale for completion of projects and reporting. All three groups also suggested that earlier opportunities to make links across projects were important (see concerns below), but there were no suggestions as to how this might have been achieved.
Fifty-nine headteachers (64%) made comments. Those involved in Projects 9 and 10 mostly indicated that it was too early to say at this stage as they were still carrying out their initial plans. Most of the comments were project-specific, though a few were cross-cutting, for example:
- involved all staff or more staff (10)
- planned better, linking work to development plan (5)
- nothing (3) with one adding 'although it nearly killed us' (Project 4).
Project-specific suggestions included:
- Project 1: limited the number of strategies used (1); informed parents more about project (1)
- Project 2: involve staff and pupils in target setting before doing PLPs (2)
One secondary headteacher responded: 'I think a working model of a PLP, developed centrally, good or bad, may have been useful; a working model - to be altered etc. in light of experience'
- Project 3: used ICT or given more time to developing ICT skills (3); involved more staff rather than SMT (3); allowed more time for work (2)
- Project 4: allowed more time for developments (6); kept non-participating staff and parents more informed (3); spent more time trialling the instruments (2); chosen different staff (1); allowed teachers to observe good practice in another school (1)
- Project 5: had more communication within school and across cluster schools (3); link to other formative assessment work (1)
- Project 8: headteachers commented mostly in relation to having a clearer structure and objectives for the project (9) - 'less talk and more action'; one suggested they should have waited until the findings of Project 2 had been published before tackling the question of ICT use for PLPs
- Project 9: involved parents at an earlier stage (1); allowed more time for developments (1); made use of website (1).
One hundred and twenty-six teachers (66%) made suggestions in response to the question about what they would have done differently. A wide range of comments were given, many project- and school-specific. Where questionnaires were received from several teachers in one school, similar points were often made. Twelve indicated that they would change nothing, with one stating: 'I was very pleased with the project'. Points which were made across the projects included:
In-school issues:
- would have involved more staff within the school (12)
- would have liked more time to have been available for staff development or discussions with colleagues (7)
- would have tried to plan better initially - for example, more effective time-planning, more careful planning of project work (5).
External issues:
- would like to have been better informed at beginning - for example, clearer guidance about project timescales, workload and outcomes (12)
- would have liked more contact with others involved in the same project (9)
- would have liked to develop project over longer timescale - for example, started earlier, taken longer, allowed more time for preparation and planning (9).
In relation to Project 1, a number suggested that they would have focused initially on fewer formative assessment strategies, or just one; in relation to Projects 2 and 3, some suggested working with groups of children rather than individuals and a small number indicated they would have changed the format of the PLP; some also suggested they would have focused on different aspects of the curriculum. In Project 4 it was suggested that it would have been better to focus on a smaller number of children rather than including the whole class. For Project 8, one respondent suggested it would have been better to use a professional software developer. Several teachers involved in Project 9 said they would have changed the parent questionnaire. A number of more personal comments were made - one teacher would have liked to have video-recorded his/her own teaching more, and two felt they should have had more confidence in themselves.
« Previous | Contents | Next »