Scottish Government

Evaluation of Project 1 of the Assessment is for Learning Development Programme:
Support for Professional Practice in Formative Assessment
Final Report

18. Appendices

Telephone Interviews

Interview schedules for head teachers/ school managers of schools involved in the programme

School name:

How many pupils are on roll in your school?

What is the nature of the catchment area?

What are the key characteristics of the school which define its ethos?

What are the aims of the support for professional practice in formative assessment project as it has been developed within your school?

How has the project been implemented?

How many classes are involved?

How were the action plans developed?

Have any difficulties arisen in relation to the implementation of the project?

What have been the positive outcomes for teachers directly involved in the project in relation to pedagogy?

What have been the positive outcomes for teachers directly involved in the project in relation to teacher understanding of assessment issues?

What have been the positive outcomes for teachers directly involved in the project in relation to attitudes?

What have been the positive outcomes for teachers directly involved in the project in relation to motivation?

Have there been any other benefits of the project?

Have there been any positive outcomes for the whole school? If so, what were they and how did they come about?

Have there been any changes in the learning climate in the school?

Have there been any positive outcomes for the pupils in relation to learning?

Have there been any positive outcomes for the pupils in relation to motivation?

Have there been any positive outcomes for the pupils in relation to behaviour?

Have there been any positive outcomes in relation to parents' interest and involvement in their children's education?

Have there been any unexpected benefits of any kind?

Have there been any negative outcomes? If so what were they?

What have been the key features of the project that have contributed to its successful outcomes?

What have been the key features which have contributed to its unsuccessful or less successful outcomes?

What difficulties have there been in implementing the project?

How effective do you think the whole assessment programme has been, not just the development of support for teachers formative assessment?

Do you think that the whole assessment programme overall is worthwhile? If so why and in what ways?

Interviews with head teachers of comparator schools

Name of School:

How many pupils are on roll?

What is the nature of the catchment area?

What are the key characteristics of the school which define its ethos?

What do you consider to be the role of formative assessment in the enhancement of learning and teaching in your school?

What are your aspirations for how the project might enhance learning and teaching in your schools in the long term when the project is rolled out to more schools?

Are you undertaking any development work in your school relating to formative assessment? If so, can you describe it and any effects it has had?

Do you have any concerns about the implementation of the nationally developing assessment project?

Interviews with Education Authority Co-ordinators and other involved managers in EAs

To what extent does the formative assessment project relate to existing Education Authority developments concerned with learning and teaching?

To what extent do you see formative assessment as being central to promoting effective learning?

To what extent do you believe that the project has been successfully implemented in the schools in your Education Authority?

Are there differences in its success in different schools? If so, what has contributed to this differential success?

How effective has the project been in generating ideas and changing practice in relation to formative assessment in the schools where it has been implemented?

How will good practice be disseminated within the Education Authority?

Interviews with Project development officers and university support personnel

How successful do you think that the project has been in the schools you have been involved with?

To what extent have there been differences between schools in the level of success? What may be the reasons for this?

What difficulties have been experienced by schools in implementing the project?

Have there been differences in the way that the Education Authorities have supported schools in implementing the project?

Has this effected the implementation of the project? How?

What difficulties have Education Authorities experienced in implementing the project?

What difficulties, if any, have you experienced in supporting the implementation of the project?

How effective do you think the project has been in generating ideas and changing practice in relation to formative assessment?

To what extent have you observed changes in teachers' thinking and practices? Can you give some examples?

To what extent have you observed changes in pupils' learning?

To what extent have you observed changes in pupils' motivation?

To what extent have you observed changes in pupils' behaviour?

To what extent have you observed changes in the nature or level of parental involvement with pupil learning?

Overall, do you think that the project has been worthwhile? If so, why and in what ways.

Interview Schedules for school visits

Interviews with head teachers / school managers

Nature of Project

What are the aims of the project as developed within your school?

What is the nature of the project as developed in your school?

How has the project been implemented?

Difficulties in implementation

What difficulties, if any, have arisen in relation to implementation of the project?

How did you over come these?

Impact on teachers

What kinds of assessment are carried out in the school?

What are their purposes?

How do the new practices differ from previous approaches?

What have been the benefits for the teachers involved in the project?

Has their pedagogy changed?

Do they understand issues related to assessment better?

Has there been any change in their attitudes or motivation?

Have there been any negative outcomes for the teachers?

Impact on the whole school

What has been the impact on the outcomes for the whole school?

Have there been any positive effects?

Have there been any negative effects?

Have there been any changes in:

• the learning climate in the school?
• pupil learning?
• pupil motivation?
• pupil behaviour?
• parental involvement in their children's education?

Have there been any unexpected benefits?

Have there been any negative outcomes?

What are the key features of the work that have contributed to its success or failure?

Development of the project as a whole

How effective has been the development and operation of the project as a whole?

• Support from LTS: clarity/timing
• Support from EA co-ordinator
• Resources
• Meetings/conferences

What might have been improved in its development? What might have been done differently?

Overall, has the project been worthwhile?

What would you like to see as the next stages of this project?

Sustainability

How sustainable do you think this project is in your school:

In relation to other teachers?

As part of school policy?

Dissemination

On the basis of your experiences, how do you think the outcomes of the project can best be disseminated and developed in the future?

Teacher interviews

Aims and implementation

As a teacher what are your aims in being involved in the project?

What is the focus of your work in the project?

How have you implemented the project?

Difficulties

What difficulties have you experienced in implementing the project?

How did you overcome these difficulties?

Changes in understanding and practice

What kinds of assessment are carried out in the school?

What are their purposes?

How do the new practices differ from previous approaches?

What kinds of assessment are carried out in the school?

What are their purposes?

How do the new practices differ from previous approaches?

Has your understanding of the nature and purpose of formative assessment changed?

Do you think that your practice has changed as a result of being involved in the project? If so, in what ways?

Outcomes

What positive outcomes have there been for you?

Have there been any negative outcomes in relation to your practice or professional life?

Impact on pupils

What have been the effects on the pupils?

Have there been any changes in relation to:

pupil learning?

pupil motivation?

pupil behaviour?

pupil attainment?

parental involvement in their children's education?

Have there been any negative outcomes?

What are the key features of the work which have contributed to its success?

What have been the weaknesses of the project?

Development of the project

How effective has the development of the project been overall?

How effective has the operation of the project as a whole been overall?

Was it worthwhile taking part in the project?

Do you think that the project has influenced the learning climate within the school as a whole?

How do you think that the project should be developed in the future?

Dissemination

On the basis of your experiences, how do you think the outcomes of the project can best be disseminated and developed in the future?

Group interviews with pupils

Group interviews will be undertaken with at least two groups of pupils in each school, to include:

• a minimum of 12 pupils;
• representing boys and girls;
• representing different levels of attainment

whose teachers have been directly involved in the project.

The composition of each group to be left to:

• the researcher's discretion to be representative of the classes involved:
• the discretion of the school to cause them the least inconvenience.

The researchers will seek informed consent from the pupils to be involved in the interviews through their school, in accordance with the recently published Code of Practice. If possible schools to gain parental permission for pupils to participate.

Questions to be based on concrete and recent experience where possible. In some cases this may be related to feedback on written work in which case the questions could centre on that.

Questions below are guidelines only and should be adapted and added to depending on the nature of the intervention and the age of the pupils. They may also change depending on whether children are interviewed immediately after observing a teaching session.

Learning in the observed lesson

What were you learning about in that lesson?

How did your teacher help you to learn?

Do you think his/her approach was successful?

What was good about it? Why?

What was bad about it? Why?

Questioning

Tell me about what happens when your teacher is asking questions.

Can you remember a question time in the last few days?

Do you put your hands up, shout out?

Do you have to answer quickly?

Can you talk about your answer with others?

Is this what happens with all your teachers? Tell me about it.

Why do you think the teacher asks you questions?

Is it helpful? Why?

How do you feel when the teacher asks questions?

What is good about questioning sessions? Why?

What is bad about questioning sessions? Why?

Do most of the children in your class like questioning sessions? Why?

Do most of the children in your class pay attention in questioning sessions? Why?

Do most of the children in your class behave in questioning sessions? Why?

Discussion

Can you remember a discussion session that you have had recently? Tell me about it.

Why do you think the teacher let you have a discussion?

Is this helpful? Why?

How do you feel about having discussions? Why?

Do you get the opportunity to put your point of view?

What is good about discussions? Why?

What is bad about discussions? Why?

Do most of the children in your class enjoy discussions? Why?

Do most of the children in your class concentrate in discussions? Why?

Do most of the children in your class behave when you have discussions? Why?

General assessment issues

Why do teachers need to assess your work?

How are you usually assessed?

Do you like being assessed?

What is good about it? Why?

What is bad about it? Why?

Does being assessed help your learning? If so, how?

Written feedback

Can you show me some examples of how your teacher marks your work?

Is it helpful? Why?

Do you take any notice of it? How? Why?

What do you feel like when you get comments about your work? When they are good? When they are critical? Why?

What makes you want to work harder? Why?

Does anything make you not want to work? If so, what?

Self-assessment

Do you assess your own work? How?

Does your assessment usually match that of the teacher?

Is it helpful to assess your own work? Why?

Peer assessment

Do you assess each other's work? If so, is that helpful? Why?

What do you learn from this?

Do your assessments match those of the teacher?

Parental reactions

Do your parents look at your work?

Do they read the comments on it?

What do they say about it?

How do you feel about what they say? Why?

Interviews with EA co-ordinators

The EA co-ordinators will be asked about each school involved in the observation.

These interviews will have to be undertaken by telephone if the co-ordinator is unable to visit the school when the interviewer is there.

Questions

How were the teachers' agreements and action plans developed in X school?

Are there any particular circumstances in X schools which may have contributed to the relative success of the project?

Are there any factors in the local environment that may have contributed to the outcomes of the project in that school?

Are there any particular characteristics of the staff that may have contributed to the outcomes of the project in that school?

Are there any particular characteristics of the school that may have contributed to the outcomes of the project?

Have there been any differences in the nature of the development of the implementation of the project in this school? If so, why?

Has the project had any impact on the school and its policies and practices?

Has the project had any impact on the staff in the school and their practices?

Do the staff in the school have a greater understanding of the issues about assessment?

Has the project had any impact on the pupils in the school in relation to learning, motivation, behaviour or attainment?

Has there been any impact on the learning climate of the school?

How has the school approached the task of self-assessment in relation to the project?

How valid is the self-evaluation?

Observation of classes

School:

Teacher name:

Department (if relevant)

Subject or topic:

Stage/age of pupils

Number in class

Type of classroom organisation

Overall aims of the lesson:

Content of lesson

Assessment approaches being employed

Aims of the lesson in relation to formative assessment:

Length of lesson

Exposition of the learning outcomes of the lesson

Format of lesson (describe combinations of teacher presentation, discussion, interactive discussion, group work, individual work and timing of different elements of the lesson)

Description of the lesson content and activities with particular respect to the aims in relation to formative assessment

Evaluation checklist

Evaluation of the formative assessment project

School…………………………………..

Role in relation to school (Head teacher, teacher, EA coordinator) ……………………………………

Years of teaching experience ………………………………………..

Please indicate below how successful the project has been in:

Please add here any further information that you feel may be helpful to the evaluation team. Feel free to write on the reverse of the paper if necessary.