The Scottish Government

« Previous | Contents | Next »

Listen

PERSONAL LEARNING PLAN PROGRAMME: 2002-2004
EVALUATION REPORT

2 EVALUATION AIMS, STRATEGY AND METHODOLOGY

The purposes of this evaluation were:

• to report on the effectiveness of personal learning plans devised by schools in addressing the key elements of the seminal framework produced by the Scottish Executive Education Department (SEED) in 2000 (Phase 1)
• to report on the effectiveness of a prescriptive model based on core skills (' Skills for Living') in addressing the key elements of the SEED framework ( Phase 2)
• to examine differences in implementation and perceived success between phase 1 and phase 2 approaches to personal learning planning
• to identify factors that support the appropriate development of personal learning plans; and factors that are unhelpful
• to comment on progress towards aims:

  Project 2:
  to develop a consistent model of reporting using a single framework for PLPs;

  Project 3:
  to produce guidance for schools and teachers on the management of the record- keeping process and PLPs. (SEED [2002] Assessment is for Learning)

The evaluation employed a mixed methodology - documentary analysis; case studies of PLP implementation; and a questionnaire to all participating schools.

2.1 Methodology for Phase 1 - School-Devised Models of PLPs

2.1.1 Questionnaires

In order to collect the views of staff across the 32 schools participating in the project, a questionnaire based on the SEED 'framework' document was designed. This was issued to all schools in November 2003, some 15 months into the project. The questionnaire consisted of 31 double items asking respondents to a) rate the strength of each feature in their PLP project; and b) rate its importance for PLP development ( Appendix D).The scales were:

For purposes of analysis, the questionnaire items were grouped in themes:

1. Impact - pupil attitude and skills
2. The PLP process
3. Pupil self-evaluation and target-setting
4. Continuity and transition
5. Partnerships: professional and personal
6. Taking account of the whole person
7. PLPs: coverage and content.

2.1.1.1 Issues

Questionnaire Distribution

At the time of questionnaire construction it was unknown to the evaluators how many staff in each school were involved in the PLP project. Therefore, four or five questionnaires were sent to each school with a letter to the headteacher requesting that all staff involved in PLP work should complete the questionnaire. Respondents were asked to identify their post in relation to PLP work - this included staff in services other than education, such as Psychological Services and Social Work, in line with the multi-agency emphasis of the project. This would provide the capacity to analyse the responses by post, service and sector.

Additionally, questionnaires were sent to the assessment coordinator in each local authority.

Return Rates

The table below shows the number of questionnaires returned from each of the sectors and from the Local Authority Coordinators. Reponses from the primary sector constitute the largest respondent group at 67%.

The low return rate from secondary schools was noticeable. A small number of returns were received from the two participating nursery schools. All non-returning schools were followed up at least once by telephone. Of the 20 primary schools, one reported that it had not continued with its initial involvement in the project.

All local authority coordinators responded, constituting 21% of the total number of respondents.

The small numbers of schools involved and the low return rates in the secondary sector in particular mean that the analysis has to be regarded very cautiously.

2.1.2 Case studies

Case studies were carried out in four clusters of schools. Each of these clusters consisted of a secondary school and a variable number of associated primary schools and nursery schools. Not every school in each cluster was visited. Altogether ten schools were visited, known by the following pseudonyms:

Torrfield HS, Longside PS, Auckland NS
Sandpiper GS, Plover PS, Willow PS
Ryeland RCHS, Fontfield RCPS
Kanmay RCHS, Jenner RCPS

The names of all other institutions and individuals participating in the PLP projects have been anonymised.

These clusters were chosen for the evaluation to provide a spread across different local authorities and a range of different approaches to personal learning planning. It was also important to include at least one cluster which was focusing on the management of personal learning plans (Torrfield cluster).

For each school the fieldwork plan consisted of

• Interviews with school managers responsible for personal learning plans
• Interview/s with teaching staff responsible for implementing personal learning plans
• Focus group/s with pupils involved in the PLP process
• Focus group/s with parents.

Additionally, several local authority co-ordinators were interviewed.

Analysis of documents from each of these clusters and from a wider range of schools was included in the evaluation work.

2.1.2.1 Issues in carrying out case studies

• No school was able to facilitate parents for interview and therefore any reporting of parents' views is second-hand, based on schools' own evaluative work.
• Likewise, no school managers thought it appropriate to arrange interviews with staff drawn from other services, such as Psychological Services, considering that such staff had either a limited or non-existent role in the PLP process.
• The selection of pupils for focus groups with the researchers did not allow fullest representation of the achievement range. Generally researchers talked with pupils of average or above average attainment.
• School timetabling constraints prevented researchers observing the personal learning plan process in a large proportion of the schools visited. Therefore any commentary on process is largely derived from the reflections of young people and teachers, rather than from direct observation.

2.2 Methodology for Phase 2 - The ' Skills for Living' Model of Personal Learning Plans

2.2.1 Case studies were carried out in four of the 10 schools involved in the Skills for Living phase of the project. The case study methodology for this round of data collection replicated that for phase 1; and several of the same issues arose in the conduct of this work.

2.2.2 The original evaluation design had indicated that the questionnaire carried out with all schools in Phase 1 would be re-administered in Phase 2, to provide a comparison of views between the two phases. However, the poor return rate in phase 1 necessitated a different approach. In all but one of the remaining schools, senior staff and/or teaching staff implementing the Skills for Living model were interviewed by telephone. The schedule for this interview paralleled the themes and questions of the questionnaire for phase 1.

2.2.3 Where available, school reports of the work were provided to the researchers by Learning and Teaching Scotland.

« Previous | Contents | Next »