The Scottish Government

Next »

Listen

INSPECTION OF THE EDUCATION FUNCTIONS OF LOCAL AUTHORITIES
ANNUAL REVIEW (2003-2004)

This document is also available in pdf format (252k)

1. BACKGROUND

The Standards in Scotland's Schools Etc. Act 2000 placed a duty on education authorities "to endeavour to secure improvement in the quality of school education which is provided in the schools managed by them". On behalf of Scottish Ministers Section 9 of the Act provided for the inspection of education authorities by HMIE or any person appointed by Scottish Ministers for this purpose to review the way in which authorities were exercising their functions in the provision of school education. In practice for all inspections HMIE has worked in close partnership with Audit Scotland on behalf of the Accounts Commission for Scotland.

The Act also provided for a Code of Practice to be drawn up setting out how Scottish Ministers intended to use their powers under Section 9 and to ensure that the framework for inspection was fully transparent. The Code was the subject of consultation with local authorities and was designed to ensure inspections were independent, open, fair and responsive to the aims and priorities of authorities. The code provided Key Principles on the Rationale, Purpose, Commissioning, Process, Reporting, Follow-up and Review of Inspections, including:

• rigorous independent external evaluation informed by the best practice that exists and which was fully consistent with the principles of Best Value;
• all local authorities to be inspected within an initial period of five years;
• each authority to have at least two months' notice of inspection;
• inspections to be carried out by HMIE in partnership with other relevant agencies and including on each inspection team at least one official (Associate Assessor) working at a senior level in a different authority to the authority being inspected;
• evaluations within inspections to be based on published performance indicators prepared by HMIE in consultation and discussion with authorities and partner agencies;
• the role of Scottish Ministers in commissioning, from time to time, a special inspection of any aspect of the educational functions of a local authority and thematic inspections relating to a particular aspect of the management of the educational functions of a local authority or a number of local authorities;
• inspection to be based on the self-evaluation of the education authority and the views of a wide range of stakeholders, with inspection procedures publicly available and accessible on the Internet;
• a published report of the outcomes of each inspection to be made to Ministers and to be discussed in draft with education managers in the authority with copies of the published report to be widely distributed and available; and
• an action plan to be drawn up by the education authority within eight weeks of publication of the report and an inspection team to revisit the authority to evaluate progress on the main points for action within two years of the report's publication.

The Code of Practice further provided for a review panel, chaired by the Secretary of the Scottish Executive Education Department, to meet at least annually to consider and review the inspections of authorities during the year and to make any general comments and recommendations about the process of inspection.

2. THE INSPECTION PROGRAMME

Inspections undertaken before June 2000

• Prior to the Standards in Scotland's Schools Etc. Act 2000 providing for the statutory inspection of the education functions of local authorities, HMIE undertook the inspections of three authorities at their own request (East Renfrewshire; West Lothian; and Aberdeenshire). These inspections resulted in published reports in the period 1998-2000 and allowed the principles and procedures for inspection to be piloted and revised as necessary.

Inspections undertaken August 2000 to February 2002

• From August 2000 to February 2002 eight authorities were inspected under the statutory provision set out by Section 9 of the Act. Criteria for the selection of education authorities for inspection were drawn up to provide a sampling frame which allows HMIE to gain an overview of quality management in different types of authorities as quickly as possible. The planned inspection programme for the five year cycle aimed to achieve a broadly balanced sample of authorities each year based primarily on characteristics of location, population, socio-economic factors and performance.
• In the period from August 2000 to February 2002 the inspected sample comprised - Highland, East Dunbartonshire, Dundee City, Midlothian, East Lothian, Inverclyde, Shetland Islands, North Ayrshire, Fife, Angus and Glasgow City. The first seven authorities in this list had reports published within that period. Inspections of North Ayrshire, Fife, Angus and Glasgow City had been completed and reports were published in March/May 2002. The first seven inspections were the subject of discussion at the first INEA review panel which met in March - May 2002.
• Up to February 2002 follow-up inspections had been undertaken for two of the authorities (East Renfrewshire; West Lothian) which had requested inspections before the statutory provision was in place.

Inspections undertaken February 2002 to June 2003

• In the period from February 2002 to June 2003 a further eight authorities were inspected and reports published. The sample comprised Scottish Borders, South Ayrshire, Western Isles, North Lanarkshire, East Ayrshire, City of Edinburgh, Perth and Kinross and West Dunbartonshire.
• The twentieth authority to be inspected, Clackmannanshire, started in April 2003 and the report was published in September 2003
• In the period from February 2002 to June 2003 a follow-up report for Aberdeenshire, which had also requested inspection before the statutory provision was in place, had been published. Interim follow-up reports, one year after the publication of the inspection report, have been published on East Dunbartonshire and Shetland Islands. Full follow-up inspections of Highland and East Dunbartonshire were undertaken and reports were published in May 2003. Midlothian and Dundee had follow-up inspections in the period up to June 2003 and their reports were published in August 2003.
• All of the inspections and follow-up inspections with reports published by August 2003 were considered by the second INEA review panel during the period September to November 2003.

Inspections undertaken August 2003 to June 2004

• During the period August 2003 to June 2004 a further seven authorities were inspected. These were Aberdeen City, Moray, South Lanarkshire, Renfrewshire, Falkirk, Stirling and West Lothian.
• Reports on these seven inspections were published by September 2004.
• During this period, interim follow-up reports were published on Scottish Borders and Western Isles. Full follow-up inspections of East Lothian, Inverclyde, Shetland, North Ayrshire, Fife, Angus, Glasgow and South Ayrshire were also carried out. Reports on the first seven were published by September 2004.
• Plans for 2004-2005 allow for the inspection of the remaining five education authorities. HMIE will therefore have completed the inspection of all authorities as planned in the five year period 2000-June 2005.

The inspection process

Inspections undertaken since August continue to have been conducted in line with the guidelines developed by HMIE in consultation with local authorities, Audit Scotland and SEED. Inspection teams have continued to use the Quality Indicator framework (QMIE) which was developed in co-operation with representatives from authorities.

Each inspection team has included, in addition to HMI, an auditor nominated by Audit Scotland and an Associate Assessor drawn from current senior officers of authorities. In the inspections of Perth and Kinross, Falkirk and Stirling councils, the teams also included a member of the Social Work Services Inspectorate (SWSI). An officer from Communities Scotland participated in the inspection of Clackmannanshire Council. These inspections provided valuable additional insights into the scope for the development of more integrated inspections in the next cycle of inspection.

3. EVALUATION OF INSPECTIONS

Local authority evaluations

Post-inspection evaluation questionnaire returns from education authorities are shown in Appendix 1. Figure 1 shows the returns from 2003-2004 while Figure 2 shows all returns to date. Local authority evaluations have been largely positive. For 12 of the 15 aspects surveyed, most authorities gave positive responses. Most authorities indicated that the inspection had been helpful, and none expressed a negative view of this aspect. All indicated that oral briefings were helpful. Most authorities were positive about pre-inspection oral and written briefings, the coverage, coordination and efficiency of the inspection, the quality of feedback, and the inclusion of Associate Assessors and Audit Scotland members on the inspection team. All of the responses in 2003-2004 were positive about the helpfulness and efficiency of the inspection, the quality of feedback, and the coordination of activities. Some concerns were expressed about the timing of the inspection and the range of work covered. A majority were concerned about the demands on staff in providing pre-inspection information, but in 2003-2004 most of the responses were positive about requirements regarding pre-inspection information.

Audit Scotland evaluations

In 2003-2004, four post-inspection evaluation questionnaires have been completed by auditors ( see Appendix 2, Figure 1). Figure 2 shows evaluations returned from all the inspections to date. Auditors' evaluations of the inspection process continue to be positive overall. Evaluations were more positive in 2003-2004 than in the previous year, especially in relation to the usefulness of pre-inspection information, the degree to which they were integrated into inspection teams, communication during the inspection, and the extent to which the inspection was able to integrate Best Value audit. Views on how well auditors had been able to manage inspection activity within the allocated time were more mixed than previously. Audit Scotland is currently involved in discussions on the future development of INEA.

Associate Assessor evaluations

Post-inspection evaluation questionnaires from Associate Assessors are shown in Appendix 3. Figure 1 shows evaluations from 2003-2004, and Figure 2 shows evaluations returned from all inspections to date. Overall, Associate Assessors continue to rate the inspection process very highly, especially in terms of their involvement in the team and the use of their professional expertise. For each aspect surveyed, over 70% of the responses were good or very good. Some Assessors were less positive about time allocation, recording evidence and administrative arrangements. These features will be reviewed as proposals for the next cycle of inspections are developed. The involvement of Associate Assessors continues to be an important contributor to the general acceptance of self-evaluation and the education authority inspection process within the system.

HMI evaluations

Evaluations were obtained through questionnaires completed by each Reporting Officer and through review meetings of the HMIE inspection network. HMI reported that:

• inspection procedures had enabled the inspection team to evaluate all aspects successfully;
• inspection guidelines were helpful but needed updating and more detail in places;
• questionnaire survey information, statistical data and DI briefings had been helpful in informing the inspection; and
• Associate Assessors and Audit Scotland members of the team made effective, sometimes very effective contributions to the inspection.

HMI continued to express concerns about time allocations for the inspection. A slight increase in time has recently been allocated to the team.

4. KEY FINDINGS FROM INSPECTIONS

Reporting against the quality indicators

Analyses of the judgements made by inspection teams against the Quality Indicators are based on the eight published reports since the last review and on all 27 published reports up to June 2004 ( see Appendix 4, figures 1, 2 and 3).

The breakdown of judgements made against the quality indicators in QMIE in 2003-2004 ( see Appendix 4, figure 1) shows that 42% of evaluations were given as very good and 33% were given as good. In almost a quarter (24%) of the evaluations a judgement of fair was given and in only 1% a judgement of unsatisfactory. The breakdown of judgements over all inspections to date (2000-2004) shows that 31% were given as very good, 43% as good, 22% as fair and 4% as unsatisfactory.

The distribution of judgements given to each council is shown in Appendix 4, figure 2. The graph shows continuing variation between councils. While all councils of the seven councils inspected since the last INEA review had some good features, four were judged to have very good performance in most or almost all aspects of their work while three councils had only fair performance in a majority of aspects. Only one of the seven councils was found to be unsatisfactory in any elements of the inspection.

Appendix 4, figure 3 illustrates the way in which judgements were made across all 27 inspections for each of the Quality Indicators. The data allows for an initial analysis of strengths and weaknesses across those councils inspected to date. A number of these are noted below:

• Vision values and aims, communication and service planning were found to be good or very good in twenty-two cases.
• Policy development was reported to be good or very good in twenty-one councils.
• There were weaknesses reported in over a third of councils in measuring, monitoring and evaluating performance and in the effectiveness of central staff.
• Effectiveness of leadership and management was found to be fair or unsatisfactory in one third of the councils inspected.
• Only one of the inspected councils was judged to be very good in ensuring continuous improvement.

Changes in evaluations from previous years are likely to be the result of features of the individual councils inspected rather than evidence of a trend of changing performance. However, it is notable that in 2003-2004, fewer aspects were judged to be unsatisfactory and the evaluations of resource and financial management are more positive with half of the inspected councils judged to be very good in these areas.

The impact of inspection on the work of authorities

There are continuing signs that the inspection of education authorities has resulted in a greater emphasis on the support and challenge of educational establishments by council officers. Authorities have evaluated their own performance by applying the set of quality indicators used in the inspection process. Many authorities have developed sound processes and procedures across the areas covered in the inspection programme, and the initial evidence suggests that the strongest performing councils are adding value to the work of schools, often breaking new ground and making a real difference to the educational experience of young people.

Published reports on the inspections of education authorities are often studied by officers in other councils in order to identify areas of good practice and suggestions for improvement which could benefit provision in their own authorities. However, not all authorities are used to benchmarking their performance against comparator authorities and best practice. To assist this process and to support further improvement to the quality of education being provided by authorities across Scotland, HMIE issued a report on best practice identified in the first 16 education authority inspections. The report was launched in February 2004 at a conference to which representatives from all authorities were invited. The report indicated that there was considerable good practice in the work of education authorities across Scotland. The most effective authorities were adding value through a positive impact on pupils' learning and achievements. However, the report also highlighted significant variation across authorities and between different aspects of their functions. Not all of the procedures and arrangements established by authorities have a clear enough positive impact on young people and their families. The report suggested that there remained considerable scope for improvement in the country as a whole and within each authority - even the most effective.

There is encouraging evidence in many authorities of real improvement in response to weaknesses identified by inspection. Authorities have made good or very good progress towards meeting almost all of the main points for action set out in inspection reports. For half of the main points for action progress was very good. Follow-up inspection reports indicate that many types of council have significantly improved aspects of their provision since the publication of the initial inspection report. There are examples of good evidence of increased capacity to add value and support improvement, and of positive impact on areas such as attainment. A number of councils have continued to provide effective support while further developing their capacity to add value. Some others, which had been less effective previously, have undertaken significant change and are in a much better position to deliver their objectives. There are also, however, a small number of aspects where sufficient improvement had not taken place. In these cases, HMIE will continue to monitor progress. In some instances authorities have been asked to provide reports on progress on specific areas and HMIE has indicated that further inspection visits might be conducted.

5. DEVELOPMENTS RELATING TO THE FIRST CYCLE OF INSPECTION

As a result of the review held in 2003, the Review Panel made eight recommendations for the remaining inspections in the first cycle of INEA. These were:

1. Improve early dialogue between HMIE and the authority.
2. Reduce the burden of preparing pre-inspection material.
3. Further develop the role of non-HMIE personnel in inspections.
4. Continue to focus on consistency of inspections.
5. Clarify the relationship between self and external evaluation.
6. Improve opportunities for dialogue at the pre-finalisation stage.
7. Improve accessibility, usage and impact of INEA reports.
8. Clarify purpose etc of follow-up inspections.

The response to each of the recommendations is set out in the following table.

6. PLANNING FOR THE SECOND CYCLE OF INSPECTION

The current cycle of INEA inspections is due to be concluded by June 2005. A development group with broad membership has been considering proposals of the future inspection model beyond 2005. The work of the group has included revision of the QMIE Framework of Quality Indicators.

Following the review of 2003, the Review Panel identified eight issues for consideration in planning for the second cycle of inspection. These were:

1. Partnership and consultation.
2. Focus on impact and the improvement framework.
3. Scope and timing.
4. Proportionate inspection.
5. The role of self-evaluation.
6. Quality indicators.
7. Stakeholder involvement and views.
8. Accessibility of reports.

The development group has given careful attention to these issues in its deliberations.

Partnership and consultation

There has been considerable joint working and consultation in preparing proposals. The development group comprises HMI, senior officers from three education authorities, health and social work professionals, and an independent education consultant. There have been numerous discussions with representatives of Audit Scotland and SWSI. Recently, links have been established with the new Services for Children Unit which is currently being set up by HMIE. This is providing contacts with a wide range of other agencies.

A major consultation conference on initial proposals was held in February 2004. This was attended by senior directorate staff from throughout Scotland. Responses were analysed by the independent consultant and have been given full consideration by the development group. Consultation has also taken place through HMIE Directorate 5 external reference group and through individual meetings and discussion with education directorate staff. A further major consultation conference will be held in December 2004.

Focus on impact and the improvement framework

A key principle of the development group has been to increase the emphasis on impact and outcomes. Proposals for revision of the QMIE framework include the addition of a wider range of outcome measures. These will be linked closely to National Priorities for Education and to the Scottish Executive vision for children and young people. Proposals for the inspection process include arrangements to make fuller use of evidence from other inspections including those of schools and CLD in planning INEA inspections. It is intended that the consideration given to processes would be reduced and would be proportionate to need as indicated by impact and outcomes.

Scope and timing

Thinking on the scope of the next cycle of inspections has developed in light of wider development relating to integrated inspection of services for children. The development group has been adjusting its proposals with the aim of ensuring that they:

• will be compatible with broader inspection of Services for Children; and
• take account of developments relating to Child Protection and Integrated Community Schools.

Options currently being considered by the group include:

• a three-year cycle of proportionate inspection to replace the current approach of a five year cycle of inspection and follow-up; or
• a proportionate approach to timing where the frequency of inspection would vary according to need. The likely time between inspections could vary from three to five years.

Proportionate inspection

A proportionate approach to INEA inspections is being developed. The first phase of inspection activity will be to gather together evidence from a wide variety of sources (including previous inspection evidence and statistical information) and use this to determine the scope of each inspection. It is likely that the issues covered, the indicators used and the number of days allocated to inspections will vary significantly from council to council as a result of the scoping exercise. Detailed proposals for scoping to ensure objectivity and transparency are being developed. There will be discussion with authorities on the outcomes of the scoping and the detailed arrangements for the inspection.

The role of self-evaluation

Self-evaluation will be a key part of the scoping activity. Education authorities will be invited to provide evaluations of performance in key aspects and to identify improvement trends and issues which are main priority areas for the authority. Education authority officers on the development team are playing a lead role in devising arrangements for self-evaluation. It is proposed that self-evaluation guides and training material will be produced to support authorities in this exercise.

Quality indicators

There has been a thorough review of the current quality indicator framework. Key principles have been to ensure that the suite of indicators more fully reflects the functions of the council and focuses more clearly on impact and outcomes. Attention has also been given to wider issues relating to services for children to ensure that indicators are suitably generic so that they can be applied in a broad range of contexts. The proposed set of indictors will be available for use in self-evaluation and in inspection. It is intended that only an appropriate selection of indicators would be used by inspection teams. This selection would be determined by the scoping process.

Stakeholder involvement and views

Consideration is being given to the range and scope of questionnaires used to survey stakeholders' views. It is likely that surveys of stakeholders will be widened to measure the perceptions of learners, staff and the wider community. The extent to which parents' and pupils' views could be gathered during school inspections is also being explored. This could be an appropriate way of involving lay members in INEA inspection.

Accessibility of reports

The format of reports will be thoroughly reviewed. One option being explored is the production of different formats for different audiences. Users' views are being sought as part of the 2003-2004 review.

Appendix 1 Chief Executive Questionnaire - Summary of Responses
Appendix 2 Audit Scotland Evaluation Questionnaire - Summary of Responses
Appendix 3 AA Evaluation Questionnaire - Summary of Responses
Appendix 4

Next »