The Scottish Government

« Previous | Contents | Next »

Listen

Chapter 9 - Evaluation of Educational Institutions and the Education System

Compulsory Descriptors
Evaluation, Quality of Education

The sections of this chapter set out in detail the evaluation arrangements applying to all the sectors of Scottish education.

9.1 Historical Overview

Compulsory Descriptors
Historical Perspective

Additional Descriptors (x to left denotes that additional descriptor is covered below)

Scotland has a very long history of external evaluation of its schools. The Act of the Scottish Parliament For Settling of Schools in 1696 laid the duty on each of the presbyteries of the church to inspect the parish schools which had been established. The first Government appointment of an Inspector was in 1840 and, in the years since then, HM Inspectors of Schools (now, since 1 April 2001, HM Inspectorate of Education) have grown in numbers, and inspection procedures have been regularly updated and developed.

Until quite recently, however, internal school and further education college self-evaluation had not been formalised, although over the years many head teachers, particularly in secondary schools, have used results in external examinations to measure the effectiveness of individual teachers and subject departments. The SEED (through HM Inspectorate of Education) has produced a pack entitled: 'How Good Is Our School?', which schools are encouraged to use to help them evaluate their work. A revised and updated version of this was published in 2002. The pack of materials, produced by the former Quality, Standards and Audit Division of HM Inspectorate of Education, proposes questions which schools should address in evaluating their own performance and offers a set of quality indicators against which they can judge their work. These indicators are the same as those employed by HMIE in inspecting schools.

HMIE initiated the process of systematic internal and external quality assurance in further education colleges by issuing a document in 1998 entitled: Quality Matters. Responsibility for quality assurance in FE colleges passed to the Scottish Further Education Funding Council ( SFEFC) on 1 July 1999. On behalf of the SFEFC, and building on earlier quality assurance frameworks, HMIE published a revised quality assurance framework in May 2004. This framework "Standards and Quality in Further Education: Quality Framework for Scottish FE Colleges" is used for both internal self-evaluation and external review.

9.2 Ongoing Debates

Compulsory Descriptors
Reform Proposal

The school inspection model has recently undergone major revision to focus it more clearly on the school's success in raising achievement and promoting educational inclusion, whilst also assessing its capacity for self-evaluation and ensuring continuous improvement. HMIE and education authorities are currently collaborating to find the best means of ensuring that schools receive follow-up support after inspections to help them effect improvements. (See also section 9.4.2 under "Inspection of Schools")

9.3 Administrative and Legislative Framework

Compulsory Descriptors
Educational Legislation

Under the Education (Scotland) Act 1980, HMI have the right to enter schools, including independent (non-state) schools, for the purpose of inspection. Under the Standards in Scotland's Schools etc Act 2000, HMIE inspects the education functions of all local authorities in a five year cycle. The first cycle will be completed by the summer of 2005. HMIE also works collaboratively with key stakeholders in developing quality assurance procedures for initial teacher education. Under the terms of the Teaching and Higher Education Act 1998, the Minister may request HMIE to undertake aspect reviews of initial teacher education.

HMIE also undertakes reviews of colleges of further education in a 4-year cycle, through a service agreement with the Scottish Further Education Funding Council ( SFEFC). HMIE is also empowered under the Education (Scotland) Act 1980 to inspect the further education provision of local authorities. The scope of inspections of Community Learning and Development ( CLD) is set out in Working and Learning Together to Build Stronger Communities. HM Inspectors evaluate the quality of local authority CLD services and may also evaluate the contribution of other publicly funded CLD partners.

Under the Regulation of Care (Scotland) Act, 2001, HMIE collaborates with the Care Commission in the registration and inspection of care services which incorporate an educational element. In pre-school settings these services include local authority nursery schools (and nursery classes within primary schools), nursery classes within independent primary schools and private and voluntary nurseries and playgroups which receive funds in partnership with local authorities. For older children they include residential special schools and secure units.

9.4 Evaluation of Schools/Institutions

Compulsory Descriptors
Educational Institution, Organising Body

Additional Descriptors (x to left denotes that additional descriptor is covered below)

Three types of evaluation are applied to the work of schools: internal school self-evaluation,

evaluation by the local education authority and inspection by HMIE. Further education,

higher education institutions and education authorities are also expected to be self-evaluative and
are subject to forms of external evaluation. The following sections explain the various systems.

9.4.1 Internal Evaluation

Compulsory Descriptors
Self-evaluation

Additional Descriptors (x to left denotes that additional descriptor is covered below)

At institutional level, the schools and further education colleges are responsible themselves for monitoring and evaluating their performance and progress and they are required to produce both an annual Standards and Quality or self-evaluation report on their own work and a development plan, setting out the results of an internal audit and their plans and objectives for the future. The Scottish Executive Education Department and the Enterprise, Transport and Lifelong Learning Department of the Scottish Executive have published guidance to assist the schools and FE colleges respectively in carrying out self-evaluation, using performance or quality indicators, in preparing development plans and using examination results and other data in the process.

In particular, the SEED/ HMIE publication, How Good Is Our School?, is widely used by schools and education authorities. In further education the Scottish Office Education and Industry Department (now the Enterprise, Transport and Lifelong Learning Department) gave guidance to FE colleges to help them prepare development plans until March 1999. For the 2000-2001 planning round and thereafter the SFEFC provided guidance to the colleges on the preparation of strategic and operational plans. These plans form the basis for both forward planning and evaluating quality improvement.

9.4.1.1 Internal Evaluation at Pre-School and Primary Levels

Additional Descriptors (x to left denotes that additional descriptor is covered below)

Pre-school establishments and primary schools, like other educational establishments, are required to produce development plans which state their overall aims, give the results of internal audit and indicate their improvement objectives. The aim is to ensure that by setting common targets, with deadlines, the staff agree to a common view of where the nursery or primary school should be going. The process is likely to be carried forward in staff meetings under the direct leadership of the head teacher, whose role in involving members of staff is crucial. National guidance is offered in carrying out the necessary internal audit in the form of sets of quality indicators which have been developed by HM Inspectorate of Education and set out in the documents, The Child at the Centre and How Good Is Our School?, for pre-school and primary education respectively. Pre-school establishments and primary schools are also encouraged to devise ways of monitoring and evaluating their own progress in carrying out the plans which they have made.

9.4.1.2 Internal Evaluation at Secondary Level

Additional Descriptors (x to left denotes that additional descriptor is covered below)

Secondary schools, like other educational establishments, are required to carry out an internal audit and produce a development plan, setting out their aims and their improvement objectives. This process involves all staff and all departments and therefore considerable consultation is required before the development plan can be finalised. National guidance (How Good Is Our School?, referred to above) has been provided for schools to help them in their internal audit and also in evaluating their own progress in implementing their plan. Guidelines have also been published on the use of examination results in school evaluation.

Monitoring progress in carrying out the plan in a secondary school requires a wide variety of techniques. Among those currently in use are review teams, questionnaires, checklists, interviews, team teaching, classroom observation, shadowing pupil as they work in various subjects, systematic planning and reporting, discussion groups and observation by external agencies.

9.4.1.3 Internal Evaluation in Further Education Colleges

Additional Descriptors (x to left denotes that additional descriptor is covered below)

Further education colleges are required to produce strategic plans setting out their aims, objectives and proposed actions for the following three academic years in terms of finance, student numbers and educational targets. These are approved by their Boards of Management and submitted to the SFEFC. The SFEFC considers the colleges' strategic plans and then issues guidance, including a list of planning assumptions to help the colleges prepare one-year operational plans. The publication Quality Matters provided national guidance for the FE colleges to help them with internal review and in evaluating progress in meeting targets for improvement identified in strategic and operational plans. Quality Matters was replaced, following the advent of SFEFC, by the "Specification for the Review of Standards and Quality in FE ( SFEFC/ HMI July 2001). This Specification has subsequently been replaced by "Standards and Quality in Scottish Further Education colleges in Academic Years 2004-05 to 2007-08: National Briefing document (August 2003)", introduced as part of the two-phase proportionate review model, which began in January 2005.

9.4.1.4 Internal Evaluation in Higher Education Institutions

Additional Descriptors (x to left denotes that additional descriptor is covered below)

The system of quality assurance in higher education institutions since April 1993 has been the responsibility of the Scottish Higher Education Funding Council ( SHEFC). Apart from a small involvement by HM Inspectorate of Education in teacher education courses, neither the Education Department nor the Enterprise, Transport and Lifelong Learning Department of the Scottish Executive plays any part in the system. Previously, the SHEFC carried out a series of quality assessments on various subjects in higher education using a team of their own experts to validate the judgement which individual subject departments within institutions have made of their own efficiency and effectiveness. Currently, however, the UK Quality Assurance Agency ( QAA) for Higher Education undertakes on its behalf the assessment of the quality assurance arrangements in Scottish higher education institutions.

9.4.2 External Evaluation

Compulsory Descriptors
Inspection

Additional Descriptors (x to left denotes that additional descriptor is covered below)

Evaluation at National Level

Evaluation at national level in schools, local authorities (see section 9.5) and further education colleges is carried out by HM Inspectorate of Education ( HMIE), who have devised sets of quality indicators which they use in their work and which they have published (on behalf of the SFEFC in the case of the further education college sector). Evaluation at national level in university level higher education is the responsibility of the Scottish Higher Education Funding Council ( SHEFC) and the Quality Assurance Agency ( QAA) for Higher Education, whose remit for validating quality assurance procedures extends over the whole of the United Kingdom.

Role of HMIE

HMIE in Scotland is an Executive Agency of the Scottish Ministers under the terms of the Scotland Act 1998. Under the Education (Scotland) Act 1980, HMI have the right to enter schools, including independent (non-state) schools, for the purpose of inspection. Under the Standards in Scotland's Schools etc Act 2000, HMIE inspects the education functions of all local authorities in a five year cycle. HMIE also works collaboratively with key stakeholders in developing quality assurance procedures for initial teacher education. Under the terms of the Teaching and Higher Education Act 1998, the Minister may request HMIE to undertake aspect reviews of initial teacher education.

HMIE also undertakes reviews of colleges of further education in a 4-year cycle, through a service agreement with the Scottish Further Education Funding Council ( SFEFC). The scope of inspections of Community Learning and Development ( CLD) is set out in Working and Learning Together to Build Stronger Communities. HM Inspectors evaluate the quality of local authority CLD services and may also evaluate the contribution of other publicly funded CLD partners.

Under the Regulation of Care (Scotland) Act, 2001, HMIE collaborates with the Care Commission in the registration and inspection of care services which incorporate an educational element. In pre-school settings these services include local authority nursery schools (and nursery classes within primary schools), nursery classes within independent primary schools and private and voluntary nurseries and playgroups which receive funds in partnership with local authorities. For older children they include residential special schools and secure units.

HMIE plays a key role in promoting improvements in standards, quality and achievement in Scottish education. It aims to:

• plan and undertake rigorous independent evaluations of educational provision and providers, through an annual programme of inspections and reviews and commissions from Scottish Ministers and others
• promote public accountability by publishing evaluations in clear, concise reports
• identify and promote best practice in improving standards and quality
• monitor standards over time
• provide independent professional advice and information to Scottish Ministers, relevant departments of the Scottish Executive and key national bodies
• provide professional advice and guidance to bodies responsible for the funding, management, quality and delivery of education.

HMIE's evidence and analyses ensure an extensive and thorough overall knowledge of the system. It draws on them in providing the Scottish Ministers, relevant departments of the Scottish Executive and key national bodies with information about current and emerging educational issues and the performance of the system as a whole. It also draws on them in providing professional advice to inform educational developments and assist in policy formulation.

HM Inspectors of Education publish reports on their inspections of pre-school centres, schools, the education functions of local authorities, community learning and development and FE colleges, and on their evaluations of education as a whole. Different kinds of reports are produced, including 'aspect reports', evaluations of the quality of particular aspects of educational provision. In any year, establishments may be visited in addition to those in the main inspection programme to obtain particular evidence for aspect reports. Triennially, a national report on Standards and Quality in Scottish Schools is published. The most recent covered the years 1999-2001. HMIE has published on behalf of the SFEFC 'aspect reports' on key areas in further education, such as quality and standards in FE colleges, improving learning through collaboration, learner representation in quality assurance in FE colleges and moving on from school to college. It has also published papers providing briefing and professional advice for SFEFC Officers.

HMIE's objectives for 2004/05 include inspecting a minimum of 242 primary schools, 52 secondary and 25 special schools. HMI will also carry out a programme of reviews of FE colleges in agreement with SFEFC; inspection of community learning and development in 8 local areas; and of 7 education authorities. HMIE will also undertake a minimum of 750 integrated inspections of pre-school centres and 7 inspections of residential special schools including secure accommodation in collaboration with the Care Commission. Reports are published on these inspections.

The Corporate Services Unit of HMIE

To complement and support the work of inspectors in the field, a specific unit was established in 1992 within the framework of HM Inspectorate. This unit, originally known as the Audit Unit, has now been incorporated within a new Corporate Services Unit ( CSU). The CSU is responsible for gathering and analysing a wide range of factual information on the education system. It publishes the results of such analyses in the form of reports summarising strengths and weaknesses identified through the inspection process and as information for parents about a range of aspects of schools. Its work greatly assists the inspection process by providing essential background information. Much of the current work of the Unit is in support of the Quality Initiative in Scottish Schools ( QISS), which builds on the extensive previous work on the characteristics of effective schools and on school development planning.

Inspection of Schools

To assist them in their task of evaluating the work of schools and other educational institutions HM Inspectors have devised a range of inspection or evaluation instruments. Most importantly, a framework of quality indicators has been established against which the work of establishments in every educational sector can now be measured. For schools these were first published in the document: How Good Is Our School? (1996), with a revised edition published in 2002. How Good Is Our School? presents the framework of quality indicators along with advice on how to build them into a systematic process of school self-evaluation.

To help ensure that stakeholder views are fully taken into account, school inspection teams contain a 'lay member' who is not involved professionally in education. The lay member may not be involved in the evaluation of professional matters but can deal with such areas as relations with parents and employers and what it is like to be a pupil or student. Inspection teams also frequently include at least one 'Associate Assessor'. Associate Assessors are practising teachers, head teachers, college lecturers or managers from colleges or other educational establishments or services, who do contribute directly in the inspection to the evaluation of professional issues.

The school inspection model has recently undergone major revision to focus it more clearly on the school's success in raising achievement and promoting educational inclusion, whilst also assessing its capacity for self-evaluation and ensuring continuous improvement. Chapter headings in the published report include: 'How good are learning, teaching and achievement?', 'How well are pupils supported?', 'How good is the environment for learning?' and 'Improving the school'. Secondary school inspections include a 'closer look' at learning, teaching and attainment in a sample of four selected subject areas.

New 'proportionate' follow-through arrangements are linked to this inspection model. The extent of follow through activity depends upon how effective the school has been found to be and its capacity to improve. HM Inspectors will spend most time monitoring and supporting those schools causing most concern. In other cases they may ask the education authority to take the lead in monitoring and reporting on progress in addressing development needs identified by the inspection. Sometimes they may also re-visit a school to explore further and disseminate very good practice.

Under the terms of Section 35 of The Children (Scotland) Act 1995 or in accordance with the Regulation of Care (Scotland) Act 2001, the Care Commission and HM Inspectorate of Education also carry out inspections relating to the care and welfare of residential pupils in schools which have boarding arrangements, depending on the nature of their provision. Where HMI inspect with the Care Commission, National Care Standards are used alongside quality indicators from How Good is Our School? (2002) or, for pre-school education, The Child at the Centre.

All school inspections survey and consider the views of staff, pupils and parents on the school and these views are recorded in the published report. All reports list key strengths and the main points on which the school has to take action.

Inspection of Education Authorities

HM Inspectors make use of the quality indicators set out in the document, Quality Management in Education (2000), in their inspection of the work of local authority education services. They inspected seven local authority education departments in the school year 2004-2005. All 32 local authority education services must be inspected within a five-year cycle and all inspections are then followed up, normally within two years. A revised approach to inspection of education authorities is being developed for the second cycle of inspection scheduled to begin in 2006.

Reviews of Further Education Colleges

Colleges of further education are also open to review by HM Inspectors of Education, but on the basis of a different set of quality indicators from those used in schools. HMIE makes use of the quality indicators set out on "Standards and Quality in Further Education: Quality Framework for Scottish FE Colleges (May 2004)".in conducting these reviews.

The reviews of colleges of further education take place over a 4-year cycle, through a service level agreement with the Scottish Further Education Funding Council ( SFEFC).

Inspection of Community Learning and Development

HMIE use the publication How Good Is Our Community Learning and Development as the basis of inspection of provision in this area. The document also serves as guidance for self-evaluation by providers of community learning and development.

Evaluation of Higher Education Institutions

The Quality Assurance Agency ( QAA) for Higher Education, a body set up on a UK basis, carries out institution-wide audits of higher education institutions ( HEI), including the universities, to check that quality assurance mechanisms are in place and are working efficiently. It operates in Scotland as well as in other parts of the UK.

Under the terms of the Teaching and Higher Education Act 1998, HMIE, at the request of the Minister of Education, may be asked to carry out inspections of aspects of initial teacher education.

9.5 Evaluation of the Education System

Compulsory Descriptors
Educational System

Additional Descriptors (x to left denotes that additional descriptor is covered below)

Evaluation at National Level

The activities and publications of HMIE, described in the previous sections, provide much of the basis for judgements about the quality of the educational system as a whole. Other factors taken into account are the patterns of pupils/students' performance in Scottish Qualifications Authority national examinations and in the "Assessment of Achievement Programme". The AAP, planned and administered by SEED and involving assessment professionals from outwith the Department, produces evidence of national standards of achievement derived from tests taken by a random national sample of pupils at Primary 3, 5 and 7 and Secondary 2. Currently there tests every four years in English, mathematics, science and social subjects. The future of AAP is under consultation at present.

Evaluation at Local Education Authority Level

The Standards in Scotland's Schools etc Act 2000 requires education authorities to "endeavour to secure improvement in the quality of the school education which is provided in the schools managed by them". The Act set out a framework for establishing national priorities in education, a system of local authority and school planning to meet related national and local objectives and a requirement to report publicly on progress in meeting these objectives. The Act also gave powers to HM Inspectorate of Education to inspect the education functions of all local authorities over a 5-year period.

In meeting duties imposed by the Act local authorities are placing a high priority on self-evaluation and on validation of school self-evaluation. Authorities had already begun to develop systems of self-evaluation prior to the Act, but in ways and to standards which differed across the country. As part of the process of inspection of education in local authorities, and in partnership with local authorities and Audit Scotland, HM Inspectorate of Education developed a set of quality indicators, set out in the document: Quality Management in Education, which authorities can also use in self-evaluation. This framework, used in conjunction with performance data, was designed to help to promote rigour and consistency in self-evaluation at local authority level. The indicators also integrate the processes of Best Value, a Scottish Executive initiative for audit and continuous improvement applying to the public sector as a whole. With the current cycle of inspections of education authorities almost completed, HM Inspectors are working with education authorities and Audit Scotland to develop a more proportionate form of inspection with a strong emphasis on self-evaluation. The revision of Quality Management in Education includes revising, updating and extending the quality indicators to provide a 'toolkit' for education authorities to use in self-evaluation and HM Inspectors in inspections.

9.6 Research on Education

Compulsory Descriptors
Educational Research

Additional Descriptors (x to left denotes that additional descriptor is covered below)

The Analytical Services Units for Schools and for Children and Young People of the Scottish Executive Education Department and the Analytical Services Division of the Scottish Executive Enterprise, Transport and Lifelong Learning Department commission and fund a range of research projects, many of which provide evaluations of the education system or of innovations and initiatives within it. Details of current projects are available on the Scottish Executive Education Department website. All education publications can be found in the publications archive section of the SE web site: http://www.scotland.gov.uk/Publications/Search

The enquirer should select 'education' from the drop-down menu under 'topic' and this brings up all recent education publications categorised by whether they refer to research, consultation, statistics, etc.

Information specifically about recent, current and planned Education Department Research can be found at: http://www.scotland.gov.uk/Topics/Statistics/14478/218

This includes the research programmes for each of the Policy Groups within the Education Department, and also includes a list of all recent publications.

9.7 Statistics

Compulsory Descriptors
Statistical Data

Additional Descriptors (x to left denotes that additional descriptor is covered below)

Inspection Statistics

There are currently 95 HM Inspectors who are mainly full-time and who carry out inspections of schools, colleges and education authorities. There are approximately 320 Associate Assessors working with the Inspectorate.

Background statistical information on the educational context and attainment of secondary schools and education authorities is made available on the HMIE extranet. This information is available for use not only by HMII during inspections, but also schools and education authorities as part of their own monitoring and self-evaluation. Information on attainment of similar schools and authorities is also provided in order to help with benchmarking.

Questionnaires evaluating the inspection process are sent to the various stakeholders after an inspection. Results from these questionnaires are fed into the HMIE quality groups with a view to monitoring and continually improving the process. Summaries of these questionnaires are published in the Annual Report.

« Previous | Contents | Next »