The Scottish Government

The sections of this chapter set out in detail the evaluation arrangements applying to all the sectors of Scottish education.

x

Scotland has a very long history of external evaluation of its schools. The Act of the Scottish Parliament For Settling of Schools in 1696 laid the duty on each of the presbyteries of the church to inspect the parish schools which had been established. The first Government appointment of an Inspector was in 1840 and, in the years since then, HM Inspectors of Schools (now, since 1 April 2001, HM Inspectorate of Education) have grown in numbers, and inspection procedures have been regularly updated and developed.

Until quite recently, however, internal school and further education college self-evaluation had not been formalised, although over the years many head teachers, particularly in secondary schools, have used results in external examinations to measure the effectiveness of individual teachers and subject departments. The SEED (through HM Inspectorate of Education) has produced a pack entitled: 'How Good Is Our School?', which schools are encouraged to use to help them evaluate their work. A revised and updated version of this was published in 2002. The pack of materials, produced by the former Quality, Standards and Audit Division of HM Inspectorate of Education, proposes questions which schools should address in evaluating their own performance and offers a set of quality indicators against which they can judge their work. These indicators are the same as those employed by HMIE in inspecting schools.

HMIE initiated the process of systematic internal and external quality assurance in further education colleges by issuing a document in 1998 entitled: Quality Matters. Responsibility for quality assurance in FE colleges passed to the Scottish Further Education Funding Council (SFEFC) on 1 July 1999. On behalf of the SFEFC, HMIE published their new Quality Assurance Framework in July 2000, making some minor adjustments subsequently in July 2001. This framework, now called Specification for the Review of Standards and Quality in Further Education, is used for both internal self-evaluation and external review.

The school inspection model has recently undergone major revision to focus it more clearly on the school's success in raising achievement and promoting educational inclusion, whilst also assessing its capacity for self-evaluation and ensuring continuous improvement. HMIE and education authorities are currently collaborating to find the best means of ensuring that schools receive follow-up support after inspections to help them effect improvements. ( See also section 9.4.2 under "Inspection of Schools")

Under the Education (Scotland) Act 1980, HMI have the right to enter schools, including independent (non-state) schools, for the purpose of inspection. Under the Standards in Scotland's Schools etc Act 2000, HMIE inspects the education functions of all local authorities in a five year cycle. HMIE also works collaboratively with key stakeholders in developing quality assurance procedures for initial teacher education. Under the terms of the Teaching and Higher Education Act 1998, the Minister may request HMIE to undertake aspect reviews of initial teacher education.

HMIE also undertakes reviews of colleges of further education in a 4-year cycle, through a service agreement with the Scottish Further Education Funding Council (SFEFC). HMIE is also empowered under the Education (Scotland) Act 1980 to inspect the further education provision of local authorities. The scope of inspections of Community Learning and Development (CLD) is set out in Working and Learning Together to Build Stronger Communities. HM Inspectors evaluate the quality of local authority CLD services and may also evaluate the contribution of other publicly funded CLD partners.

Under the Regulation of Care (Scotland) Act, 2001, HMIE collaborates with the Care Commission in the registration and inspection of care services which incorporate an educational element. In pre-school settings these services include local authority nursery schools (and nursery classes within primary schools), nursery classes within independent primary schools and private and voluntary nurseries and playgroups which receive funds in partnership with local authorities. For older children they include residential special schools and secure units.

x

x

Three types of evaluation are applied to the work of schools: internal school self-evaluation, evaluation by the local education authority and inspection by HMIE. Further education and higher education institutions are also expected to be self-evaluative and are subject to forms of external evaluation. The following sections explain the various systems.

x

x

x

At institutional level, the schools and further education colleges are responsible themselves for monitoring and evaluating their performance and progress and they are required to produce both an annual Standards and Quality or self-evaluation report on their own work and a development plan, setting out the results of an internal audit and their plans and objectives for the future. The Scottish Executive Education Department and the Enterprise, Transport and Lifelong Learning Department of the Scottish Executive have published guidance to assist the schools and FE colleges respectively in carrying out self-evaluation, using performance or quality indicators, in preparing development plans and using examination results and other data in the process.

In particular, the SEED/HMIE publication, How Good Is Our School?, is widely used by schools and education authorities. In further education the Scottish Office Education and Industry Department (now the Enterprise, Transport and Lifelong Learning Department) gave guidance to FE colleges to help them prepare development plans until March 1999. For the 2000-2001 planning round and thereafter the SFEFC provided guidance to the colleges on the preparation of strategic and operational plans. These plans form the basis for both forward planning and evaluating quality improvement.

9.4.1.1 Internal Evaluation at Pre-School and Primary Levels

Additional Descriptors (x to left denotes that additional descriptor is covered below)

Pre-school establishments and primary schools, like other educational establishments, are required to produce development plans which state their overall aims, give the results of internal audit and indicate their improvement objectives. The aim is to ensure that by setting common targets, with deadlines, the staff agree to a common view of where the nursery or primary school should be going. The process is likely to be carried forward in staff meetings under the direct leadership of the head teacher, whose role in involving members of staff is crucial. National guidance is offered in carrying out the necessary internal audit in the form of sets of quality indicators which have been developed by HM Inspectorate of Education and set out in the documents, The Child at the Centre and How Good Is Our School?, for pre-school and primary education respectively. Pre-school establishments and primary schools are also encouraged to devise ways of monitoring and evaluating their own progress in carrying out the plans which they have made.

9.4.1.2 Internal Evaluation at Secondary Level

Additional Descriptors (x to left denotes that additional descriptor is covered below)

Secondary schools, like other educational establishments, are required to carry out an internal audit and produce a development plan, setting out their aims and their improvement objectives. This process involves all staff and all departments and therefore considerable consultation is required before the development plan can be finalised. National guidance (How Good Is Our School?, referred to above) has been provided for schools to help them in their internal audit and also in evaluating their own progress in implementing their plan. Guidelines have also been published on the use of examination results in school evaluation.

Monitoring progress in carrying out the plan in a secondary school requires a wide variety of techniques. Among those currently in use are review teams, questionnaires, checklists, interviews, team teaching, classroom observation, shadowing pupil as they work in various subjects, systematic planning and reporting, discussion groups and observation by external agencies.

9.4.1.3 Internal Evaluation in Further Education Colleges

Additional Descriptors (x to left denotes that additional descriptor is covered below)

Further education colleges are required to produce strategic plans setting out their aims, objectives and proposed actions for the following three academic years in terms of finance, student numbers and educational targets. These are approved by their Boards of Management and submitted to the SFEFC. The SFEFC considers the colleges' strategic plans and then issues guidance, including a list of planning assumptions to help the colleges prepare one-year operational plans. The publication Quality Matters provides national guidance for the FE colleges to help them with internal review and in evaluating progress in meeting targets for improvement identified in strategic and operational plans. Quality Matters will be replaced by a new publication in the near future.

9.4.1.4 Internal Evaluation in Higher Education Institutions

Additional Descriptors (x to left denotes that additional descriptor is covered below)

The system of quality assurance in higher education institutions since April 1993 has been the responsibility of the Scottish Higher Education Funding Council (SHEFC). Apart from a small involvement by HM Inspectorate of Education in teacher education courses, neither the Education Department nor the Enterprise, Transport and Lifelong Learning Department of the Scottish Executive plays any part in the system. Previously, the SHEFC carried out a series of quality assessments on various subjects in higher education using a team of their own experts to validate the judgement which individual subject departments within institutions have made of their own efficiency and effectiveness. Currently, however, the UK Quality Assurance Agency (QAA) for Higher Education undertakes on its behalf the assessment of the quality assurance arrangements in Scottish higher education institutions.

x

Evaluation at Local Education Authority Level

The Standards in Scotland's Schools etc Act 2000 requires education authorities to "endeavour to secure improvement in the quality of the school education which is provided in the schools managed by them". The Act set out a framework for establishing national priorities in education, a system of local authority and school planning to meet related national and local objectives and a requirement to report publicly on progress in meeting these objectives. The Act also gave powers to HM Inspectorate of Education to inspect the education functions of all local authorities over a 5-year period.

In meeting duties imposed by the Act local authorities will need to place a high priority on self-evaluation and on validation of school self-evaluation. Authorities had already begun to develop systems of self-evaluation prior to the Act, but in ways and to standards which differed across the country. As part of the process of inspection of education in local authorities, and in partnership with local authorities and Audit Scotland, HM Inspectorate of Education developed a set of quality indicators, set out in the document: Quality Management in Education, which authorities can also use in self-evaluation. This framework, used in conjunction with performance data, should help to promote rigour and consistency in self-evaluation at local authority level. The indicators also integrate the processes of Best Value, a Scottish Executive initiative for audit and continuous improvement applying to the public sector as a whole.

Evaluation at National Level

Evaluation at national level in schools, local authorities and further education colleges is carried out by HM Inspectorate of Education (HMIE), who have devised sets of quality indicators which they use in their work and which they have published (on behalf of the SFEFC in the case of the further education college sector). Evaluation at national level in university level higher education is the responsibility of the Scottish Higher Education Funding Council (SHEFC) and the Quality Assurance Agency (QAA) for Higher Education, whose remit for validating quality assurance procedures extends over the whole of the United Kingdom.

Role of HMIE

HMIE in Scotland is an Executive Agency of the Scottish Ministers under the terms of the Scotland Act 1998. Under the Education (Scotland) Act 1980, HMI have the right to enter schools, including independent (non-state) schools, for the purpose of inspection. Under the Standards in Scotland's Schools etc Act 2000, HMIE inspects the education functions of all local authorities in a five year cycle. HMIE also works collaboratively with key stakeholders in developing quality assurance procedures for initial teacher education. Under the terms of the Teaching and Higher Education Act 1998, the Minister may request HMIE to undertake aspect reviews of initial teacher education.

HMIE also undertakes reviews of colleges of further education in a 4-year cycle, through a service agreement with the Scottish Further Education Funding Council (SFEFC). HMIE is also empowered under the Education (Scotland) Act 1980 to inspect the further education provision of local authorities. The scope of inspections of Community Learning and Development (CLD) is set out in Working and Learning Together to Build Stronger Communities. HM Inspectors evaluate the quality of local authority CLD services and may also evaluate the contribution of other publicly funded CLD partners.

Under the Regulation of Care (Scotland) Act, 2001, HMIE collaborates with the Care Commission in the registration and inspection of care services which incorporate an educational element. In pre-school settings these services include local authority nursery schools (and nursery classes within primary schools), nursery classes within independent primary schools and private and voluntary nurseries and playgroups which receive funds in partnership with local authorities. For older children they include residential special schools and secure units.

HMIE plays a key role in promoting improvements in standards, quality and achievement in Scottish education. It aims to:

• plan and undertake rigorous independent evaluations of educational provision and providers, through an annual programme of inspections and reviews and commissions from Scottish Ministers and others
• promote public accountability by publishing evaluations in clear, concise reports
• identify and promote best practice in improving standards and quality
• monitor standards over time
• provide independent professional advice and information to Scottish Ministers, relevant departments of the Scottish Executive and key national bodies
• provide professional advice and guidance to bodies responsible for the funding, management, quality and delivery of education.

HMIE's evidence and analyses ensure an extensive and thorough overall knowledge of the system. It draws on them in providing the Scottish Ministers, relevant departments of the Scottish Executive and key national bodies with information about current and emerging educational issues and the performance of the system as a whole. It also draws on them in providing professional advice to inform educational developments and assist in policy formulation.

HM Inspectors of Education publish reports on their inspections of pre-school centres, schools, the education functions of local authorities, community learning and development and FE colleges, and on their evaluations of education as a whole. Different kinds of reports are produced, including 'aspect reports', evaluations of the quality of particular aspects of educational provision. In any year, establishments may be visited in addition to those in the main inspection programme to obtain particular evidence for aspect reports. Triennially, a national report on Standards and Quality in Scottish Schools is published. The most recent covered the years 1999-2001. HMIE has published on behalf of the SFEFC 'aspect reports' on key areas in further education, such as quality and standards in FE colleges, improving learning through collaboration, learner representation in quality assurance in FE colleges and moving on from school to college. It has also published papers providing briefing and professional advice for SFEFC Officers.

HMIE's objectives for 2003/04 include inspecting a minimum of 220 primary schools, 43 secondary and 24 special schools. HMI will also carry out a planned inspection and review programme including a minimum of 12 FE college and subject reviews; inspection of community learning and development in 8 local areas; and of 7 education authorities. HMIE will also undertake a minimum of 400 integrated inspections of pre-school centres, two inspections of secure accommodation services and 15 inspections of residential special schools in collaboration with the Care Commission. Reports are published on these inspections.

There are currently 85 full-time HM Inspectors and 9 part-time inspectors, who carry out inspections of schools, colleges and education authorities. Nine full-time Associate Assessors are also deployed in the inspections of pre-school provision and take full responsibility for the inspection process on behalf of HMIE.

The Corporate Services Unit of HMIE

To complement and support the work of inspectors in the field, a specific unit was established in 1992 within the framework of HM Inspectorate. This unit, originally known as the Audit Unit, has now been incorporated within a new Corporate Services Unit (CSU). The CSU is responsible for gathering and analysing a wide range of factual information on the education system. It publishes the results of such analyses in the form of reports summarising strengths and weaknesses identified through the inspection process and as information for parents about a range of aspects of schools. Its work greatly assists the inspection process by providing essential background information. Much of the current work of the Unit is in support of the Quality Initiative in Scottish Schools (QISS), which builds on the extensive previous work on the characteristics of effective schools and on school development planning.

I nspection of Schools

To assist them in their task of evaluating the work of schools and other educational institutions HM Inspectors have devised a range of inspection or evaluation instruments. Most importantly, a framework of quality indicators has been established against which the work of establishments in every educational sector can now be measured. For schools these were first published in the document: How Good Is Our School? (1996), with a revised edition published in 2002. How Good Is Our School? presents the framework of quality indicators along with advice on how to build them into a systematic process of school self-evaluation.

To help ensure that stakeholder views are fully taken into account, school inspection teams contain a 'lay member' who is not involved professionally in education. The lay member may not be involved in the evaluation of professional matters but can deal with such areas as relations with parents and employers and what it is like to be a pupil or student. Inspection teams also frequently include at least one 'Associate Assessor'. Associate Assessors are practising teachers, head teachers, college lecturers or managers from colleges or other educational establishments or services, who do contribute directly in the inspection to the evaluation of professional issues.

The school inspection model has recently undergone major revision to focus it more clearly on the school's success in raising achievement and promoting educational inclusion, whilst also assessing its capacity for self-evaluation and ensuring continuous improvement. Chapter headings in the published report include: 'How good are learning, teaching and achievement?', 'How well are pupils supported?', 'How good is the environment for learning?' and 'Improving the school'. Secondary school inspections include a 'closer look' at learning, teaching and attainment in a sample of four selected subject areas.

New 'proportionate' follow-through arrangements are linked to this inspection model. The extent of follow through activity depends upon how effective the school has been found to be and its capacity to improve. HM Inspectors will spend most time monitoring and supporting those schools causing most concern. In other cases they may ask the education authority to take the lead in monitoring and reporting on progress in addressing development needs identified by the inspection. Sometimes they may also re-visit a school to explore further and disseminate very good practice.

Under the terms of Section 35 of The Children (Scotland) Act 1995 or in accordance with the Regulation of Care (Scotland) Act 2001, the Care Commission and HM Inspectorate of Education also carry out inspections relating to the care and welfare of residential pupils in schools which have boarding arrangements, depending on the nature of their provision. Where HMI inspect with the Care Commission, National Care Standards are used alongside quality indicators from How Good is Our School? (2002) or, for pre-school education, The Child at the Centre.

All school inspections survey and consider the views of staff, pupils and parents on the school and these views are recorded in the published report. All reports list key strengths and the main points on which the school has to take action.

Inspection of Education Authorities

HM Inspectors make use of the quality indicators set out in the document, Quality Management in Education (2000), in their inspection of the work of local authority education services. They inspected seven local authority education departments in the school year 2003 - 2004. All 32 local authority education services must be inspected within a five-year cycle and all inspections are then followed up, normally within two years.

Reviews of Further Education Colleges

Colleges of further education are also open to review by HM Inspectors of Education, but on the basis of a different set of quality indicators from those used in schools. HMIE makes use of the quality indicators set out in Quality Matters in conducting these reviews.

The reviews of colleges of further education take place over a 4-year cycle, through a service level agreement with the Scottish Further Education Funding Council (SFEFC).

Inspection of Community Learning and Development

HMIE use the publication How Good Is Our Community Learning and Development as the basis of inspection of provision in this area. The document also serves as guidance for self-evaluation by providers of community learning and development.

Evaluation of Higher Education Institutions

The Quality Assurance Agency (QAA) for Higher Education, a body set up on a UK basis, carries out institution-wide audits of higher education institutions (HEI), including the universities, to check that quality assurance mechanisms are in place and are working efficiently. It operates in Scotland as well as in other parts of the UK.

Under the terms of the Teaching and Higher Education Act 1998, HMIE, at the request of the Minister of Education, may be asked to carry out inspections of aspects of initial teacher education.

x

The activities and publications of HMIE, described in the previous sections, provide much of the basis for judgements about the quality of the educational system as a whole. Other factors taken into account are the patterns of pupils/students' performance in Scottish Qualifications Authority national examinations and in the "Assessment of Achievement Programme". The AAP, planned and administered by SEED and involving assessment professionals from outwith the Department, produces evidence of national standards of achievement derived from tests taken by a random national sample of pupils at Primary 3, 5 and 7 and Secondary 2. Currently there tests every four years in English, mathematics, science and social subjects. The future of AAP is under consultation at present.

Programmes of educational research in Scotland are funded by the Scottish Executive both through the Education department (for research on children & young people and school education) and the Enterprise, Transport and Lifelong Learning Department (for research on post-school education, training and lifelong learning). Educational research is also carried out by universities and a variety of other research organisations with Scottish Executive funding or with support from research councils and other agencies. In addition, there are a number of organisations, originally set up by government, in which research is an integral part of their work.

The Research, Economic and Corporate Strategy (RECS) Unit, SEED

The Research, Economic and Corporate Strategy (RECS) Unit is a new Unit within the Information, Analysis and Communication Division of SEED, which brings together the Scottish Executive Education Department's analytical services. RECS combines the knowledge and skills of economists and researchers and, by working closely with statisticians in SEED's Education and Children Statistics Unit, aims to provide an integrated analytical service.

RECS's functions include advice on the development, appraisal, monitoring and evaluation of policies, and the management of, and advice on, social and economic research projects. RECS's work contributes to the Education Department's aim to give every child and young person the best possible start in life and enhance everyone's quality of life through culture, sport and the quality of the physical environment. RECS's remit covers school education, children and young people, social work, and tourism, culture and sport.

How Research is Funded

The bulk of the Education Department's research is commissioned on the basis of competitive tendering to address identified priorities. Contracts are awarded to tenders whose proposals are judged as best meeting the specified research objectives and providing best value for money.

Sponsored Research

A small proportion of the Education Department's research budget is reserved to fund ideas which come from the research community. This allows a wider range of research projects to be supported, including early work in fields which might subsequently become Departmental priorities. Sponsored research funds are also used to enable less experienced researchers, including teachers, to be supported and encouraged.

Research Programme

RECS are currently operating a three-year rolling programme of research, based on priorities which are agreed by the Education Department's Research Committee. These have been published as Scottish Executive Education Department Research Priorities 2002-2004. Current priorities include Impact of Pre-School Education and Early Intervention on Attainment; Impact of ICT Initiatives in Pupil Attainment; ICT Competence of Final Year Student Teachers; Teachers' Knowledge Needs of Information and Communication Technology; Reading Strategies; Synthetic Phonics; Extension; Impact of Mainstreaming Pupils with Additional Support Needs; Citizenship amongst Pupils; Effective Drug Education; Parents' Access to Pre-School Childcare Packages; and Creativity in the Context of Cultural Activities.

Other Research and Development Supported by the SEED

The Scottish Executive Education Department (SEED) also contributes to research programmes conducted on a nationwide basis by UK research agencies where a specific Scottish component is appropriate.

A considerable amount of finance (approximately 2 million) is disbursed by the SEED's Staff Development for Effective Teaching programme. This is earmarked for development in the area of teacher support, whether through the production of materials, training packages or training expertise. While this is not, strictly speaking, educational research, it has considerable impact on the effective delivery of teaching in the classroom and on research documentation.

The Research Programme, SEETLLD

A programme of research to support policy development relating to lifelong learning, including further and higher education, vocational education and training and adult literacy and numeracy, is managed by the Analytical Services Division in the Scottish Executive Enterprise, Transport and Lifelong Learning Department (SEETLLD). The programme is focused on a number of themes relating to transitions to the labour market, widening access to FE and HE, participation in adult education and lifelong learning, returns to learning and work force development.

Research Organisations

Educational research is undertaken by Scottish universities and private research organisations. The Scottish Council for Research in Education (SCRE), an independent research organisation, which has recently merged with the University of Glasgow Faculty of Education, tenders for research projects from the SEED and other agencies. A major investment in educational research capacity is being made by the Scottish Executive Education Department (SEED) and the Scottish Higher Education Funding Council (SHEFC) via the Applied Educational Research Scheme.

Learning and Teaching Scotland

In addition to its considerable amount of curriculum development work, for which it received a core grant of 3,155,000 in 2002-03, LTS also carries out or commissions development projects and provides on-line service for the National Grid for Learning Scotland. The budget for development of digital content and provision of web services for 2003-2004 is 3.8 million.

The Scottish Qualifications Authority

Under statute, the Scottish Qualifications Authority (SQA) is required to keep under review and develop its portfolio of qualifications. These include National Qualifications, Higher National Certificates and Diplomas and Scottish Vocational Qualifications. SQA is provided with funding towards the development of national testing at various stages in primary and secondary schools.

x

Inspection Statistics

Background statistical information on the educational context and attainment of secondary schools and education authorities is made available on the HMIE extranet. This information is available for use not only by HMII during inspections, but also schools and education authorities as part of their own monitoring and self-evaluation. Information on attainment of similar schools and authorities is also provided in order to help with benchmarking.

Questionnaires evaluating the inspection process are sent to the various stakeholders after an inspection. Results from these questionnaires are fed into the HMIE quality groups with a view to monitoring and continually improving the process. Summaries of these questionnaires are published in the Annual Report.