National Evaluation of the New Community Schools Pilot Programme in Scotland: Phase 1: Interim Findings
This document is also available in pdf format (212k)
Research cannot make the decisions for policy makers and others concerned with improving the quality of education in our schools and colleges. Nor can it by itself bring about change. However, it can create a better basis for decisions, by providing information and explanation about educational practice and by clarifying and challenging ideas and assumptions.
It is important that every opportunity should be taken to communicate research findings, both inside and outside the Scottish Executive Education Department (SEED). Moreover, if research is to have the greatest possible impact on policy and practice, the findings need to be presented in an accessible, interesting and attractive form to policy makers, teachers, lecturers, parents and employers.
Interchange aims to further improve the Education and Young People Research Unit's dissemination of the findings of research funded by SEED. We hope you will find that Interchange is long enough to give the flavour of the complexities, subtleties and limitations of a research study but concise enough to give a good feeling for the findings and in some cases to encourage you to obtain the full report.
The views expressed in this Interchange are those of the author(s) and do not necessarily reflect those of the Scottish Executive or any other organisation(s) by whom the author(s) is/are employed.
Interchange may be photocopied for use within your own institution.
A limited number of additional copies can be obtained by writing to the Education and Young People Research Unit Dissemination Officer at the Scottish Executive Education Department, Victoria Quay, Edinburgh EH6 6QQ. File copies for electronic downloading are available on the Scottish Executive server at http://www.scotland.gov.uk/insight/
National Evaluation of the New Community Schools Pilot Programme in Scotland: Phase 1 (1999-2002)
Interim findings and emerging issues for policy and practice
Pamela Sammons, Sally Power, Pamela Robertson, Karen Elliot, Carol Campbell, and Geoff Whitty (Institute of Education, University of London)
The New Community Schools Programme was launched by the Scottish Office Education and Industry Department 1 in November 1998. Funding for the three-year pilot phase became available to local authorities in early 1999. The national evaluation of the pilot phase was commissioned from the Institute of Education, University of London, in Spring 2000 with final reporting due in 2003.
This summary presents some key interim analyses from the first two years of the evaluation, based on surveys across all pilot New Community Schools and on the early stages of five case studies. For policy makers, practitioners and managers in education and the NCS partner agencies, these preliminary findings contribute to development, within both the pilot phases and in the current programme of roll-out, announced by the Executive in late 2001. With final reporting further information will become available: from the final year of the pilot programme - survey analysis and exploration of the views of young people and other user groups, embracing issues of inclusion and support for vulnerable young people; and for the entire period of the pilot programme, analysis of data on pupil and school performance, such as exam results, exclusions and attendance.
These interim findings are framed in the present tense to emphasise the ongoing nature of the national evaluation and its associated analyses.
The New Community School pilot programme
'The pilot programme....... will be concentrated in disadvantaged areas where children face significant risk of social exclusion and formidable barriers to learning in their everyday environment. But the principles on which it is based are applicable to all schools.' (Scottish Office, 1998)
Scottish Executive policy for New Community Schools can be seen as part of its wider Social Inclusion Strategy. In recognition of the relationship between educational achievement, health, social and economic factors, New Community Schools were charged with expanding the range of support and services offered to young people. The New Community School Prospectus (1998) outlines five key goals:
- modernisation of schools and the promotion of social inclusion
- increasing the attainment of young people facing 'the destructive cycle of underachievement' (SOEID, 1998)
- early intervention to address barriers to learning and maximise potential
- meeting the needs of every child, ensuring that services are focused through New Community Schools
- raising parental and family expectations and participation in their children's education.
The 37 NCS initiatives in phase 1 of the pilot involve 170 schools or institutions in 30 local authorities. Some projects are single schools but more commonly comprise clusters of nursery, primary and secondary schools, with funding in the form of specific grant from The Scottish Office Excellence Fund. The cost of service delivery is drawn principally from existing education, social work and health programmes.
The national evaluation project: context, aims and methods
The national evaluation aims to find out to what extent these 37 pilot projects have achieved the 'essential' or 'likely' characteristics of New Community Schools based on the programme goals set out above. Its three main aims are:
- to inform the development of the phase 1 NCS pilots and aid future decision making
- to provide data and analysis on key, school-based outcome measures so that a comprehensive national overview is available at regular intervals
- to provide a summative judgement on the effectiveness of New Community Schools and hence inform local and national policy-making about their longer term future.
Additionally, factors that help New Community Schools to be effective, barriers to success and examples of good practice will be discussed in the final report.
Five major themes were identified by SEED in setting up the pilot NCS programme and the evaluation, employing both qualitative and quantitative approaches, uses these as initial foci:
Leadership and Management (including curriculum reform)
Surveys have been administered to each pilot NCS project with separate questionnaires for secondary, primary and nursery schools/classes. 2 These provide indicators, in key areas of NCS activity, at Baseline (1998/99), Year 1 (1999/2000) and Year 3 3 (2001/02) of the pilot phase. 4 A separate 'Vulnerable Children' questionnaire was included in the Year 3 survey.
To complement the surveys and other numerical information, five case studies of NCS pilot projects were selected to represent: different management and organisational models; local contexts, such as rural or urban; size in terms of the number of schools, staff and young people involved; and focus, scope and range of activities.
The case studies are ongoing. They provide evidence for evaluating the impact of pilot NCS projects drawn from the 'professional' perspective of staff from the agencies involved and, importantly, from the 'user' perspective of young people, their families and those directly supporting them. While this Interchange mainly reports emerging findings on leadership, management and organisational issues, largely from the perspectives of professionals, the user perspective will be addressed in the final evaluation report.
The research team has relied on the generous cooperation of many professionals associated with NCS projects and is grateful for their crucial contribution.
Change over the first year of the pilot programme: results from the national evaluation surveys
This section summarises responses to key items in the surveys for two years - Baseline (1998/99) and Year 1 (1999/2000). The response rate is high - above 90% for questionnaires at project, primary and secondary school level. In addition to numerical information, the questionnaires record perceptions of NCS-related activities, using a four point rating scale ( none/minimal; limited; moderate; and considerable).
Involvement in other initiatives
'New Community Schools will... make integrated provision of school education, family support and health education and promotion services.' (Scottish Office, 1998:5)
The NCS initiative, in common with the English Education Action Zones initiative, employs a 'cocktail approach to intervention, tackling obstacles from many directions and providing a wide range of programmes in which individuals and institutions can participate' (Power et al, 2001). All the NCS phase 1 pilot projects are involved in diverse other educational initiatives and most in health and social initiatives. Much of this work draws on the Core Programmes of the Scottish Office Excellence Fund.
Table 1: Number of schools reporting involvement in other education, health policy and social policy initiatives during year 1 of pilot
Health Policy Initiatives
Social Policy Initiatives
As Table 1 shows, schools in each sector report increased involvement in education, health policy and social policy initiatives (either local government, Scottish Executive, Health Board or Social Services). In particular, almost twice the number of primary and secondary schools report involvement in health policy initiatives during the first year of the NCS initiative compared to baseline. In Year 1 just over double the number of nursery, primary and secondary schools report involvement in all three types of initiatives. The increase in the number of schools involved in all three types of initiatives suggests that joint ways of working, bringing together education, health and social work in the delivery of services, have substantially increased following the start of NCS work.
NCS projects report increased involvement in education, health and social policy initiatives. Can this increase be attributed mainly to involvement in the NCS initiative? To what extent is there a general increase in Scottish schools' involvement in similar initiatives?
The extent to which involvement in other initiatives can be seen as separate from NCS activity in some projects is unclear. However, NCS funding and philosophy appear to support the combination and integration of different initiatives, acting as a catalyst to promote change more effectively.
'In principle and in the main, the cost of service delivery will continue to be drawn from existing programmes - education, social work and health.' (Scottish Office, 1998: 16)
'Authorities should also seek to attract and use other potential sources of funds.' (Scottish Office, 1998: 17)
The Year 1 project questionnaire asks about matched funding from health and social work services; and other additional funding. It was anticipated that social work departments and health boards/trusts would match the NCS funding from education. Analysis shows that:
- Over two thirds of projects report receipt of matched funding from health trusts or boards; and half of projects report matched funding from social work services.
- Approximately half of projects received matched funding from both health boards and social work services. However, a number report that, although there had been no direct financial input from health boards and/or social work departments, they had received 'in kind' funding, largely staffing.
- Comments from questionnaires reveal a diverse range of approaches to funding, with flexibility in funding arrangements seen as enabling joint working in some projects.
- For just under two-thirds of projects, the Core Programmes of the Scottish Office Excellence Fund provided additional funding in the first year of the pilot phase. These include initiatives such as Alternatives to Exclusion, Family Support Worker and Early Intervention. In addition, just under one third of projects received finance for out-of-school-hours work through the New Opportunities Fund resourcing, for example, breakfast clubs and study support.
- Approximately half of projects received support from 'other sources' - either financial or staffing.
- In all, 12% of projects reported no additional funding. Another 12%, in contrast, received additional funding from all three areas (the Core Programme, the New Opportunities Fund and other sources). It is likely that, in many cases, NCS status both stimulates and facilitates the attraction of additional funding.
Multi-disciplinary Staff Development and Training
'All proposals for the pilot programme should contain a programme of staff development and training for joint groups of teachers, social workers, community education workers, health professionals and childcare workers linked directly to their involvement in the NCS project.' (Scottish Office, 1998: 13)
Responses for the Year 1 questionnaire at project level show that:
- One fifth of projects report joint training to be none/minimal. Respondents comment on the effects of delay in appointing NCS staff and the long-term pre-booking of school development days by education authorities.
- Just under half of respondents indicate that joint training had been limited during Year 1, commenting on the prevalence of planning activities in preparation for training in the following year, in contrast with actual training events. 5
- A quarter of respondents report moderate levels of joint training in Year 1, illustrated, for example, by the use of NCS training materials, a series of training days and job shadowing.
- Less than 10% report a considerable level of interdisciplinary staff training with one project offering a full staff development programme at a cost of 50,000.
The extent and emphases of joint staff development and training vary considerably across projects. What effect might this have had on the quality of NCS provision for young people? What are priorities for staff development and training? And for whom?
Health Promoting Schools
The pre-requisite for eligibility for participation in the NCS initiative will be commitment by schools to work towards achievement of the formal status of a health promoting school.' (Scottish Office, 1998: 12) 6
Respondents were asked to rate the extent to which their schools could be described as health promoting for both baseline and Year 1. 7
Ratings for combined moderate and considerable categories increased in Year 1, showing consistent perceptions of positive change in schools in all sectors:
from 62% to 80%
from 43% to 66%
from 35% to 60%.
These findings are amplified by further analysis:
- To a greater extent than primary or secondary schools, nursery schools/classes are perceived to be 'health promoting' in both years. Comment on healthy eating programmes is pre-eminent in this sector. Other programmes emphasised in the nursery sector are: dental hygiene; monitoring growth height and weight; general hygiene; promoting physical development and movement/exercise.
- Primary schools also emphasise healthy eating programmes, some involving parents and community agencies such as local food co-operatives.
- Secondary schools put less emphasis on healthy eating. Priorities here are the promotion of healthy living, drugs and smoking awareness, HIV issues and sex education. Secondary schools also emphasise psychological aspects such as creating a positive school ethos and raising self-esteem.
Perceptions suggest that health promotion is gaining ground in all sectors. What approaches might be used to evaluate its impact?
Personal Learning Plans (PLPs)
'New Community Schools will be based on the development of personal plans..... individual learning programmes developed and discussed with parents and reflecting the full needs of the child and his or her family.' (Scottish Office, 1998: 3)
SEED held a seminar on personal learning plans for Integration Managers in May 2000. The NCS web site also channels discussion and dissemination of information about PLPs. ( www.scotland.gov.uk/education/newcommunityschools/reports.htm ).
Table 2: Reported Implementation of Personal Learning Plans in Schools
Table 2 shows that for baseline and Year 1 the majority of nursery, primary and secondary schools report that PLPs were not yet in place, or that their implementation was minimal. However, there are indications that in Year 1 a minority of schools, particularly primaries, had made some progress in implementing PLPs. Comment on the survey forms suggests that:
- For all three phases of education, only a small number of schools had implemented PLPs prior to the start of the phase 1 NCS pilots. In Year 1 most PLP activity was associated with planning and piloting.
- Schools more commonly had Individual Education Plans (IEPs) for children with special needs and considered this higher priority. Alternatively, PLPs were often used to support very small numbers of young people with particular difficulties or vulnerability.
- A few primary schools had implemented PLPs with all pupils and one secondary school had given significant attention to the planning of PLPs.
- Reported levels of parental involvement in PLPs increased in most schools in Year 1. However, in the light of the general findings about implementation, it is not surprising that half of responses indicate parental involvement to be none/minimal.
Although PLPs continue to be developed and piloted in some projects, others report that they chose not to work on these during the first year, in the hope of further national guidance and exemplars. Reported difficulties in implementation relate mostly to workload and staffing. Progress in implementing PLPs and parental involvement in the process will be further explored in the final evaluation report.
In Year 1 implementation of PLPs has been slow. What are the main factors causing this? From national to local level what might be done to remedy the situation?
'New Community Schools will have a positive, inclusive ethos in which children, their parents and teachers are focused on learning in a supportive but challenging environment and on improving attainment in formal and informal learning settings.' (Scottish Office, 1998: 10)
Several aspects of NCS work concern the generation of greater pupil engagement in school, particularly for disaffected groups of young people. Examples include opportunities for pupils to express their views, such as school councils, and extra curricular provision, such as breakfast clubs. Results from the survey indicate that breakfast clubs in particular received a strong emphasis during Year 1. Other main findings include the following:
- The majority of secondary schools report provision of extra-curricular activities as moderate or considerable for all year groups. From Baseline to Year 1, the ratings for these two categories combined increased from approximately 75% to 95%.
- In primary schools, the perceived extent of extra-curricular activity increased during the first year of the pilot. Ratings for combined moderate and considerable categories increased for all year groups, being most marked for older primary pupils:
from 20% to 30%
from 40% to 60%.
- Commentary describes a breadth of extra-curricular activity in primary and secondary schools, including, for example, sports clubs, arts and crafts, cultural trips and residential experiences.
- The majority of nurseries rated the extent of extra-curricular activities for both 3 and 4 year olds as none/minimal in both years.
- During the first year of the pilot project the percentage of primary schools with pupil councils shows a marked increase, almost doubling, from 25% in the Baseline year to 47% in Year 1. A relatively greater percentage of secondary schools already had a school council in operation prior to the NCS pilot, with a reported increase for Year 1 from 54% to 65%.
- The number of primary and secondary schools reporting the introduction of other ways for pupils to express their views also increased. In primary schools this rose from 59% to 75% and in secondary schools from 69% to 89%. Comments include the following examples:
- circle time in class and pupil councillors
- surveys of pupils' views and an 'open door' policy for pupils
- during PSE lessons with guidance teachers.
- In general, breakfast clubs feature prominently in NCS planning documents. Subsequently, however, the surveys show that fewer than 10% of primary and secondary schools provided breakfast clubs before the NCS initiative. This increased to 25% for primary schools and 36% for secondary schools in Year 1.
- The pre-school sector consistently reports little use of breakfast clubs.
- More detailed analysis of Year 1 data shows that a third of primary school breakfast clubs involved parents and/or other community members. Most clubs were reported to be open to all, but a few breakfast clubs specifically catered for targeted pupils, such as children who had difficulty getting to school on time.
One of the most prevalent approaches to increasing young people's engagement with school is through provision outside usual school hours. What are the potential benefits and limitations of this strategy? What other approaches might also be effective?
'The development of a New Community School will provide an important opportunity and mechanism to build the capacity of the local community.' (Scottish Office, 1998: 8)
The Year 1 school questionnaires asked about the extent of increased involvement in activities within the community under the auspices of the NCS initiative. In general, perceptions of community involvement for secondary schools are stronger than in the primary and nursery sectors. For all sectors, however, over 80% of the responses fall in the none/minimal and limited categories in Year 1. While this indicates that community engagement has generally made limited progress during the first year of the pilot, commentary suggests that more schools are in the process of becoming involved in the community's strategic planning process.
In Year 1 engagement with the community shows limited progress. What might be key strategies for boosting the rate of progress?
Examples of the activities that schools report as representative of involvement in the community within the NCS remit include:
- range of taster courses arranged jointly with Community Education
- nativity concert at hospice and entertainment for the elderly
- involvement with local voluntary organisations supporting vulnerable children
- Community Art Project; management and development of Credit Union
- 'Now You Are Cooking' provision for young adults
- parent and toddler; and lone parent groups.
Parent and Family Engagement
'New Community Schools will... adopt strategies to encourage pupil and parents, together and separately, to develop positive attitudes to learning.' (Scottish Office, 1998: 5)
The concept of parental and family involvement in the child's education is fundamental to NCS policy. Embedded in many school improvement activities, it also underpins a number of the questions in the evaluation surveys concerning, for example, learning opportunities provided for parents and other adults and parenting skills advice or courses.
For the Baseline year over 85% of nursery and primary schools report none or limited learning opportunities or parenting skills advice/courses for parents and other adults. Similarly, very few secondary schools offer such provision. In Year 1 of the NCS initiative more schools report moderate or considerable learning opportunities and parenting skills advice/courses. These schools report a fairly diverse range of learning opportunities available to parents and other adults, including:
- literacy and numeracy classes available free for all
- Confident Parents: Confident Children course
- arts, crafts, first aid and basic computer skills
- taster courses, jointly with Community Education - aromatherapy, stress busting, local history, introduction to computers, sewing, cookery
- healthy lifestyles: dental health, First Aid, healthy eating, fire safety, road safety
- Adult Learning Base, IT suite, Adult Learning Team
- summer school for adults.
The results for both years suggest that the lifelong learning agenda for adults, including parenting skills, has been emphasised less in the secondary sector than in nursery and primary schools, most probably reflecting the greater opportunity for parental contact although not necessarily greater need.
Support for parents is largely concentrated in the nursery and primary sectors. Supporting children and young people as they develop presents challenges to parents at all stages. What approaches might extend good practice to the secondary sector?
Commentary shows a wide range of provision in parenting skills advice/courses including one-to-one support, confidence building and practical advice about support from different agencies.
Interim findings from case studies
There is broad consensus among respondents that multi-agency work, under the auspices of the NCS initiative, is increasingly effective and is beginning to have a valuable impact on young people's lives. However, this preliminary finding and the more detailed analysis presented here are based on the first stages of fieldwork in five case studies carried out mainly with participants in school and service management. They therefore largely address a number of common themes concerning NCS leadership, management and organisation. The subsequent stages of the case studies focus more directly on the experiences of young people, their families and communities as they are influenced by NCS activities. Complete case study analysis will be presented in the final report planned for 2003.
Management roles and responsibilities in inter-agency working
' Integration of services is essential... This will require radically new approaches....will enable action to be taken early to meet the needs of vulnerable children through swift identification of problems and immediate referral to support services.' (Scottish Office, 1998:4)
The integration manager would
- be responsible for the operational management of the pilot project...accountable to the project steering group
- oversee the integration of the work of the specialist agencies providing support to students and members of the local community...
- ensure and promote effective communication between schools, service agencies and the local community
- monitor the ongoing work of the pilot project and report to the steering group on the project's impact on educational attainment and social welfare.' (Scottish Office, 1998: Annex C)
- Operational and line management. Most projects in the phase 1 pilot have appointed an integration manager to coordinate the work of the several agencies involved. The addition of this extra layer, complementing service and school management, creates considerable challenge. There is consensus about the importance of managerial commitment, skills, energy and enthusiasm in establishing and developing the NCS approach, particularly in the complex circumstances of NCS organisation. Operational management combines the roles of leader, manager, facilitator, trainer and intermediary. In some cases a tension appears, especially in the relationship between school and NCS operational management, over issues of remit, responsibility and line management. In addition to being sensitive to these issues, it is important that NCS managers are seen to take an equitable, diplomatic and holistic approach, engaging all agencies as appropriate.
For operational managers of NCS projects, the title 'Integration Manager' may not be the most appropriate, suggesting management of a nebulous process, rather than an organisation or body. It may also suggest to co-workers a subsidiary status to those with responsibility for institutional management. The inter-agency nature of the post means that the line management of IMs' work within local authorities is not straightforward. Their line managers ideally need to be part of, or have access to, an integrated framework of local authority senior management.
Most interviewees, not only managers, have been concerned, as the phase 1 pilot draws to an end, about a potential leadership vacuum in multi-agency working. While there is widespread belief that much good inter-agency practice is becoming embedded, there is also worry that much that has been achieved may be dissipated, should management arrangements become more diffuse. Aware of this possibility, several managers have been trying to build self-managing groups to carry forward NCS work.
Refining existing inter-agency approaches. An early task for NCS management is the audit and review of existing inter-agency work against NCS aims and objectives. In some case studies, the NCS initiative is seen as bringing together and developing existing projects, providing a clearer and more cohesive focus. Additional resourcing, increased staffing, and more streamlined management and administrative structures were seen as key drivers of improvement by the majority of interviewees.
- Recognising barriers to multi-agency working. Case study data indicate that multi-agency working is often very difficult to achieve. Further analysis suggests three types of barrier or tensions across services:
- structural - for example, the school timetable, holiday arrangements and line management structures
- cultural and attitudinal - for example, as reflected in dress codes, levels of formality and values expressed about the purpose of the NCS
- professional - for example, confidentiality procedures.
Cultural and attitudinal barriers are perhaps best exemplified by varying approaches to exclusion and inclusion, particularly between education and social work staff. Such barriers can also impinge on outside perceptions of NCS managers and on their professional interrelationships, and previous experience in inter-agency work appears to be helpful for NCS operational managers.
Multi-disciplinary staff development. Across the case studies, perceptions of the extent and quality of staff development vary. Where training reveals the work experiences of colleagues from other disciplines it is seen as especially beneficial. The perceived essentials for multi-disciplinary training are that it should be based on participants' working circumstances and involve a wider range of staff, not only the core NCS team.
Issues of location. Location of NCS services in the community - not always in the school - appears to be helpful in ensuring awareness and participation. However, when NCS core teams are located in one school, this can provide particular benefits of accessibility to this school (often the secondary). Overall, co-location of multi-agency staff is considered helpful in enabling formal and informal working practices to develop, and in offering accessibility for young people, families and other clients. Finding sufficient space and suitable physical environments for NCS staff are continuing concerns.
The remit of the integration manager is diverse and challenging. What are the advantages and disadvantages of locating this degree of responsibility in one person? What other approaches might be used?
NCS governing arrangements
'The management structure... includes a single reporting and accountability framework ... for all of the core services involved...' (Scottish Office, 1998:9)
- NCS steering and management groups. At the inception of NCS projects there may be many existing overlapping committee and management structures sharing elements of NCS remit. The addition of NCS steering and executive groups has the potential to create further confusion about responsibility for decisions and actions. Some local authorities have addressed this effectively by rationalising the number of committees, clarifying remits, improving decision-making and delivery. Local authority commitment to NCS projects is often evidenced by the way committees are constituted with directors of education and their partner agencies ensuring an appropriate input from NCS operational managers.
The roles of governing bodies have tended to change over the life of the NCS pilot, shifting from strategic planning to operational management as NCS projects become more established. Several case study projects make use of short-life working groups to plan, establish and monitor individual initiatives.
Governing arrangements can both help and hinder effectiveness. What steps can be taken to streamline governance and accountability in a multi-agency setting?
The New Community School as an overarching structure.
'Integration of services is essential and the school is an excellent place for this to become a reality.' (Scottish Office, 1998:4)
- The size of the NCS project and its perceived impact. Case study data suggest that perceptions of NCS interventions are related to the size of the NCS and the number of schools and staff involved. It seems, perhaps inevitably, that the larger the cluster the more attention focuses on bureaucratic procedures. Although perhaps necessary for developing a new infrastructure, they make it more difficult for professionals working 'on the ground' to see development and benefits in the short term.
- The New Community Schools approach and participating schools. Headteachers have differing views about the NCS project being part of a whole-school development and the quality of the professional relationship between school management and IMs varies. This has implications at local authority level and also inside schools. In some case studies, relations between IMs and school staff too are problematic. In part, this may be because of the differing nature of their tasks and their associated perceptions. In general, there is less awareness about the activities and impact of the NCS amongst middle management and class teachers than in school managers and those with dedicated responsibilities, for instance, for learning/behaviour support. There is a need for ongoing communication with teaching staff about the aims of NCS.
Many factors influence the relationships between schools and their NCS partner agencies. What are the best ways of clarifying and enhancing the school/agency relationship?
'The (Scottish Office) grant will provide the means for the New Community School to operate in the integrated way set out.... In principle and in the main, the cost of service delivery will continue to be drawn from existing programmes - education, social work and health. (Scottish Office, 1998:16)
- Matched and additional funding. Securing matched funding varies considerably amongst NCS projects. The majority of case study projects use additional funding to expand existing activities and enhance staffing, rather than being innovative, often considered an appropriate way to get value for money from limited resources. There have been varying attitudes to the perceived short time scale for the pilot and some uncertainty over funding after March 2002. Some NCS management teams have postponed or cancelled development while others have continued planning and activity, believing that resourcing will follow success and need.
- The deployment of funds. It is often difficult to distinguish NCS work according to its funding stream, as resources from NCS, SIP and many other funding programmes are often combined. NCS funding distribution has varied in deployment across staffing, professional development and purchase of material resources. Additional staffing for NCS activities has been important in making an impact, but the case study NCS projects have found recruiting and retaining such posts has been difficult mainly due to short-term funding. NCS project control or direct influence over resourcing is perceived to support the effectiveness of service delivery.
- Resources and project size. Within the case studies, it would appear that the larger clusters studied suffer from a dilution effect in resourcing, at the level of services for young people, their families and communities. However, it may be that, with the rolling out of the programme, these larger projects will not only provide valuable lessons for wider-scale implementation but will have built a platform for effective roll-out.
'Proposals should specify and measure outcomes... set targets on all aspects...set out baseline measures ....raised attainment... raised attendance and reduced exclusion; improved service integration; improved learning; improved health.' (Scottish Office, 1998:9)
Internal or local evaluation is not generally well developed across the NCS pilot projects. What are the main obstacles to effective local evaluation of New Community schools? What approaches would be most useful?
- The extent and nature of local evaluation. In general, national evaluation data suggest that local evaluation has been a low priority in the pilot phase. In some NCS projects evaluation is perceived largely as a means of gauging general impact rather than providing information for strategic planning throughout the project. Few projects have defined clear baseline indicators. One case study project, however, has built evaluation of its activities into its development from the start with apparent benefit in refining strategy and practice.
IMs make use of independent HMI and local authority quality assurance reports, where available. Also, several NCS projects have used survey and audit approaches to identify needs and target provision.
Overview and emerging issues
NCS projects are diverse both in the extent and nature of their associated activities. In a national programme where local authorities have had freedom to interpret Executive policy, this variation would be expected, reflecting the multiple influences within local history and strategy development, and in the communities served.
However, from this diversity some clear patterns are emerging. Despite the variation in attracting matched funding from partner agencies in Year 1, evaluation evidence indicates that cross-agency liaison and practice have been substantially enhanced. The weight of this finding should not be underestimated as case study analysis highlights the difficulties of initiating and sustaining effective multi-agency working. Understanding and overcoming the various barriers are continuing tasks for NCS operational managers. Although multi-agency staff development and training have been relatively under-developed in the first year of the pilot phase, there are indications from the case studies of its powerful potential for overcoming these barriers.
Overall, it would seem that education-based initiatives have, perhaps not surprisingly, been more readily implemented with help from associated funding. An exception to this is the relatively slow development of personal learning plans. Many projects also report involvement in a wider range of initiatives - health promotion in particular features relatively strongly in the questionnaire returns for Year 1. Also a good number of projects indicate they have obtained additional funding from a variety of sources, enabling a wide range of new multi-agency activities to be undertaken.
There appear to be significant school-based developments to promote pupil engagement outside the usual curriculum framework with an increased range of after-school and holiday activities. There is evidence that action to 'listen to the pupils' voice' increased during Year 1. Projects designed to augment family and community engagement have been slower to get off the ground, but many projects report further development due in the subsequent year.
The further establishing of principles of multi-agency working and service delivery in community and school life will likely depend on appropriate management, funding and governance structures, as well as high levels of commitment from key individuals in local authority, NCS programme and school management.
In addition to completing case studies, the national evaluation team is currently analysing information from Year 3. These results, including data for vulnerable pupil groups, will combine with further case study findings to provide a more complete picture of impacts, successes and areas of difficulty. In addition, the analysis of school outcome data from SEED will supplement the evidence collected from NCS projects themselves. Thus the evaluation will explore change and progress over the full three years of the pilot period.
References and bibliography
Campbell, C., Gillborn, D., Lunt, I., Robertson, P., Sammons, P., Vincent, C., Warren, S. and Whitty, G. (2000) A Review of Developments in Inclusive Schooling: A Report to the Scottish Executive Education Department. London: Institute of Education, University of London.
Garmanikow, E. and Green, A. (1999) 'Developing Social Capital: Dilemmas, Possibilities and Limitations in Education'. In A. Hayton (ed.) Tackling Disaffection and Social Exclusion: Education Perspectives and Policies. London: Kogan Page.
Mortimore, P & Whitty, G (1997) Can School Improvement Overcome Disadvantage? London: Institute of Education, University of London.
Parsons, C. (2001) Evaluating the Health Promoting School - a Contested Global Vision for Change. Paper presented at the British Educational Research Association Conference, Leeds.
Power, S., Campbell, C., Elliot, K., Robertson, P., Sammons, P. and Whitty, G. (2001) New Community Schools in Scotland: Strategies for Inclusive Schooling. Paper presented at the Scottish Educational Research Association Conference, Dundee.
Scottish Executive (2001) Learning With Care: The Education of Children Looked After Away from Home by Local Authorities. Edinburgh: Scottish Executive
Scottish Executive (2001) For Scotland's Children: Better Integrated Children's Services. Edinburgh: Scottish Executive
Scottish Executive Education Department (1999a) A Route to Health Promotion: Self-Evaluation Using Performance Indicators. Edinburgh: Scottish Executive
Scottish Executive (1999b), Social Justice..... a Scotland where everyone matters. Edinburgh: Scottish Executive: The Stationery Office Ltd.
Scottish Executive (1999c), Social Inclusion - Opening the doors to a better Scotland. Edinburgh: Scottish Executive: The Stationery Office Ltd.
Scottish Executive (1999d), The Scottish Executive Response to the Report of the Advisory Committee on the Education of Children with Severe and Low Incidence Disabilities. Edinburgh: Scottish Executive: The Stationery Office Ltd.
Scottish Office (1998), New Community Schools: The Prospectus. Edinburgh: Scottish Office.
Whitty, G., Aggleton, P., Gamarnikow, E. and Tyrer, P. (1998) 'Education and Health Inequalities Input Paper 10 to the Independent Inquiry into Inequalities in Health'. Journal of Education Policy, 13, 641-652.
The Interchange series