INTERCHANGE 71

Early Intervention in Literacy and Numeracy: Key Issues from the National Evaluation of the Programme

Helen Fraser (University of Edinburgh), Audrey MacDougall (KPMG), Anne Pirrie and Linda Croxford (University of Edinburgh)

This document is also available in pdf format (146k)

Introduction

The Early Intervention Programme (EIP) was launched in Scotland in June 1997. The programme was 'aimed directly at raising the standards of literacy (reading and writing) and numeracy skills in primary years 1 and 2' (SOEID, 1998a). The programme was informed by a number of different developments, policy issues and research findings and became a five-year, £60m project incorporated within the Excellence Funding initiative announced in November 1998 by Helen Liddell, the then Scottish Education Minister. The Excellence Fund impacted on the way in which local authorities organised both interventions at local level, and the integration of intervention activity, with other initiatives and funding strands, eg classroom assistants, family literacy and childcare.

Early intervention was a response to the concern that 'all children must have the right start in primary school ... if they are to access all later stages of the curriculum' (SOEID, 1998a). It was widely recognised that there was a particularly vulnerable group in terms of children most likely to make poor progress. This group largely comprised those children who lived in disadvantaged socio-economic circumstances and in neighbourhoods where such circumstances were prevalent.

National Evaluation of Early Intervention

In 1998 the then Scottish Office Education and Industry Department (SOEID) commissioned a three-year evaluation of the initiative. The evaluation was carried out by a team comprising University of Edinburgh academics and a chartered accountant from the financial consultants KPMG. The aim of the evaluation was to determine the impact of the developments on pupils, classrooms and the range of stakeholders. We also had to consider the concept of 'best value' which local authorities were expected to implement as a way of improving local government performance in the delivery of services to local communities throughout Scotland.

The findings are reported in summary form in this Interchange.

Objectives of the evaluation

The evaluation began in February 1998 with the following broad objectives:

• To monitor the development and evaluation of the local authority schemes;
• To determine the impact of the initiative on pupil attainment in literacy and numeracy;
• To explore any other impact of the initiative from the perspective of the various stakeholders;
• To gauge what had been the most cost effective approaches.

Design and method

The evaluation was designed to be both quantitative and qualitative and entailed gathering data from:

• Documentary material from local authority annual returns and evaluations
• Standardised test results from PIPS (Performance Indicators in Primary Schools) administered to samples of Primary 3 pupils in 1998 and 2000
• Questionnaire surveys of authority and school staff
• Interviews in a small sample of schools and with some home-link staff
• Questionnaire sent to parents
• Financial data gathered from documentary sources and questionnaire

Different complementary layers of data were examined to provide answers to the major research questions. For example, pupil attainment was analysed from three data sets: first-hand test results; the assessments made by individual local authorities; and the survey data on how attainment was perceived by local authority co-ordinators, head teachers, teachers and educational psychologists.

The findings

Here we summarise the main findings from our evaluation. We look first at how the EIP programme progressed over the first three years and how different local authorities focused their resources. This is followed by a summary of our conclusions on pupil attainment during this period. We then report what effects there were on classroom practice and go on to consider how parents were involved and their perceptions, particularly on their children learning to read. Finally, we review how resources were managed at local level.

Implementation

There was considerable interchange of ideas across Scotland as the programme developed. In addition to one national event organised by HMI, the evaluation team set up two such opportunities for formative feedback and, as local authority EIP co-ordinators of programmes became established, several local conferences were organised in order to exchange experiences of implementation.

• By June 2000 there were 67% of Scottish primary schools included in the EIP programme. Some local authorities focused their resources on a small number of schools and others dispersed their monies over a larger proportion.
• Local authorities had a considerable degree of autonomy in the approaches they chose to take to early intervention. However there was a clear consensus amongst local authorities around the curriculum and organisational approaches of phonological awareness, onset and rime analogy, emergent writing, mental maths and whole class/interactive teaching. Approaches such as synthetic phonics, paired reading and the use of 'story sacks' as part of home-school link initiatives became more prominent as the EIP developed.
• Numeracy intervention took much longer to become established over the three-year period than literacy. The evaluation could therefore capture only the early effects of change.
• Most head teachers in the survey sample (schools in the programme from the beginning) reported that their schools were located in areas of multiple disadvantage, with significant numbers of pupils experiencing learning difficulties. Class sizes in these schools were already well under the statutory limits. However more than one-third of the head teachers were continuing to employ strategies to reduce class sizes.
• Head teachers and their teaching staff valued continuity in the deployment of staff within the infant stages. Staying with the same stage, following a class into the next stage and simply remaining within Primary 1 to 3 each had their adherents.
• Most head teachers used the flexibility that they had locally to 'top up' core early intervention funding from their own school budgets.
• While most local authorities by 1999 had developed position statements on progression through literacy skills, a fairly high proportion of schools had not done so. There were rather fewer authorities and schools reporting position statements on progression through numeracy skills.
• Most local authority co-ordinators considered consultation and planning with head teachers to be very or extremely important.
• Most local authority co-ordinators considered staff development of all classroom staff to be very or extremely important. There was a consensus that the emphasis was on developing methodology rather than on providing material resources.

Impact of early intervention on pupils

Evidence from assessment data, from local evaluations and from our survey of stakeholders all comes together in suggesting that the EIP has indeed been successful in raising attainment in literacy and numeracy in the early stages of primary school.

• The PIPS test showed that there was a significant overall increase in reading attainment at Primary 3 between 1998 and 2000. There was no such overall significant change in mathematics attainment in the PIPS test results.
• The PIPS results did not show a pattern of significantly greater improvement in reading in 'EIP' as opposed to 'non-EIP' schools. However the difference in favour of EIP schools was significant in local authorities where the EIP funding had been focused on a smaller proportion of schools.
• There was some evidence that the amount of increase in reading attainment was greater in those schools where reading recovery approaches had been implemented. Home-link support was also associated with an increase in both reading and mathematics attainment.
• Attainment in mathematics on PIPS was, on average, lower in 2000 than in 1998 in schools where 'more phonics', or 'synthetic' or 'Jolly Phonics' had been introduced to improve reading attainment.
• Local authorities own evaluations reported considerable improvement in both literacy and numeracy. Their data, in varying degrees, included results from standardised tests and National 5-14 Tests. Their findings were broadly in line with the PIPS results, showing progress, but still a 'gap'.
• More than half of the local authorities used terms like 'significant' 'marked', 'considerable' or 'steady' to describe the progress made. Our survey data confirm these findings. A small number of local authorities attributed their success to the introduction of synthetic phonics. Others cited that the mix of approaches taken was what was effective.
• Respondents in schools were also confident that the EIP had been effective in increasing attainment in reading, writing and numeracy. The perceptions of effectiveness were particularly marked in reading and writing.
• Class teachers in Primary 1 and 2, although generally very positive about the extent of attainment gains, were rather more measured in their responses than other stakeholders ( eg learning support teachers, local authority co-ordinators, educational psychologists).

Fig 1: Views on the extent of improvement in reading

Looking more closely at pupil characteristics and whether there was any link between these and the reading and mathematics scores on the PIPS test we find the following significance levels:

Table1: Background factors influencing reading and mathematics scores in 1998 and 2000

+ denotes a positive effect at the 95% confidence level
- denotes a negative effect at the 95% confidence level
ns denotes that there is no statistically significant effect of the variable
FME Free meal entitlement

Impact of early intervention on practice

Another broad measure of success must be the impact on teachers and on practice. The three years covered by this evaluation saw considerable efforts being made at all levels to bring about change.

• Three-quarters of Primary 1 and 2 teachers', and the learning support teachers working with them, reported increasing levels of professional confidence, and enthusiasm for teaching, since the introduction of the early intervention programme. They also reported having higher expectations of pupils and in addition were more aware of the research evidence on literacy and numeracy. Levels of satisfaction with staff development for early intervention were also generally high. Some ideas were very much encouraging a collegial style of development. For example, a local authority EIP co-ordinator reported:

• The majority of teachers identified the most satisfactory element of the EIP as being the enhanced rate of progress made by the pupils, and their evident enthusiasm. However about one third had concerns that there was now too much pressure on Primary 1 and 2 pupils.

  Is there a case for debating the relationship between play, pupil initiated learning and the teaching and learning approaches to literacy and numeracy?

• Most teachers reported spending more time overall on literacy and a third said the same of numeracy. However they were also concerned that the place of play and self-directed activity in Primary 1 and 2 was now under-valued.
• Head teachers and teachers reported increased pressure on early stages teachers and a significant increase in workload, but most teachers rated very highly the contribution of the nursery nurses or classroom assistants working with them. Most authorities employed nursery nurses or classroom assistants as part of the EIP. Numbers grew over the
  three-year period, and at the same time the ratio of nursery nurses to classroom assistants altered with a higher proportion of classroom assistants being employed. Most nursery nurses/classroom assistants reported that they 'heard' pupils read in the classroom.
• There was a clear consensus that having a nursery nurse or classroom assistant increased the amount of adult child/interaction and provided class teachers with support for teaching. About three-quarters of teachers and nursery nurses/classroom assistants found collaboration easy. However head teachers and class teachers considered that the presence of additional adults in the classroom posed some organisational challenges, although only a third of all staff groups reported that they had had staff development on working with others.
• The level of job satisfaction among nursery nurses and classroom assistants was high, although there was a substantial decline between 1999 and 2000 in the numbers reporting 'extremely high' levels of job satisfaction.
• There was some evidence that the contribution of learning support teachers to early intervention was relatively underdeveloped compared to that of nursery nurses or classroom assistants.

Involving parents

The involvement of parents in efforts to raise attainment was considered from the perspective of the schools, home-link staff and parents themselves.

• Two thirds of head teachers said they had a formal school policy on parental involvement and most local authorities included the development of home school links in their plans.
• Home-link staff reported occupying an uneasy position between schools and other community services. They felt they needed more support from head teachers to make them more effective as mediators between home and school. However they also reported very supportive reactions from classroom teachers.
• The majority of head teachers, learning support teachers, educational psychologists and local authority personnel felt that parents 'were more involved' in the development of their children's literacy than they had been previously. Rather fewer class teachers were of this opinion. Almost all teachers felt that involving parents at home was very important, but only one third felt the same about involving them in the classroom.

Fig 2: Perceptions of more parental involvement since EIP

• A quarter of head teachers reported having parents helping with reading in the classroom. However, teachers' responses suggest that there was a substantial reduction in classroom parental involvement as the EIP developed. At school level the provision of written advice was by far the most common way of encouraging parental involvement. Discussions with parents were perceived by nursery nurses and classroom assistants to be very much the preserve of the class teacher.
• The majority of parent respondents - from a small sample survey - felt that their children were making good progress in reading. However almost a third also reported that their children were anxious about reading, and this was more often the case for boys and the younger children. A quarter of the parent respondents were not confident that their child could cope with the work expected of them at school.
• More than a third of the parent sample reported that they did not find the information on how to help with reading 'very useful' and some parents expressed dissatisfaction with the amount of information on children's progress.

Managing the resources

Since the commencement of the evaluation the concept of best value has taken a firm hold within local government. A key part of this study was to assess the value for money offered by the EIP.

• The EIP ultimately became a significant investment by government. The £7m made available in the first year was later doubled to £14m annually. Authorities were expected to match this contribution by providing funding equivalent to 50%. EIP spending as a percentage of the overall education budget for councils ranged from just above 0.3% to 0.7%.
• Initial bids and subsequent monitoring documentation contained variable financial data, both in terms of quality and quantity. This would be expected in the context of the timeframe available for the preparation of the initial bids and the variability of Council management information systems throughout the country.
• The extent to which Councils operated devolved budgeting systems also had an impact. Where such systems were operated it appeared to be difficult to combine the devolved elements to gain an oversight of the total. This had the potential to lead to a lack of control or knowledge regarding the overall progress of the programme.
• Many councils adopted a focused approach, the average council concentrated its resources on 40% of its total pupil population. Expenditure per child ranged from an average of £200 to a maximum of £1,200.
• The pattern of resource usage changed over the duration of the programme. In 1997-98 councils spent 56.3% on additional staffing and 32.7% on staff development. By 1999-00, this had changed somewhat and councils spent 77.5% on staffing and 8.9% on staff development as staff were recruited into post and more of the existing staff were already trained. (These figures should be treated as indicative due to difficulties with definitions).

  What are the best structures at local authority level for maintaining a year on year focus on early intervention?

• Initially expenditure on management was low as it was expected that this would be incorporated into existing roles. However, as the programme developed, the need to employ specific managers and increase the spend on management was recognised and actioned.

Conclusions

Attainment

The Early Intervention Programme has had an enormous impact on many schools in Scotland. The findings of this research study suggest that attainment overall rose, particularly in literacy. On average, the gap between the socio-economically advantaged and disadvantaged remained, but the evidence is that the increase in reading attainment was greater for the most disadvantaged pupils in those local authorities where a policy was adopted of focusing the available resources on fewer of their schools. The quantitative data also suggest a tentative link between home-link and the attainment of some children.

Teachers

Another broad measure of success was the impact on teachers and on practice. The evidence suggests that teachers who were involved in early intervention with Primary 1 and 2 have enjoyed renewed confidence and increased enthusiasm, and remain broadly convinced of the efficacy of the EIP. A culture of mutual learning, particularly around literacy, was in evidence across the country.

The majority of early stages teachers surveyed were more enthusiastic about their teaching, and more confident in themselves as teachers. Staff development had made them more aware of the research background to their teaching than they had been previously, and they derived great satisfaction from the achievements and enjoyment of the children in their classes. A high proportion of these children were from disadvantaged circumstances.

Additional staff

There was a clear consensus that having a nursery nurse or classroom assistant in Primary 1 and 2 classrooms was enormously beneficial, and there was a good deal of job satisfaction reported by those extra staff. However, there are grounds for examining the organisational challenge that these new working practices present.

Collegial emphasis

It is likely that the early spirit of the initiative, the openness of the debate and the sense of active and influential partnership that developed at local authority and at school level were contributory factors to the positive ethos. Maintaining this collegial emphasis was seen to be important.

Teaching and learning

As to classroom practice, our data reflect undoubted changes. More phonics-based approaches to reading, and more interactive mental mathematics featured frequently in local authority and school data. The value of a more learner-centred approach to writing with the emphasis on 'emergent writing' was also heralded. Whole-class teaching and more direct teaching were recurring refrains. It is difficult at a national level to be certain of the particular effects of these changes. Some local authorities attributed attainment success to a specific approach -- the role of synthetic phonics is one example. Others were convinced that it was the mix of provision that worked: the staff development; the extra classroom staff; home-link; the research-based curriculum and more focused methods of teaching. Although teachers were on the whole confident that pupils were enjoying their learning, there was also widespread concern that there was now too little play and opportunity for self-directed learning in the early stages of the primary school.

Recommendations

• Further early intervention should resolve and clarify policy and resource issues in respect of raising attainment and addressing educational inequalities associated with disadvantage.
• The history of early interventions should not be ignored. Longer-term success is not assured from promising beginnings. Pupils at risk will need focused support well beyond the early primary stages.
• Other inequalities, such as those attributable to age and gender, should continue to be a concern. Policy at national and local levels should explore strategies for greater flexibility of age at entry regulations.
• Teachers' involvement, enthusiasm and professional satisfaction appear to have been part of the success of early intervention. Maintaining this should be a school management priority.
• There should be a focus on taking forward a style whereby schools and teachers continue to be actively involved in the development, management and evaluation of interventions/projects/teaching and learning strategies.
• Local authorities should continue to share developments and evaluation findings.
• The perceived success of the introduction of nursery nurses, classroom assistants, learning support or additional teachers is universal in schools. Our evidence is that retaining this resource is justified both in terms of attainment gains and sustaining teachers' commitment.
• As a resource, the cost of the additional staffing is significant, and it will be important to ensure that its value is fully exploited. The classroom team should be seen as one that capitalises on the full potential of all
  its members, and also functions as a team. Staff development, and
  on-going exploration of good practice and enhanced roles and contributions should be priorities.
• There is a need to debate curriculum balance in the early stages of the primary school, and consider whether play and self-directed learning opportunities are under-represented.

References

FRASER, H. (1998) Early Intervention: the key issues for research, Interchange 50, Edinburgh: SOEID

McMILLAN, G. and LESLIE, M. (1998) The Early Intervention Handbook, Edinburgh:The City of Edinburgh Council.

NISBET, J. and WATT, J. (1994) Educational Disadvantage in Scotland.
A 1990's Perspective, Edinburgh: Scottish Community Education Council.

SCOTTISH OFFICE EDUCATION AND INDUSTRY DEPARTMENT (1998) The Early Intervention Programme. Raising Standards in Literacy and Numeracy, Edinburgh: The Stationery Office.

SMITH, T. and NOBLE , M. (1995) Education Divides: Poverty and schooling in the 1990s, London: Child Poverty Action Group.

For further information about the research and copies of the full report, please contact Helen Fraser at the Faculty of Education, University of Edinburgh, Holyrood Road, Edinburgh, EH8 8AQ. Tel: 0131 650 1000 / email: helen.fraser@ed.ac.uk

Why Interchange?

Research cannot make the decisions for policy makers and others concerned with improving the quality of education in our schools and colleges. Nor can it by itself bring about change. However, it can create a better basis for decisions, by providing information and explanation about educational practice and by clarifying and challenging ideas and assumptions.

It is important that every opportunity should be taken to communicate research findings, both inside and outside the Scottish Executive Education Department (SEED). Moreover, if research is to have the greatest possible impact on policy and practice, the findings need to be presented in an accessible, interesting and attractive form to policy makers, teachers, lecturers, parents and employers.

Interchange aims to further improve the Education Department Research Unit's dissemination of the findings of research funded by SEED. We hope you will find that Interchange is long enough to give the flavour of the complexities, subtleties and limitations of a research study but concise enough to give a good feeling for the findings and in some cases to encourage you to obtain the full report.

The views expressed in this Interchange are those of the author(s) and do not necessarily reflect those of the Scottish Executive or any other organisation(s) by whom the author(s) is/are employed.

Interchange may be photocopied for use within your own institution.

A limited number of additional copies can be obtained by writing to the SEED Research Unit Dissemination Officer at the Scottish Executive Education Department, Victoria Quay, Edinburgh EH6 6QQ. File copies for electronic downloading are available on the the Scottish Executive server (http://www.scotland.gov.uk/edru.asp).

The Interchange series

1. Homework Policy and Practice
2. School to Higher Education: Bridging the Gap
3. Teaching, Learning and Assessment in the National Certificate
4. Developing School Managers
5. Transition from School to Adulthood of Young People with Recorded Special Educational Needs
6. Discipline in Scottish Schools
7. Training the Trainers' Programmes: Effective Management and Monitoring
8. Introduction of the New Further Education College Council System
9. Young People's Experience of National Certificate Modules
10. Costs and Benefits of Adult Basic Education
11. Performance Indicators and Examination Results
12. An Evaluation of the Advanced Courses Development Programme
13. Staying the Course
14. A Study of Probationer Teachers
15. Making School Boards Work
16. Professional Development through Research
17. Students' Views on SWAP
18. Specific Learning Difficulties: Policy, Practice and Provision
19. Foreign Languages in Primary Schools: the National Pilot Projects in Scotland
20. Towards More School Based Training?
21. Patterns of Attainment in Standard Grade Mathematics 3-6
22. Patterns of Attainment in Standard Grade English 3-6
23. Implementing 5-14: a Progress Report
24. Education-Business Links: Patterns of Partnership
25. Foreign Languages for Vocational Purposes in Further and Higher Education
26. School for Skills
27. Effective Support for Learning: Themes from the RAISE Project
28. Marketing Means Business
29. Adult Education: Participation, Guidance and Progression
30. Studies of Differentiation Practices in Primary and Secondary Schools
31. Health Education: What Do Young People Want to Know?
32. Social and Educational Services for Children Under Five
33. Issues in Teachers' Continuing Professional Development
34. Primary Teachers' Understanding of Concepts in Science and Technology
35. Putting 5-14 in Place: An Overview of the Methods and Findings of the Evaluation
1991-95
36. Implementing 5-14 in Primary Schools
37. Implementing 5-14 in Secondary Schools
38. Provision for Special Educational Needs
39. Methods of Teaching Reading
40. Criteria for Opening Records of Needs
41. Guidance in Secondary Schools
42. Higher Grade Examination Performance (1987-1994)
43. Pupils with Special Educational Needs: The Role of Speech & Language Therapists
44. Evaluation of the National Record of Achievement in Scotland
45. Part-time Higher Education in Scotland
46. Educational Provision for Children with Autism in Scotland
47. Exclusions and In-school Alternatives
48. Devolved School Management
49. Implementing 5-14 in Primary and Secondary Schools: Steady Development?
50. Early Intervention: Key Issues from Research
51. Youth Work with Vulnerable Young People
52. Evaluation of the Technical and Vocational Education Initiative (TVEI) Extension
53. Cost of Pre-School Education Provision
54. Managing Change in Small Primary Schools
55. Baseline Assessment Literature Review and Pre-School Record Keeping in Scotland
56. Schools and Community Education for the Learning Age
57. Accelerating Reading Attainment: The Effectiveness of Synthetic Phonics
58. Teachers' ICT Skills and Knowledge Needs
59. Foreign Languages in the Upper Secondary School: A Study of the Causes of Decline
60. Practices and Interactions in the Primary Classroom
61. Providing Drug Education to Meet Young People's Needs
62. The Attainments of Pupils Receiving Gaelic-medium Primary Education in Scotland
63. The Impact of Information and Communications Technology Initiatives
64. Parents' Demand for Childcare in Scotland
65. Uptake and Perceptions of Early-years' Qualifications
66. Developments in Inclusive Schooling
67. Raising Attainment of Pupils in Special Schools
68. All Day Provision for 3 and 4 Year Olds
69. Pre-School Educational Provision in Rural Areas
70. Gender and Pupil Performance
71. Early Intervention in Literacy and Numeracy

Further information

If you have views on Interchange and/or wish to find out

more about SEED's research programme, contact
the SEED Research Unit, The Scottish Executive Education Department,
Room 1B Dockside, Victoria Quay, Edinburgh EH6 6QQ