« Previous | Contents | Next »
Listen
Growing Support
3. The nature and evidence of effectiveness
It is apparent from the preceding sections and other evidence (Alderson et al 1995, Macdonald 1997, Newman and Roberts 1997) that the early years are critical for healthy future development, and that some families living in disadvantaged situations will need support from services other than those universally available in the community. In providing support for these families it is important to ascertain the effectiveness of the interventions, as it has been widely recognised that 'meaning well and doing good' are not the same thing, and good intentions may well have harmful affects (McNeish and Newman 1996:55). However, ascertaining what works is not always easy to achieve. In the first instance, there is a paucity of information on the quality and effectiveness of different types of support, particularly in Scotland (Henderson 1999). Secondly, the meaning of effectiveness is complex: establishing that something clearly works is less straightforward than appears at first sight. This review intends to provide evidence of the effectiveness of different kinds of support for vulnerable families who have children aged 0-3 years in the UK and where appropriate in other countries as well. In order to carry out this task it is important to explore briefly what is meant by the term 'effectiveness' (see Hill 1999 for a more full discussion).
In assessing effectiveness the achievement of desired goals is a primary consideration. Aims of intervention are often multiple or vague, however. It may be necessary to weigh up emotional, social and educational consequences, which may not all go in the same direction. Generally early years interventions are judged according to the impact on the child, but consequences for mothers, fathers and others can all be relevant. If the primary purpose is to assist parents (e.g. to work, have a break) then it may be sufficient that the children are not adversely affected, though usually it would be hoped that they have positive benefits.
Normally it is hoped that interventions achieve positive change and that the change(s) would not otherwise have happened. Thus research needs to demonstrate not only that change actually occurred (or deterioration was stemmed) but also that this change is a direct result of the intervention. This can be difficult to establish, since there is much evidence that children tend to progress except in extreme circumstances and may recover from difficulties without formal help (Rachman and Wilson 1980). In order to establish clearly that a change has resulted from an intervention, the ideal research method is a randomised control trial or RCT, where children or families are allocated to either a 'treatment' or 'non-treatment' group at random. Comparisons are then made between those who received the service and those who did not. Since the random allocation should mean that both groups started with the same range of initial characteristics, any difference found (positive or negative) should be due to the service. Few studies use this approach for a combination of practical and ethical reasons (Fuller 1996).
Quasi-experimental designs can be used when random allocation of participants is not possible. A naturally occurring sample is taken of those who receive an intervention and compared with others who either receive a different intervention or none at all. The initial characteristics of both samples need to be very similar, as otherwise different starting points may account for different outcomes. Pre-tests and post-tests are carried out for both groups in order to establish a comparison of outcomes and thereby assess the effectiveness of the intervention. Such quantitative comparisons can provide strong indications, but often it is necessary to supplement these with a range of qualitative data (Cheetham and Kazi 1998). Projects, services and centres often have multiple functions and these tend to change over time. Even with a positive evaluation from a RCT or quasi-experimental comparison, it may be hard to specify which aspect(s) or stages of development or a programme or project were responsible for the good results.
A large amount of evaluation carried out in the area of child welfare does not fall into the category of experimental design. Macdonald and Roberts (1995:13) suggest that findings from studies employing other methods are 'at best suggestive'. However, when a number of studies produce similar findings, more confidence can be expressed in the results. Methods that do not use experimental designs include cohort studies, surveys and studies based on the views of service users. Studies based around user views provide useful information about the perceptions of those using a service. Sometimes feedback from service users and/or professionals is the only data available and it can provide valuable insights into the consequences of a service (Gribben 1992). Care should be taken when using the results of satisfaction surveys as a means of assessing effectiveness. Vulnerable families will have a tendency to view the intervention and professionals positively, whether or not it actually makes much difference to their wellbeing.
Even when change has been shown to occur as a result of a particular form of intervention, it is sometimes a matter of interpretation whether the change is good. For example it has been found that nursery attendance tends to encourage a greater peer-orientation among children (Moore 1975, Rubin 1980). Some (adults) see this as desirable independence, others as undesirable lower regard for authority.
With respect to generally agreed improvements, questions arise about how long they last and whether the input of resources (costs) is justified by the outcomes. Perhaps a cheaper method would work just as well? It is also necessary to look at whether there have been any unintended effects. In some cases the unintended consequences will outweigh the benefits of the planned outcomes and therefore the intervention loses its effectiveness. Hill (1999: 11) gives the example of young offenders in institutions who make friends and learn skills which then assist in future law-breaking. In the early years, a short-term negative consequence of starting nursery early can be increased risk of infection.
In view of the frequent lack of experimental or quasi-experimental evidence, and the complexity of assessing effectiveness, this review has drawn on a wide range of evidence. We try to indicate the strength of support for any conclusion, depending on the rigour of evaluation or the number of studies that reach similar conclusions.
« Previous | Contents | Next »