The Scottish Government

« Previous | Contents | Next »

Listen

The Independent Evaluation of 'Starting Well' Final Report

Appendix IV: Contextual comparisons of study areas

IV.1 Introduction

This chapter, as part of Aim 2.3c, does not comment directly on the operation or effectiveness of the intervention, but aims to describe dimensions of area context that might be hypothesised to influence health-related outcomes over-and-above the important individual-level characteristics identified in section 2 of this report. The dimensions considered (derived from census, routine and cohort data) are not intended to be exhaustive but illustrative and include: basic demography; material and built environment; health; and social context. By making descriptive contextual comparison between intervention and comparison-areas, we aim to complement the impact study findings and to explore the potential for separating out individual- and area-level effects more formally using multi-level analyses More broadly, we aim to illustrate the kinds of area-level factors that may indirectly influence the effectiveness of the intervention.

IV.2 Methods

This section describes the process of defining study areas and the various measures used.

IV.2.1 Area definitions

Problems of definition: Any attempt to describe area context must first define 'area'. This is far from straightforward as the operational definitions of service-providers rarely match the administrative boundaries at which contextual data are available (e.g. council wards, postcode sectors, census units). The task is further complicated by the fact that administrative boundaries change over time. This section describes a functional definition of 'area' based on combinations of whole postcode sectors, the boundaries of which were fixed in 2001 ⁴⁴. The use of postcode sectors to define area fulfils two important criteria: first, it permits the collection and comparison of aggregated (e.g. 2001 Census) data; and second, approximates to the geographies in which services were offered.

General orientation: figure IV.1 (below) shows approximations of study areas relative to the administrative boundaries of Greater Glasgow NHS Board and Glasgow City Council. Note that the geographic size of each area is not proportional to its resident population (see table IV.1), however, the 'northern' comparison is roughly as populous as the 'eastern' and 'southern' intervention areas combined.

Intervention areas: At the outset of the project, the 'eastern' intervention area was defined to consist of four postcode sectors (G33 3 and 5; G34 0 and 9, see figure IV.2) covering the 'communities' of Cranhill, Ruchazie, Craigend, Garthamlock, Easterhouse and Gartloch. Since the initial proposal, the boundaries of these postcode sectors have changed and now include G69 8. This postcode sector is included in figures IV.1-IV.3 in order to reflect both geographic contiguity with G34 0 and 9 and the fact that the intervention was offered here, however, its very low population count means that fewer census and routine statistics are available and it is therefore not included in the analyses below. The 'southern' project area was also operationally defined by geography but not in terms of whole postcode sectors. We define it here to consist of the postcode sectors G5 9 and 0; G42 0, 7 and 8 (the 'communities' of Gorbals/Hutchesontown, Oatlands, Polmadie, Govanhill and north Toryglen), although the service was not offered to all residents in G42 8 (Govanhill) or G42 0 (Oatlands) ⁴⁵.

Figure IV.1: Study areas in relation to local administrative units

Figure IV.2: Postcode sectors of areas analysed in this section

Comparison areas: in contrast to the above, comparison 'areas' were defined as part of the independent evaluation impact study and related to a population of births visited over a specified period by GP-attached health visiting teams ⁴⁶. The teams visited families registered at specific surgeries and consequently there was a strong but not defining geographic focus to the caseload. In order to permit comparative analysis of census and routine data and summarised cohort data (e.g. on social context - see below), we have defined the comparison area as contiguous postcode sectors in which the clear majority of survey respondents lived at the eighteen-month assessment. Eight-one percent (99/122) of respondents lived in the following postcode sectors: G21 1, 3 and 4; G22 5, 6 and 7; G64 1 ⁴⁷. Other postcode sectors with large populations but that provided few survey respondents ⁴⁸ were not included in analysis due to the possibility that they would skew comparisons of census data whilst having relatively little impact on cohort measures. Figure IV.2 shows each postcode sector used in analysis, key properties of which are described below in table IV.1.

Table IV.1: Persons and households by postcode sector and study area (2001 Census data); resident cohort members at 18-months by study area(cohort data).

The table demonstrates that whilst the total number of persons and households in intervention and comparison areas are roughly comparable, around twice as many intervention cohort members were available for analysis. This latter fact reflects the selection of a sub-set of comparison postcode sectors and the lower opt-in rates described in section 2.

IV.2.2 Dimensions of context and associated data

In selecting relevant dimensions of context, our general approach was to gather data that was available at the appropriate level (postcode sector or smaller) and that either illustrated basic and relevant aspects of demography and population health, or that could be hypothesised to influence child health and well-being directly or indirectly (e.g. by impacting on caregivers). The following domains were of interest:

Aspects of general demography: One simple contextual difference may be the proportion of pre-school children (aged 0-4) in the local populations; this was established at the postcode sector level from 2001 Census table UV04 ⁴⁹ . In addition, information on population mobility was collected as indexed by the proportion of people not resident at their current address one year previous to April 2001 (table UV52). Whilst this has no direct effect on child health, it may affect parental 'stake' in the area and the ability to maintain the kind of effective social networks that could predict better health (see below). Census data on ethnic group was not available at postcode sector level due to disclosure concerns. Finally information on household tenure (specifically the proportion of social rented housing) was derived from CAS table UV64.

Material and built environment: Low socio-economic position and associated poverty has the potential to impact on child health and development by adversely affecting family processes and the resources available to caregivers. We indexed this at the postcode sector level from CAS tables UV76 (the proportion of adults classified as National Statistics Socio-Economic Class 8: never worked and unemployed) and CAS031 (% workless parents aged 16-74 and with dependent children). Adverse living conditions relating to the built environment also have the potential to affect health. We gathered data on two relevant domains: overcrowding, as indexed by the proportion of households with more than one person per room (CAS052); and high-rise living (the percentage of residents living on the 5 ^th floor of a building or higher: CAS020).

Health: Two postcode sector level child-relevant datasets (both relating to 2001) were obtained from Scottish Neighbourhood Statistics ( http://www.sns.gov.uk). The first indexed the rate/1000 births of low birthweight children (<2500g); the second the rate/100000 population of children (aged 0-14) admitted to hospital due to accidental injury. Finally, an indication of the self-reported health of resident women of childbearing age (16-49) was derived from CAS table 025.

Social context: In contrast to the above variables which were all derived from either the 2001 Census or routinely available medical data, social context variables were constructed from the aggregated responses of impact study participants. Proceeding from reviews relating area-level social dynamics to a number of dimensions of child health and well-being (see e.g. Sampson, Morenoff & Gannon-Rowley, 2002), a range of measures were included in the 18-month survey:

• 'Global' social capital. The sum of four scored items derived from the ONS Social Capital Question Bank (ONS, 2002): 'generally speaking, most people can be trusted'; 'generally speaking, people help each other out'; 'by working together, people here can influence decisions that affect the area'; and 'this area has a good reputation'. Each item was scored 0-4 on a five-point Likert-type strength of agreement response set ('strongly disagree' = 0 to 'strongly agree' = 4, with 'not sure' scored as '2'), resulting in a measure with a possible range of 0-16.
• Perceptions of child-relevant area safety. A measure developed 'in-house' and derived from four items relating to: the general safety of the area for play; safety for play after dark; safety of local playgrounds and parks; and local roads. Respondents rated each item on a four-point scale ranging from 'very unsafe' (scoring 0) to 'very safe' (scoring 3). An overall score was constructed by summing responses (possible scoring range 0-12).
• Informal social control. From Sampson, Raudenbush & Earls (1997), the perceived likelihood of neighbours intervening in three situations involving children's anti-social behaviour (truanting and 'hanging around'; spraying graffiti; showing disrespect to an adult). An overall score (range 0-12) was constructed from combining responses to a five-point likelihood rating ('very unlikely' = 0 to 'very likely' agree' = 4, with 'not sure' scored as '2').
• Parenting-relevant reciprocity. Another 'in-house' measure expressing the sum of six items rating the frequency with which: respondents look after neighbours' children; have neighbour's children round to play; both these favours are returned by neighbours (two separate items); respondents talk to neighbours about their children; and exchange information or advice about parenting. All items were rated on a four point scale ('not at all' = 0 to 'very often' = 3), and an overall score with a possible range 0-18 constructed.
• The extent to which supporting parents is a local priority (measure developed 'in-house'). Using the same response set as for the social capital items, the extent to which respondents agreed with the six statements: 'people around here look out for each other's children'; 'this is a good area to bring children up in'; 'most people around here can be trusted with children'; 'if a parent with a young child was standing on a bus, people would give up their seats'; 'people around here hold shop doors open for parents with pushchairs; and 'bringing up kids well is a priority for people in this area'. An overall score (possible range 0-24) was constructed.

IV.3 Comparisons

IV.3.1 Highest level of aggregation: intervention and comparison areas

Table IV.2 compares aggregated census and routine data measures across intervention and comparison areas. Beginning with basic demography, the two areas are very similar in terms of the proportions of children aged 0-5 but the combined intervention areas have significantly more mobile populations and significantly fewer social rented properties. This latter statistic is likely to reflect a high percentage of private rented properties in the southern sub-area of Govanhill. In absolute terms, however, the proportion of social rented housing is high.

Table IV.2: comparisons of intervention and comparison areas on selected 2001 Census and routine data

Notes:

^a in this column, (p) refers to 'persons' and (h) refers to 'households'

^b in terms of values for constituent postcode sectors

^c statistic for comparison of proportions

^d significance tests were not possible due to the unavailability of numerator and denominator informationMoving onto material and built context, the intervention area has a non-significantly higher proportion of people classified as NS-SEC Class 8 (never worked and unemployed) and contains a higher proportion of overcrowded households high-rise housing and a non-significantly higher proportion of workless parents ⁵⁰.

Finally, regarding health indicators, the intervention areas have higher rates of low birthweight children and hospital admissions due to accidents. In contrast, the self-reported health of intervention women was significantly better than in comparison areas.

Data relating to area-level social context is displayed in table IV.3. Comparison area families score slightly more highly on every measure, although only significantly so in relation to levels of informal social control.

Table IV.3: comparison of social context variables by main study area

Note: ^a in all cases, the p-value relates to an independent t-test (details available on request)

It is important to note, with all the comparisons made in tables IV.2. and IV.3, that even very statistically significant group differences are not the product of very marked or obvious differences in means and proportions. The nature and meaningfulness of these results are discussed in the final sub-section below.

IV.3.2 Lower levels of aggregation: 'east' and 'south'; postcode sector

As with the impact study, the primary comparative focus of this section has so far been at the level of intervention versus comparison area(s). We now turn briefly to an illustration of the kinds of variation observed at lower levels of aggregation.

a) 'East', 'south' and 'north': one obvious source of comparison is within-intervention, that is between eastern and southern areas. Tables IV.4a and IV.4b, respectively, compare the east, south and north (comparison) areas on two dimensions of social context and two census-derived variables. It can be seen that the aggregated intervention measures described above mask variation at this lower area level. Specifically, with regard to levels of informal social control (table IV.4a), analysis of variance confirmed a significant group effect, but post-hoc (Bonferroni) tests revealed no significant difference between eastern and northern area means; significant differences existed between southern and eastern and between southern and northern means only. Similarly, a significant group effect was obtained for the 'prioritising parenting' measure but the only significant difference in group means was between eastern and southern project areas.

Essentially the same point can be made regarding variation in census data (table IV.4b): the higher level of residential mobility observed in the intervention area (see table IV.2) is largely the product of high mobility in the southern area; the eastern figure is actually lower than that for the comparison area. Similarly, overcrowded households are substantially more common in the eastern as opposed to the southern project areas.

Table IV.4a. Comparing selected social context measures: east vs. south vs. north

Note: ^a p-values relate to one-way analysis of variance (ANOVA).

Table IV.4b. Comparing selected census-derived measures: east vs. south vs. north

It might be reasonable to expect that a further 'community'-level analysis of the comparison area would yield similar variation.

b) Postcode sector level: Even more variation exists at smaller area levels and whilst postcode sector boundaries do not generally map well onto residents' definitions of 'community/neighbourhood', they serve to illustrate this point. Figure IV.3 depicts the variation across study postcode sectors of the percentage of resident parents (aged 16-74 and with dependent children) who are workless. This statistic is sensitive, amongst other things, to the proportion of economically inactive lone-parent families, however, two points can be made. First, as described at the highest levels of aggregation (see table 4.2), the figures are generally high ⁵¹ confirming the disadvantaged nature of the study communities. Second, there is at least twenty percentage points of variation within a particular study area and an overall range of forty-four percent (23.3 - 67.3%). The extent of this variation points to real micro-level differences in the nature of the communities served in this study.

Figure IV.3. Comparison of % workless parents by postcode sector

IV.4 Discussion and conclusions

The purpose of this section was to describe and compare the evaluation areas on dimensions of context that are potentially relevant to child health and development. Naturally, the most pertinent contextual difference between intervention and comparison areas was the service context and the fact that an enhanced home-visiting service was being offered in one set of areas and not the other. However, we have attempted to comment on other contextual domains including measures that reflect aspects of the material and social environment. Despite the various approximations involved in defining 'area' and the manifestly limited datasets available, at least three key observations may be made.

Firstly, statistics relating to material resources (e.g. parental worklessness) are stark and confirm that the areas under study are highly disadvantaged regardless of level of aggregation. The scale of this disadvantage combined with the well-described relationships between poverty and ill-health illustrate the magnitude of the task faced by 'Starting Well' and other service-providers in promoting positive changes in population health.

Secondly, whilst at the highest level of aggregation/comparison, (intervention versus comparison area), a number of highly statistically significant group differences in census-derived measures were found, the precise meaning of these differences remains obscure. Partly this is due to the fact that related findings are not always in the same direction ⁵² but a more technical reason relates to statistical power; comparisons of census variables at area level involve very large populations of people and households which means that modest group differences translate easily into very significant statistical differences. Whilst this means that the observed differences are very unlikely to have occurred by chance, it does not mean that they are meaningful differences with respect to their impact on the health and well-being of residents. Given these problems of interpretation, it is difficult to reach firm conclusions about the pertinence of differences observed at this level aggregation. Additionally, whilst findings relating to cohort-derived social context variables are at least consistent (intervention areas seem less well-endowed than comparison areas), differences in absolute means are again typically very small. Taken together, therefore, these findings of 'no reliable area differences' have very little capacity to shed light on the conclusions of the impact study.

Thirdly, considerable variation is found for most variables at lower levels of aggregation. To an extent, this is to be expected (an aggregated statistic, by definition, averages out variation in constituent units), however, these smaller areas may be intrinsically more appropriate for considering the operation of some dimensions of context, for example, social dynamics that are the product of localised face-to-face interactions. Additionally, the amount of variation, both between intervention areas and individual postcode sectors points to the utility of carrying out at least two further sets of analyses. At its most simple, the first might involve repeating the individual-level regression analyses in section 2 but with area of residence ('east' or 'south' with 'north' as a reference category) as an additional predictor. This would help detect substantive differences in outcome between intervention areas after controlling for individual-level factors. Secondly, it may be possible to assess more formally the relative contribution of people and place to health outcomes using multi-level analysis. Candidates for appropriate 'levels' may be postcode sectors or (better) an attempt to define 'natural communities' that, for example, respected existing community identities but could be described as aggregates of smaller data-rich units (e.g. census output areas, Scottish Neighbourhood Statistics 'data zones').

In sum, this section has shown both the difficulties of describing context meaningfully at high levels of aggregation but also the potential perhaps for more sophisticated lower-level analyses that may help tease out the relative contribution of individual and area-level factors to health-related outcomes. More extended individual-level regression analyses remain our immediate priority, however, if future opportunities can be found to explore these possibilities, we may not only explain more of the variance in outcomes but also gain a more informed sense of the kinds of emergent community-level factors that constrain and facilitate the operation and effectiveness of Starting Well.

« Previous | Contents | Next »