The Scottish Government

« Previous | Contents | Next »

Listen

9. Results

9.1. Univariate Shrinkage Methods

9.1.1. Observed Indicator Data

Table 9.1 shows the correlations between the original SIMD 2004 methodology and alternatives based on using the same univariate shrinkage method, but applied using different shrinkage areas. Included in these comparisons is the no shrinkage alternative, which can be thought of as shrinkage using data zones as their own shrinkage area. The other shrinkage area options shown are the use of a single (national) shrinkage area, the use of groups of data zones into intermediate geographies ( IGs) below local authority ( LA) level, and the use of groups of data zones defined by the SE Urban/Rural indicator (rurality groups).

All correlations are high, indicating that none of the alternatives has an excessive influence on the domain or multiple deprivation scores or rankings. In general, shrinkage towards rurality group averages produces deprivation measures most strongly correlated with the original algorithm, and use of an intermediate geography produces measures that are least strongly correlated to the original, though these correlations are still very high.

Correlations for the SIMD score are greater than for the SIMD ranks, though for the domain scores, this is reversed. Altering the shrinkage areas used has the largest effect on the health domain score, in the construction of which 79 separate indicator variables are shrunk; for the education domain, 19 raw indicator variables undergo shrinkage, and there is less impact on these domain measures. The impact on the overall SIMD scores and ranks are less since these two domains have a combined weight in the final calculation of only 28%; calculations for domain scores making up the remaining 72% remain unchanged in these modified methodologies.

Table 9.1Correlations of SIMD, health domain and education domain scores and ranks between original algorithm and methods using alternative shrinkage areas

Table 9.2Number of data zones in each rurality group, and number (%) of data zones amongst the 15% most deprived data zones nationally, according to the SIMD calculated using the original method and alternative univariate shrinkage options

Table 9.2 shows for groups of data zones defined by the SE Urban/Rural indicator, the number of data zones ranked within the most deprived 15% nationally, under the original SIMD algorithm and under each univariate shrinkage alternative. The use of a single national shrinkage unit, or no shrinkage at all both result in slightly fewer data zones in large urban areas being classified as highly deprived, and more in other urban areas and accessible small towns. This could be a result of these methods not over-shrinking isolated pockets of deprivation. The use of IGs as shrinkage units appears to give very similar results to the original algorithm, with some trade-off between the two urban groups. The use of rurality groups themselves as shrinkage units has the effect of classifying more urban data zones as deprived at the expense of data zones in small towns and rural areas. This could be a result of over-shrinkage as previously discussed, since the trend for lower prevalence of extreme deprivation in more rural areas will lead to greater shrinkage in those rural areas where deprivation is observed to be extreme.

The same summaries are shown for each LA in Table A.1 ( Appendix A). Glasgow City has by far the highest proportion of such data zones, at 54%, though under each alternative method, this proportion is reduced by 1-2%. Similarly, the proportion of very deprived data zones in North Lanarkshire is reduced under every alternative method. The opposite is seen in Aberdeen City, East Renfrewshire, Angus, Perth & Kinross and Aberdeenshire Council areas, with an increase in the number of deprived data zones under each univariate shrinkage alternative.

The differences between the numbers of data zones classified as highly deprived using the various different univariate approaches to shrinkage are, in general, small. This does not imply that these differences are insignificant, since there is a tendency for the alternative approaches to result in more data zones being classified as highly deprived amongst the otherwise less deprived LAs.

Ranking LAs as in Table A.1, by the proportion of data zones classified as highly deprived under the original algorithm, those LAs in the lower half of the distribution (Stirling - Shetland) contain 53 deprived data zones under the original method, and 64, 65, 63 or 59 under the alternatives of no, national, IG or rural group shrinkage respectively. Under all four alternative methods in these 16 LAs, only once is a LA determined to have fewer deprived data zones (West Lothian, under the no shrinkage option). For comparison, in the more deprived half of the distribution (Glasgow City - Aberdeen City), amongst the 64 LA-by-shrinkage combinations, 21 result in more deprived data zones, 21 in fewer deprived data zones and 22 result in no difference, compared to the original method.

Figure 9.1Median and 5-95% range of SIMD ranks for each data zone, ordered by original SIMD rank

9.1.2. Simulated Indicator Datasets

Figure 9.1 shows the median, 5th and 95th percentiles of simulated SIMD ranks using the original algorithm, plotted against the observed SIMD rank for each data zone. Note that the median simulated SIMD rank for each data zone is not in general the same as the observed SIMD rank, though these are very highly correlated (99.98%). The variation in ranks is less than often seen in the literature regarding rank uncertainty estimates. These applications generally deal with smaller numbers of units of analysis, ranked according to fewer indicators than being used here. The apparent stability of SIMD ranks may reflect the use of a large number of individual indicator variables, so that the variance of the composite multiple deprivation index is small. Also, since a large number of data zones are being characterised, and these data zones are very heterogeneous with respect to the deprivation indicators being used, it is relatively easy to discriminate between data zones with high and low levels of deprivation.

The fact that the variance of rankings near to the extremes of the deprivation scale converge towards zero indicates that the algorithm as a whole can reliably identify these extreme data zones. The way the SIMD is constructed requires that a data zone must show high levels of deprivation on a number of domains in order to be labeled as extremely deprived. Furthermore, each domain is constructed from a number of indicator variables which are positively correlated nationally, so such data zones will tend to have high levels of deprivation on many of these indicators. It is therefore unlikely, even allowing for random variation of the indicators, that the most deprived data zones will be assigned SIMD ranks far from its observed value.

Figure 9.2 presents the standard deviation ( SD) of simulated SIMD rankings for each data zone, plotted against the original SIMD rank. Rank variability is least at the extremes of the distribution, but in general, is seen to be less in the more deprived half of the distribution. This is in agreement with the impression given by Figure 9.1, and reflects a favourable quality of the SIMD algorithm, since a major purpose of the SIMD is to identify the most deprived data zones, rather than the least deprived.

Figure 9.2Standard deviation of simulated SIMD ranks for each data zone, ordered by original SIMD rank, with estimated association between rank variation and original SIMD rank

In order to evaluate alternatives to the original SIMD method, we can calculate, in each data zone, the SD of the SIMD ranks across the simulations. Expressing these SDs relative to those obtained using the original algorithm gives a measure of variation of the SIMD ranks provided by each method, with values >1 indicating greater variability compared to the original algorithm. In general, the different methods for calculating SIMD ranks do not result in uniformly greater or lesser variability compared to the original algorithm.

Plotting these ratios against the original observed SIMD ranks shows the variation of SIMD ranks produced by each shrinkage alternative, relative to the original method, over the full range of multiple deprivation. Figure 9.3 shows these estimates for the alternative shrinkage methods of no shrinkage, shrinkage to national average values, shrinkage to IG averages and shrinkage to average values in groups of data zones defined by the SE Urban/Rural Indicator.

No shrinkage and shrinkage to IG averages produce SIMD ranks that show greater variability than the original algorithm over the whole deprivation range. Ratios are least within the third quartile of ranks and increase sharply at the least deprived end of the distribution. Towards the more deprived end of the distribution, SD ratios are relatively constant, with less than 4% additional variation compared to the original SIMD ranks. Shrinkage to national and rural group averages produce similar patterns, though levels of variability in the third quartile are similar to or less than with the original algorithm.

Previously, we reported the numbers of data zones classified as severely deprived (defined as lying within the most deprived 15% of data zones nationally) in subgroups of data zones defined by the SE Urban/Rural Indicator and by LAs. Using the simulation results, it is possible to estimate the probability that each data zone lies within the most deprived 15%, as the number of simulations in which this is the case, divided by the total number of simulations (here, 1,000). Some data zones will have an estimated probability of 1 of being severely deprived, and for some the probability will be zero, though for a number of data zones around the 15% threshold, the probability will be strictly between 0 and 1.

Figure 9.3Alternative shrinkage methods: estimated association (with 95% CI) between standard deviation ( SD) of simulated SIMD ranks (relative to SD of ranks using original algorithm) and original SIMD rank

The sum of these probabilities over subgroups of data zones relative to the total number of data zones in the subgroup, gives a summary measure akin to the percentage of data zones in a subgroup that are severely deprived. Those data zones certainly within the 15% threshold are counted fully, and those certainly outwith the threshold are not counted at all, but those data zones near to the cut-off are included with weights equal to the likelihood that they truly are severely deprived. Data zones that are, on average, within the 15% most deprived are given weights greater than ½, and those occasionally found to have severe deprivation are given weights less than ½.

If the SIMD ranks were calculated with estimates of uncertainty attached, it would be possible to consider the share of the most deprived areas or allocate funding based on these probabilities, rather than the strict inclusion method generally used. LAs with data zones just outside the chosen threshold would have these areas included in their allocation calculation, and those data zones just within the threshold would be given less weight than the most deprived areas.

Table 9.3Number of data zones in each rurality group, and probability-weighted number (%) of data zones amongst the 15% most deprived data zones nationally, according to the SIMD calculated using the original method and alternative univariate shrinkage options

Table 9.3 shows the probability-weighted numbers of data zones classified as severely deprived in subgroups of data zones defined by the SE Urban/Rural Indicator. Overall, patterns of deprivation are very similar to those found previously (Table 9.2). No shrinkage and shrinkage towards a single national average both result in fewer severely deprived data zones from large urban areas, and slightly more in all other types of area, supporting the view that these methods do not mask isolated areas of extreme deprivation. Shrinkage to rural group averages has the opposite effect, again supporting the idea that this approach accentuates this over shrinkage. Using IG averages for shrinkage has no clear pattern of effects on the distribution of data zones classified as suffering high levels of multiple deprivation. The same summaries for LAs are shown in Table B.1 ( Appendix B).

Whereas using a strict threshold to define data zones amongst the most deprived 15% nationally identifies 975 such areas, the number of data zones with a non-zero probability of lying within the most deprived 15% (as defined by the simulated indicator datasets method used here) is generally much larger. In this way, for example, the distribution of a larger number of data zones contribute towards decisions based on the locations of the most deprived data zones. For the shrinkage alternative considered here, the numbers of data zones with non-zero probabilities of being severely deprived are 1,354 for the original method, 1,355 for both the no shrinkage and rurality group options, 1,360 for the IG shrinkage method and 1,362 when using a single, national shrinkage area.

9.2. Multivariate Shrinkage Methods

9.2.1. Multivariate Analogue to Univariate Method

The method of Longford ¹³, a matrix-based analogue of the univariate shrinkage method:

is appealing for the apparent simplicity of the above formula. However, in the original SIMD algorithm, there are a number of data issues that mean that the basic univariate shrinkage formula cannot be applied directly.

A number of raw indicator variables have zero denominators in some data zones. For example, the data used to construct the CMF and CIF indicators include the numbers of males and females within 5-year age bands up to age 89, or over 90 years of age. At the extremes of these age ranges, some data zones will include no individuals. Consequently, neither the observed rate, nor the variance of its logit can be calculated, and the data zone cannot be included in the shrinkage formula.

Table 9.4Correlations between shrunken indicators from the Health domain derived using random effects models and those derived using the original shrinkage method

In the univariate case this is not a major problem. Such data zones are removed prior to shrinkage of rates, after which a "shrunken" value of zero is inserted. This decision is unlikely to adversely affect the results in any way, since with a denominator of zero, the particular age-sex group can contribute nothing to the expected numbers of events used to calculate age- and sex-standardised indicators. For single Binomial indicator variables, this process of shrinkage followed by insertion of zeros at data zones with no relevant population has little impact, since such data zones, having no "at risk" individuals, are observed to have no deprivation according to that indicator.

In the matrix-based multivariate case, such a procedure is not possible, since the removal of data for a single indicator results in the loss of all data for that data zone. There is no immediately apparent analogue to the procedure used in the univariate setting that could be applied to modify the multivariate approach, and this method was not pursued further.

9.2.2. Random Effects Models

Table 9.4 shows the correlations between the shrunken estimates of the five indicator variables of alcohol-related discharges, drug-related discharges, emergency admissions, prescriptions for anxiety, depression or psychosis and low birthweight singleton births obtained using the original method and those obtained using random effects models. Shrunken estimates were obtained from random effects models for each indicator variable individually, and from a multivariate model of all five indicators simultaneously.

Correlations are higher when each indicator is shrunk independently, as might be expected since under these models each indicator value is being shrunk towards its LA average, whereas under the multivariate model, the movement of each indicator is affected by the values of every other indicator in the same data zone. Correlations under both models for the hospital episode data are very high, in excess of 99%; for the prescriptions indicator this is slightly reduced, possibly reflecting a slight deviation of the data from the assumed distribution, bearing in mind that the values of this indicator are themselves estimated.

The low birthweight indicator shows much lower correlation between the original method and both random effects models. It is probable that this indicates a more severe departure from the Poisson distribution (and from the Binomial distribution). However, since this is a single indicator amongst seven used in the health domain, the effects on the final domain score and SIMD are likely to be small. Nevertheless, a more detailed consideration of this indicator would be required if such a method were to be implemented in practice.

Figure 9.5Relationships between shrunken values of prescriptions for anxiety, depression or psychosis and live singleton births of low birthweight derived using multivariate random effects model and those derived using the original shrinkage method

The relationships between the shrunken estimates for the hospital episode indicators obtained by the original method and those obtained using random effects models are almost perfectly linear. For prescriptions for anxiety, depression or psychosis and live singleton births of low birthweight, the relationships between the original method and the multivariate shrinkage method are shown in Figure 9.4. The poor association observed for the low birthweight indicator is clear.

Whilst this method can be applied to small groups of indicator variables, it is not feasible to fit corresponding models to larger groups of variables. For example, the mortality data used to construct the SMF indicator consist of the numbers of deaths in 38 age-sex groups for each data zone over a four-year period. Fitting an analogous model to that used for the other health domain indicator variables would involve 741 variance-covariance terms at both the between- LA and between-data zone levels.

An alternative parameterisation was used in an attempt to obtain useful shrunken estimates; it was assumed that there were sex-specific quadratic associations between mortality rates and age, with the terms for age and age2 in males and females being random effects over LAs and data zones. In practice, however, this model failed to converge, and was not explored further.

« Previous | Contents | Next »